Endoscopic Video

An exciting and challenging research topic I am working on is content understanding in videos from surgical procedures and interaction with medical videos. The rational behind is the fact that these days many surgeons record videos during endoscopic/microscopic surgeries and inspections and archive them for later use, since these videos contain highly important information that can be used later for:

  • revisit specific moments of the surgery for retrospective analysis
  • explanations to patients
  • discussions among surgeons
  • reviews of technical skills
  • teaching and training

While this valuable video data enables completely new use scenarios it also bears new challenges, such as managing the storage of a medical video archive – that increases on a daily basis – as well as automatic content analysis (e.g., phase recognition, instrument detection, surgical action classification, etc.) in order to make it efficiently accessible. A broad overview of this special field of medical multimedia (including its challenges and opportunities, as well as different aspects and specialities) can be found in the slides of our tutorial, presented at the ACM Multimedia 2018 conference in Seoul, South Korea (slides on the right).

Lange Nacht der Forschung 2018

In 2018 at the ‘Lange Nacht der Forschung 2018‘ (long research night) we could also demonstrate a few methods we developed in our research on medical multimedia systems. Our booth at Klagenfurt University showed ‘How neural networks can help surgeons to improve surgical quality‘ and it was a great success: several thousands of people came to see our research efforts and they were stunned by our interesting demos and hands-on experiences: real-time instruments detection, real-time smoke detection (with dry ice), and several laparoscopic operation simulator boxes where visitors could test their surgical skills. Here are a few pics from this great event: