LSC 2019 @ ICMR 2019

The second Lifelog Search Challenge (LSC 2019) will take place at ICMR 2019 in Ottawa, Canada in June 2019. Just like in 2018, LSC 2019 will be a highly interactive and entertaining workshop modelled on the successful Video Browser Showdown annual competition at the MMM conference. LSC is a participation workshop, which means that all participants will write and present a paper, as well as taking part in the live interactive search challenge. Consequently, the workshop will have two parts, (1) oral presentations, and (2) the search challenge.

More information can be found here:

8th VBS with Great Success

The 8th Video Browser Showdown (VBS) took place a week ago (on January 8th and 9th) at MMM2019 in Thessaloniki, and it was a great success. For the first time we used the V3C1 dataset (Part 1 of the Vimeo Creative Commons Collection), which consists of 7475 video files that amount for about 1000 hours of content. The six participating teams could solve all visual and textual Known-Item Search (KIS) tasks, as well as all Ad-Hoc Video Search (AVS) tasks within a short amount of time! The teams have clearly demonstrated that their sophisticated video retrieval systems are very powerful and allow fast and effective content-based search in videos. We look forward to the next VBS in January 2020 in Daejeon, Korea at MMM2020! More information here:

FWF Research Project SQUASH Approved

I am very happy that a new research project on Surgical Quality Assessment in Gynecologic Laparoscopy (SQUASH) – in collaboration with the Medical University of Vienna – has been approved by the FWF Austrian Science Fund. In this research project we will investigate if we can improve the efficiency and feasibility of technical skill assessment by automatic video content analysis, i.e, by content-based video retrieval and current methods of machine learning/deep learning.

We consider this research project as a pioneering work in the interdisciplinary overlap of computer- and medical science, which will investigate fundamental research questions that should provide the basis for future computer-aided surgical quality assessment (SQA). We expect that our research results will help to significantly facilitate the currently cumbersome and error-prone SQA process, and hence enable more surgeons to actually perform error ratings.

More information is available here.

Detecting Semantics in Endoscopic Videos with Deep Neural Networks

Here are the slides of my talk given at the 4th European Congress on Endometriosis (EEC2018) in Vienna, Austria, on November 23, 2018.

Interactive Video Search: Where is the User in the Age of Deep Learning? @ ACMMM18

Here are the slides of our tutorial presented on Monday, October 23, 2018 at ACM Multimedia 2018 in Seoul.

The Importance of Medical Multimedia @ACMMM18

Here are the slides of our tutorial presented on Monday, October 23, 2018 at ACM Multimedia 2018 in Seoul.

Residual Motion Improves Deep Learning Performance for Surgical Actions in Gynecologic Laparoscopy

In a recent work, presented at the 31st IEEE International Symposium on Computer-Based Medical Systems (CBMS2018), we could show that the inclusion of Residual Motion improves classification performance of surgical actions in videos from gynecologic laparoscopy significantly (resulting in a boost of Recall and Precision of 5% and 9% with the GoogLeNet CNN architecture). This performance can be improved even further (to a boost of 13% and 25% in terms of Recall and Precision) by using a late fusion approach for frame classification in the videos. The corresponding paper can be found here.

diveXplore Interactive Video Retrieval System

We have created a video of the diveXplore system, which we used for the Video Browser Showdown in 2017 and 2018 (as well as for the Lifelog Search Challenge 2018 in slightly modified form) quite successfully (2nd place in all three competitions). The video shows the different features of the system when applied to the IACC.3 dataset that consists of 600 hours of video content (around 300000 shots):

Cataract-101 Video Dataset

The ITEC Cataract-101 Dataset is available under here:

It consists of videos from 101 cataract surgeries, annotated with different operation phases that were performed by four different surgeons over a period of 9 months. These surgeons are grouped into moderately experienced and highly experienced surgeons (assistant vs. senior physicians), providing the basis for experience-based video analytics, as described in detail in the corresponding paper presented at MMSYS 2018.

LapGyn4 Gynecologic Laparoscopy Dataset

The ITEC LapGyn4 Gynecologic Laparoscopy Image Dataset is available under here:

It comprises four individual datasets (surgical actions, anatomical structures, actions on anatomy, and instrument count) taken from 500+ gynecologic laparoscopic surgeries for the task of automatic content analysis, as described in detail in the corresponding paper presented at MMSYS 2018.