TeleSTAR

The project TeleSTAR uses a full digital surgical microscope and augmented/mixed reality to provide new tools that permit teaching and intraoperative assistance. EIT Health is supported by EIT, a body of the European Union.

PROJECT DETAILS

Goals and Motivation

TeleSTAR-Telepresence for Surgical Assistance and Training using Augmented Reality, utilizes a fully digital surgical microscope and augmented/mixed reality to provide new tools that permit teaching and intraoperative assistance. The system offers new ways to visualise relevant surgical information, allowing surgeons to improve, and also explain, their decisions. Surgical decisions can be easily visualised during course units, on high-resolution screens or a head-mounted display, in any local or remote location. The system guarantees the exact same surgeon’s field-of-view for all attending healthcare professionals. “Simple” standard procedures, as well as rare and complicated procedures, can be visually annotated and augmented – and supplemented with audio comments – to make every step of the surgery transparent. A bidirectional communication platform allows all attendees to address questions to the surgeon at the right time, which creates a dynamic teaching environment despite local distance. The teaching concept can be scaled to other surgical domains and existing degree programmes.

Current Problem

There is currently a lack of pedagogical and technical education and training measures for surgical interventions to reduce the associated digital illiteracy. The intraoperative surgeon’s view can only be conveyed to third parties (senior/chief physicians, biomedical engineering students) to a limited extent during advanced trainings and surgical workshops. This specifically concerns the surgical field-of-view as well as the lack of image-based annotation and commentary options that are directly linked to the anatomical scene. Within the framework of EIT-Health, two projects are currently being funded that try to address the above-mentioned problems with VR. But, there are currently no new training concepts for the promising approaches based on augmented reality (AR). Furthermore, there is a lack of networked IP-based solutions that are able to transfer intraoperative assistance and teaching systems with additional AR information to remote operating rooms or lecture halls. Especially in Germany, there is currently no program that is awarded with CME.

Solution

TeleSTAR will develop a training concept for healthcare professionals namely senior and chief physicians in ENT surgery using augmented reality (AR) in combination with computer vision algorithms. The healthcare professional networks of TU Delft, Charité – Universitätsmedizin Berlin and ARRI Medical will be brought together. ENT procedures will be performed using the full digital surgical microscope “ARRISCOPE”. ENT chief and senior physicians from the German network will attend the operation in Berlin. The procedures will be transmitted to remote lecture rooms in Delft. The recorded annotated video material will then be made available to all participants for discussion and teaching purposes. The training effect will be monitored with a digital questionnaire. Because of limited me for physicians, we will organize intensive 1-2 days sessions 0. Technical Demo, 1. Description of patient’s history, 2. Preoperative examinations and imaging will be presented to the learners, 3. Participants can suggest therapeutic strategy, which is discussed with the expert surgeon, 4. We will use teledialog tools to ensure interactivity and make live transmissions in 3D, 5. During live transmission, relevant surgical information will be highlighted with AR tools, 6. Finally, a 2nd expert will create an ad hoc questionnaire based on the performed intervention and sent to remote learners. The learning outcome will be monitored based on the correct answers and will be awarded with CME points.

Learning Outcomes

The course concept addresses three pillars for two focus groups (1) healthcare professionals and (2) assistant healthcare workers. The three pillars are as follows.

  1. Surgical Pillar
    Teaching the anatomy of the ear with focus on temporal bone and middle ear
  2. Digital literacy Pillar
    Discussing opportunities and pitfalls of advanced imaging systems (3D,AR)
  3. Remote/interactive Teaching Pillar for surgical training/education
    Best practice using interactive tools in a remote teaching environment by offering audio, video, touch-based interface to support collaboration between trainer and trainees.

The course will be officially awarded with CME points.

TeleSTAR @ Charité Berlin

19 January 2020
Berlin, Germany

TeleSTAR‘s first technical 3D-AR-based test run with assistant surgeons in a remote lecture room at the Charité in Berlin.

Talk @ XR4ALL Workshop

20 March 2020
Berlin, Germany

The event was canceled due to COVID-19.

Find more information here.

Live surgery transmission @ German Society of Oto-Rhino-Laryngology Head and Neck Surgery

20-23 May 2020
Berlin, Germany

The congress was canceled due to COVID-19.

Find more information here.

TeleSTAR talk & discussion panel @ Health Reality Labs – Games for Health

17 June 2020
Online event

Find more information here.

TeleSTAR talk & discussion panel @ DMEA Sparks

18 June 2020
Online event

Find more information here and register here.

ENT course in Delft/Rotterdam

25 September 2020
Delft/Rotterdam, Netherlands

Topic: Broadcasting of a live cochlea implant surgery using 3D & Augmented Reality.

Registration: The Registration will be anounced on the the landing page: www.telestar.health. Registration will be only available for healthcare professionals, physicians and master students of Biomedical engineering and related fields. CME credit points might be applicable.

ENT course in Berlin

14/15 November 2020
Berlin, Germany

Topic: Hands-On course temporal bone dissection incl. live surgery using 3D & Augmented Reality.

Registration: Registration will be be anounced on the the landing page: www.telestar.health. Registration will be only available for healthcare professionals, physicians and master students of Biomedical engineering and related fields. CME credit points might be applicable.

The 3D market, which is constantly evolving, also provides innovative solutions for the medical and industrial sectors. The further development of the medical engineering industry is supported by the German government with our latest project.

The project MultiARC is engaged with developing an interactive and multimodal-based augmented reality system for computer-assisted surgery in the context of ear, nose and throat (ENT) treatment.

Cochlea implant surgery – 3D AR-based Live-Stream:

For a higher resolution of the video, please click here.

Project Consortium