The constantly changing and advancing 3D market also provides new innovative solutions for the medical and industrial sectors. The further development of the medical engineering industry is supported by the Federal Ministry for Economic Affairs and Energy (BMWi) with our latest project: 3DInMed.

Project Details


The project has essentially four main goals, including the development of new, solid processes of depth estimation and object detection as well as the design of high frequency and low latency transmission lines for 3D signals. Furthermore the development of real time processes to create endoscopic 3D panoramas through texture- and depth information from stereo-endoscopic views is essential. As well as the development of spatial mapping processes and measurement methods for 3D microscopy/endoscopy data and related AR-applications are planned.

Results and Partners

The main objective is to develop, evaluate and validate key technologies for 3D-capturing, -processing, -transmission and autostereoscopic visualization and integrate these technologies into and beyond the targeted application areas. The consortium is led by the Fraunhofer Gesellschaft and the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute. The project partners are ARRI Medical, C.R.S. iiMotion, Fraunhofer HHI, Fraunhofer IIS, Schölly, SeeFront and Solectrix. Ultimately the project has associated partners in particular the Technical University Munich / Hospital “rechts der Isar”, research group for Minimal Invasive, Interdisciplinary, Therapeutic Intervention (MITI); Ludwig-Maximilians-University Munich, Großhadern Campus, Cochlea-Implant-Centre; Neurosurgical clinic and polyclinic, hospital “rechts der Isar”, Technical University Munich; University Hospital Erlangen and Siemens Energy.

Industrial Details

3D applications in industry are primarily used in the field of automotive and robotics by improving driver assistant systems, autonomous navigation and related capturing of surroundings. Touchless inspection and measurement processes of inaccessible tubular parts are another field of application. Supported by 3D visualizations in computer assisted designs, these inspection and diagnostic applications are used as well in 3D X-Ray or CT-Volume of chemical or biological structures.

Medical Details

3D applications in a medical environment enables doctors to record stereoscopic data and its three dimensional visualizations to guarantee an enhanced and improved workflow for minimal invasive operations. Associated technologies are surgery-microscopy, spatial approximation of operating areas, recognition and labelling of risk tissues (e.g. nerve tract) and robot or telemanipulator supported operations. Furthermore it aims at enabling the combination of preoperative and intraoperative diagnostic images, for documentation and education purposes.

3DInMed @ MEDICA 2017

Medica 2017: Fraunhofer HHI and ARRI Medical present digital 3D surgical microscope 

Fraunhofer Heinrich Hertz Institute HHI and ARRI Medical present new processes for digital 3D measuring technology at Medica in Düsseldorf, Germany, at Fraunhofer booth G05 in hall 10.

Digital 3D Surgical Microscope 
The ARRISCOPE is a digital stereo surgical microscope. As part of the Federal Ministry for Economic Affairs and Energy (BMWi) research project 3DInMed, Fraunhofer HHI has developed a method for intraoperative measurements with high precision.
As a result, it is now possible for the first time to accurately estimate the required prosthesis size or to specify tumor dimensions.

3DInMed – Digital 3D endoscopy for inspection and metrology 
Fraunhofer HHI and ARRI Medical present results of the BMWi-funded project 3DInMed.
Digital 3D endoscopes used for industrial applications open up completely new possibilities for extracting and visualizing additional information by using advanced stereo algorithms for depth estimation. In particular, the survey of cavities that are difficult to access and the inspection of wearing parts by such methods, allow a more precise statement about the surveyed items condition.

Please find more information here.
Download 3DInMed @ Medica 2017 Flyer

3DInMed @ Control 2017

Fraunhofer HHI and Schölly present a technical case study of a 3D endoscope used for online visual inspection tasks of hardly accessible parts at Control 2017.

Download 3DInMed @ Control 2017 Flyer


Fraunhofer HHI presented first results of the 3IT project 3DInMed at Medica 2016. Digital stereo camera systems offer new possibilities to extract and visualize additional information resulting in significantly enhanced conditions for diagnostics and surgical interventions. Shown are the latest research results for measuring, tracking of medical instruments, augmented reality applications (AR) and optimized 3D playback.

A positive feedback was received from many sides, including engineers, surgeons and product managers. This result is a strong indicator that the developed methods and solutions of 3DInMed are of high relevance for digital 3D imaging technologies in medical applications.

Download 3DInMed@Medica 2016 Flyer

CARS 2017, Barcelona
CARS 2017-Computer Assisted Radiology and Surgery, 31th International Congress and Exhibition Barcelona, Spain, 20 – 24 June, 2017.
New Tools for Digital Surgical Microscopy
N. Gard, S. Jurk, A. Schneider, P. Kauff, P. Eisert, J.-C. Rosenthal, Fraunhofer Heinrich-Hertz-Institut, Berlin; ARRI, München

Please note: the conference language was German therefore some of the following titles are also in German language.

DGE-BV MEETS ENDOSKOPIE LIVE in Berlin, 06 – 08 April 2017

Herausforderungen, Lösungen und Potenziale der Stereo- Bildverarbeitung für die 3D-Endoskopie
Jean-Claude Rosenthal, Niklas Gard

SID-ME Fall Meeting 2016 in Berlin, 03 – 04 November 2016

Depth-Based Quality Control for Medical Applications
Jean-Claude Rosenthal, Niklas Gard, Peter Kauff

15th CURAC Annual Conference in Bern, 29 Septmber – 01 October 2016

Session: Endoscopy & Microscopy

3D-Reconstruction by Polarization Imaging in Endoscopy
J. Sandvoss, T. Wittenberg, A. Nowak, J. Ernst

Evaluierung von 3D-Rekonstruktionsverfahren in der Stereo-Laparoskopie
D. Erpenbeck, T. Wittenberg, J.-C. Rosenthal, P. Kauff, N. Lemke, T. Bergen

Program: http://curac.org/programbern2016#Endoscopy-Microscopy

Special session: Optische 3D-Endoskopie & Mikroskopie

Miniaturisierte 3D-Endoskopie
Johannes Ruhammer, Niels Lemke, Schölly Fiberoptic GmbH, Denzlingen

3D-Techniken in der Klinik
Dirk Wilhelm, Nils Kohn, TU München

Potential der digitalen 3D Operationsmikroskopie
Armin Schneider, ARRI Medical, München

Von der 2D zur 3D Panorama-Endoskopie
Daniel Erpenbeck, Thomas Wittenberg, Fraunhofer IIS, Erlangen

Tiefenbasierte Stereo-Bildverarbeitung für medizinische Anwendungen
Peter Kauff, Niklas Gard, Jean-Claude Rosenthal, Fraunhofer HHI, Berlin

Program: http://curac.org/programbern2016#Special-Session-Endoscopy-Microscopy

14th CURAC Annual Conference in Bremen, 17 – 19 Septmber 2015

Session: Optische 3D-Modalitäten in der Chirurgie

3D-Verarbeitung und Visualisierung mit Optischen Modalitäten
Jean-Claude Rosenthal, Fraunhofer HHI, Berlin

Die Digitalisierung der Operationsmikroskopie
Christoph Bichlmeir, ARRI, München

Aspekte der 3D Endoskopie
Niels Lemke, Schölly, Denzlingen

Panorama Endoskopie – von 2D zu 3D
Thomas Wittenberg, Fraunhofer IIS

Program: https://www.curac.org/programm2015#Sondersitzung-CURAC-DGBMT

Graphical Material

Copyright: Fraunhofer IIS - (a) Left image of a typical 3D laparoscope (b) L-HRM disparity map (c) 3D reconstructed point cloud - 2 instruments have been removed
Fraunhofer HHI: (a) Real-time tracking result of instrument tip inside a temporal bone model. (b) Related result of real-time depth map estimation.

Consortium Leader

Project Partners

Associated Partners

Ludwig-Maximilians-University Munich
Neurosurgical clinic and polyclinic, hospital “rechts der Isar”, Technical University Munich
University Hospital Erlangen
Siemens Energy