3IT SUMMIT 2022 – 3D, VR & AR in Industry, Health & Entertainment

We are happy to once again welcome you to the 3IT SUMMIT! Our annual partner & friends event will finally take place again this year and we are eager to see all of you back again in this familiar format! The 3IT SUMMIT is open for everybody who is interested in immersive imaging technologies like 3D, VR and 360-degree video in different fields of application.

The event is dedicated to our Partners in the first place: This year’s 3IT SUMMIT will be focusing on the most recent projects from our Partner network that have been hosted at the 3IT. Learn about the perks of AR-based clinical education from the projects TeleSTAR and MultiARC. Find out how Mixed Reality can be used to create a more efficient way of direct, interactive and user-oriented teaching via photo- and audiorealistic avatars of real lecturers for remote learning in the panel talk of the project VoluProf. And last but not least, get up to date on the possibilities concerning the virtualization of live events via audiovisual immersion from the project Virtual LiVe! Due to the special character of the 3IT SUMMIT, the event will take place in person.
This year, the official part will be followed by a VR demonstration of a visionary concert for the audience to experience hands on, as well as technical tours showcasing brand new demonstrators!

A casual get-together during which you will have the chance to network and exchange recent developments and new ideas will conclude the program of the 3IT SUMMIT 2022.

Find the agenda below and register now!

Please note that the presentations will be given in English language.

To attend the event, proof of testing, recovery or vaccination for Covid-19 must be presented at the registration.

Friday 06 May 2022
1pm – 9pm

Salzufer 6
Entrance Otto-Dibelius-Strasse
10587 Berlin

TeleSTAR/ MultiARC

TeleSTAR (Telepresence for Surgical Assistance and Training using Augmented Reality) utilizes a digital surgical microscope and augmented/mixed reality to provide new tools for teaching and intraoperative assistance. Standard procedures as well as rare and complicated ones can be visually annotated and augmented to make every step of the surgery transparent. A bidirectional communication platform allows questions to the surgeon. This concept can be scaled to other surgical domains and existing degree programs.

The project MultiARC engages with developing an interactive and multimodal-based augmented reality system for computer-assisted surgery for ear, nose and throat treatment. Input/output data handling, a hybrid multimodal image analysis, as well as a bi-directional interactive augmented reality (AR) and mixed reality (MR) interface for local and remote surgical assistance are part of the framework. The benefits are manifold and will lead to a highly improved patient outcome.

The speaker Eric L. Wisotzky is a research associate at the Computer Vision & Graphics Group of the Vision & Imaging Technologies Department.



The project VoluProf (Volumetric professor for omnipresent and user-optimized teaching in mixed reality) aims to develop a user-oriented, interactive Mixed Reality application enabling discourses between teachers, represented as avatars, and users by means of multimodal chatbots. Its basis are volumetric videos of teachers and a voice generated from a text provided by them. Learners then stream the volumetric avatar to their MR headsets.

Dr. Cornelius Hellge is head of the Multimedia Communications Group and Dr. Anna Hillsmann  is Head of Vision & Imaging Technologies Department at Fraunhofer Heinrich Hertz Institute. The speaker Thomas Buchholz is a Senior Expert at Telekom Innovation Laboratories. Dr. Andy Räder is a Research Assistant (PostDoc) at the Institute for Media Research at the University of Rostock.


Virtual LiVe

Virtual LiVe (Virtualization of Live Events) aims at digitizing classic event formats that require physical presence via the use of new immersive media formats. The project’s three partner institutes Fraunhofer FOKUS, Fraunhofer HHI and Fraunhofer IIS strive to find applicable high-end solutions for the event-streaming needs. The aim is to develop and evolve Platforms & Distribution, Interactive 3D Content and 360°-Video & 3D-Audio.

The speaker Christian Weissig is the head of Capture & Display Systems Group at  the Vision & Imaging Technology Department Fraunhofer HHI and project leader of Virtual Live.


Validation Project: Hybrid Live Concert at Kesselhaus Berlin, Planetarium Bochum and Remote

On 11 December 2021, concert visitors of Billy Andrew’s Classic RoXX Tour could either physically attend Kesselhaus Berlin where the concert was held, Planetarium Bochum where it was streamed, or join online via smartphone, tablet, either PC or smart TV. For the broadcast at the planetarium, a 360-degree live video and 3D sound were recorded at Kesselhaus and transmitted via the internet. In addition, the artists Queenz of Piano, who played from a different location in Berlin and were incorporated into the concert via an ultra-low latency audio connection, accompanied Andrews. This allowed the artists to play together live on stage while based in different locations.

The speaker Robert Seeliger is a Senior Project Manager and DAI solutions lead at the Business Unit FAME at Fraunhofer FOKUS.


Validation Project: 360° Live-Keying with PHØNIX16

In April 2022, the project “Virtual LiVe” will be presenting a further validation step with a 360 degree keyed broadcast of INDUSTRY, an experimental musical concert by the ensemble PHØNIX16. The team of PHØNIX16, a soloist ensemble founded in Berlin in 2012, partnered up with the team of the 3IT-hosted Project Virtual LiVe to examine how the use of cutting-edge immersive technology could contribute to their experimental format. In the production of the Industry, the artists performed around the Omnicam360 and were recorded with a panorama camera from 360 degrees in a green screen studio.

The OmniCam360, developed at Fraunhofer HHI, is a scalable, mirror-based multi-camera system that allows the recording of live videos in a 360° panoramic format. In addition, the Fraunhofer HHI developed the Real Time Stitching Engine (RTSE), a software-based solution for the real-time processing of ultra-HD panoramic recordings. RTSE allows for high resolution 360 degree live keying. Keying is the process of removing a green screen background, making it fully transparent, which allows to fill in the transparent area with a different image or video. During the production of the concert the RTSE provided a preview in real-time, in which parts of the 360 degree stage that were not green were masked as green. This created a fully green screen background and allowed for the insertion of the planned video on site. This material in turn was made available to the creative director and operators who were able to immediately identify critical scenes and find a solution on the spot.

The speaker Thomas Koch is Project Manager for Audio related topics at the Capture & Display Systems Group in the Vision & Imaging Technologies Department at Fraunhofer Heinrich Hertz Institute.


Validation Project: The Future of Volumetric (Live) Concerts (“Live from the Other Room”)

The Virtual LiVe Validation Project “The Future of Volumetric (Live) Concerts (“Live from the Other Room”)” develops and tests new and seminal immersive audio and video technologies in practice. The Fraunhofer-Institutes FOKUS and IIS have, based on the concept of a volumetric live-concert by Radar Media and High Road Stories, tested further technological possibilities by using MPEG-H 3D-audio to stream and the live-mix of object-oriented 3D-sound, MPEG-I MIV (MPEG Immersive Video) – a possible future standard for compressing 6DoF-videocontent – as an alternative to 6DoF-video real-time-rendering and MPEG-DASH/HLS for adaptive streaming of the volumetric video, which then adapts to whichever data rate and computing capacity available.

Based on the work and in cooperation with Radar Media and Higher Road Stories, Fraunhofer FOKUS and IIS could develop and test immersive audio and video technologies for future volumetric live concerts in order to create the best experience possible.

The two speakers Michael Bätz and Johannes Schmidl are Research Assistant and Business Development Manager in the Streaming Applications Department at Fraunhofer IIS.


Workshop Report: Live from the Other Room: Martin Kohlstedt

Live from The Other Room is a VR concert venue in which musicians perform in volumetric video, live streamed to audiences worldwide. Fans can book tickets to live shows in virtual reality, alone or with friends. Audience members can move freely through the space, communicate with each other and with the artist directly. Custom creative concepts are designed in collaboration with the artists, and musicians are recorded in a comfortable environment with direct audience feedback. The immersive experiences will be streamable on the most widespread 6DoF VR headsets.

Since the beginning of the COVID pandemic many musicians have been trying to bring their music to audiences remotely. Some have been singing into webcams, some have made avatars out of themselves and performed in popular multiplayer video games like Fortnite. These performances spiked some interest, but are artistically not always interesting to serious musicians who thrive on audience presence and authentic communication.

Humans have been performing for each other for thousands of years, and a concert is a cultural expression that can affect a person profoundly – alone and/or in a collective context. Currently in its development stage, LFTOR aims to explore the elementary aspects of musical performance and what constitutes a satisfying experience for both artist and audience. Exactly replicating a real life concert situation in VR is technically not feasible and not useful, whereas a well-considered virtual space offers an opportunity to avoid the current disadvantages of established music industry practices and create a more inclusive standard.

Martin Kohlstedt is a German composer and pianist, known for his mixture of classical and avant-garde music, improvised on stage during passionate performances.

The two speakers are Harmke Heezen of creative studio High Road Stories (Berlin) and Oliver Ihrens of communication agency Radar Media (Bochum).

The registration is closed.

Salzufer 6
Entrance Otto-Dibelius-Strasse
10587 Berlin