Virtual LiVe

The project Virtual LiVe aims at digitizing classic event formats that require physical presence via the use of new immersive media formats (e.g. 3D-Audio, 360°-Video, Light-fields, Volumetric Video). Virtual LiVe is funded within the framework of the KMU-akut program “Research for SMEs” of the Fraunhofer-Gesellschaft.

The three partner institutes of this project Fraunhofer FOKUS, Fraunhofer HHI and Fraunhofer IIS strive to find applicable high-end solutions for the event-streaming needs of various stakeholders from different industries.

Register for the monthly colloquium (in the News & Events section) and subscribe for the Virtual LiVe newsletter in the menu below and keep up-to-date with all the news concerning this project!

PROJECT DETAILS

Goals and Motivation

A large number of requests show the massive increase in demand for immersive exchange formats due to COVID-19 and require a quick response.

Virtual LiVe – Virtualization of Live Events via Audiovisual Immersion strives to develop a pool of technological solutions where players from the industry can find a remedy for this problem.

The aim is to develop and evolve:

Platforms & Distribution
Identification, evaluation and implementation of technical and non-technical requirements and corresponding design of a platform for near-real-time and protected transmission of media content and interactions. Furthermore of marketing options (pay-per-use (tickets), subscription or advertising-supported models), and of necessary changes and adaptation along the distribution chain.

Interactive 3D Content
Identification, evaluation and implementation of workflows and tools for audiovisual interaction with realistic spatial perception through the combination of representation formats (i.e. 360°-Video, Light-fields, 3D-Audio), data reduction for immersive interactive content, efficient use of interaction possibilities and integration into communication platforms.

360°-Video & 3D-Audio
Identification, evaluation and implementation of requirements for high quality 360°-Video and 3D-Audio live-streaming for concerts that is compatible with various devices. Consideration of the requirements of cultural events, data protection and data security according to German/EU law, viewing direction and excerpt selection with object-based 3D audio and integration into communication platforms.

Current Problem

Due to the Corona crisis, face-to-face events such as concerts, trade fairs or workshops are no longer possible in the classical format with physical presence. This leads to cultural impoverishment and endangers business and cultural exchange in society. The cultural and creative industries are particularly dominated by SMEs and are the driving force behind such face-to-face events. They generate a turnover of more than EUR 170 billion in sales with 1.8 million employees and have been hit particularly hard as one of the most important sectors of the German economy.

At the same time, however, new immersive media formats (including 3D-Audio, 360°-Video, Light-fields, Volumetric Video) are opening up the possibility of digitizing classic event formats. This promises additional revenue streams and target audiences. Virtual LiVe therefore investigates how present-day solutions can be extended via high-quality virtual presence with immersive content to create new formats for virtual and hybrid cultural events as well as platforms for congresses and trade fairs.

 

Solution

Virtual LiVe will develop a tool box with individual elements from the technological expertise of the project partners. SMEs can put together a customized program from this tool box depending on the needs and requirements for their respective event needs and thus obtain a technology-based solution for the realization of their formats.

By combining competencies of the project partners, the entire production field of virtual events can be covered by a broad portfolio of immersive media formats (e.g. 3D-Audio, 360°-Video, Light-fields, Volumetric Video). The experience of the participating partner institutes in the respective core topics allows for a highly efficient and synergetic approach, enabling innovative and practical commercial applications and value creation opportunities for SMEs within a short processing time. Virtual LiVe offers the following benefits especially for SMEs:

  • Modular and scalable, thus both customer and application-specific optimized cost / benefit ratio
  • Complete solution including licensing and payment system from a single source, thus particularly attractive for SMEs
  • Higher degree of immersion compared to competitive solutions thanks to broad technology portfolio

 

Find videos from events in the Video Material section!

Monthly Colloquium

The events will be held in German language.

Due to the great demand since our Kick-Off Event on 26 May, we are organizing a monthly colloquium where you can keep up to date with the latest developments and also get in touch with the consortium. Also, sign up for our newsletter in the section below to stay informed about the latest dates of this format.

1st Colloquium:

Date: 23 June 2021
Time: 10 am – 11 am
Place: virtual

Registration: here

Title: Future of Live Entertainment
Speaker: Sven Bliedung von der Heide (Volucap)

Abstract: From mega events in Fortnite to musicians in the living room. New opportunities, formats and digital platforms for live events – Sven Bliedung von der Heide provides an overview and introduction to the status quo and latest trends.

Kick-Off Event: Virtual LiVe – Virtualization of Live Events via Audiovisual Immersion

The event was held in German language.

Date: May 26 2021
Time: 10:00 a.m. – 1:30 p.m.
Place: virtual

Agenda: GER

We want to actively shape the future of virtual events with you! The different needs of the cultural and event industry for virtual events will be identified within the framework of this project. The goal is to develop a platform that serves applicable high-end solutions for the event streaming needs of various actors from different industries in the form of a modular system.

Renowned speakers at the event were:

Moderation:
AC Coppens, THE CATALYSTS

Project presentation:
Angela Raguse, Fraunhofer IIS, Business Area Digital Media

Keynotes:
Kavaye Ozong, ARTE
Harmke Heezen, High Road Stories & Oliver Ihrens, Radar Media
Thomas Bedenk, Freelancer

Panel:
Billy Andrews, The Dark Tenor
Christoph Becher, Gahrens & Battermann
Felipe Sanchez Luna, KlingKlangKlong
Stefan Trieb, mac
Annemie Vanackere, Hebbel am Ufer
Oliver Warnke, Iventic
Julia Wissert, Schauspiel Dortmund

Kick-Off Event Virtual LiVe 26 May 2021: General Project Overview

Presented by: Angela Raguse, Fraunhofer IIS // The event was held in German language.

Kick-Off Event Virtual LiVe 26 May 2021: Fraunhofer Technology Presentation

Presented by: AC Coppens, the Catalysts // Christian Weißig, Fraunhofer HHI // Stephan Steglich, Fraunhofer FOKUS // Siegfried Fößel, Fraunhofer IIS // The event was held in German language.

The communication of the newsletter is in German.

If you’re already subscribed to the 3IT newsletter and also want to receive the Virtual LiVe newsletter, please contact us.

Project Partners

Fraunhofer FOKUS, Fraunhofer HHI and Fraunhofer IIS develop cutting-edge technologies in the field of audio & video production as well as coding and transmission, with the core research fields complementing each other in an optimal way:

 

Fraunhofer FOKUS: Immersive A/V Streaming & Platform Capabilities

  • Real-Time Immersive AV Streaming
  • DRM, ad-based Models, Interaction and Social Media Integration
  • Scalable Playback (Smartphone, Web, VR / AR, Dome)

 

Fraunhofer HHI: Immersive Video Recording and Encoding

  • 360°-Video
  • Volumetric Video
  • Video Coding (H.26x)

 

Fraunhofer IIS: Immersive Audio Technologies and 6DoF Light-Field Rendering

  • 3D-Audio (MPEG-H)
  • High-Quality Audio Communication (EVS)
  • Immersive Image-Based Rendering (Light-field)
Expertise
Position
Name
Contact
Coordination & Communication Fraunhofer HHI
Project Manager 3IT
Maria Ott E-mail
Coordination & Communication Fraunhofer HHI
Deputy Manager 3IT
Jennifer Chyla E-mail
Web Technologies, Standardization (W3C, HbbTV, CTA), Media Streaming Fraunhofer FOKUS
Senior Project Manager, Business Area FAME
Dr. Louay Bassbouss E-mail
Light-field Fraunhofer IIS
Sub-project leader Virtual LiVe IIS
Dr. Michel Bätz E-mail
Immersive Media, Object-Based Audio  Fraunhofer HHI
Research Assistant,
Capture & Display Systems
Christoph Ende E-mail
Light-field, 3D Audio Fraunhofer IIS
Head of Moving Picture Technology Department
Dr. Siegfried Fößel E-mail
MPEG-H, Broadcast, Streaming  Fraunhofer IIS Harald Fuchs E-mail
360° Film Experiences, Algorithms for visualization, tracking, data fusion and interaction Fraunhofer FOKUS
Deputy Head Business Unit VISCOM
Dipl.-Inf. Ivo Haulsen E-mail
Light-field Fraunhofer IIS
Head of Group Computational Imaging and Algorithms
Dr. Joachim Keinert E-mail
360 degree video
volumetric video
 
Fraunhofer HHI
Project Manager
at the Capture & Display Systems Group
Thomas Koch E-mail
Audio Codecs (i.e. AAC, EVS, LC3/LC3Plus), Communication applications Fraunhofer IIS
Head of Audio for Communications
Manfred Lutzky E-mail
Algorithms for visualization,
VR/AR, 360° projection systems
Fraunhofer FOKUS
Business Area VISCOM
Dipl.-Inf. Manuel Schiewe E-mail
Immersive and object based audio, binaura laudio, MPEG-H, interactive audio  Fraunhofer IIS
Head of Group, SoundLab
Dr. Ulli Scuda E-mail
Adaptive Media Streaming (DASH, HLS) Fraunhofer FOKUS
Project Manager, Future
Applications and Media (FAME)
 
M.Sc. Daniel Silhavy E-mail
Transmission and playback of immersive media, interaction Fraunhofer FOKUS,
Head of Department Future Applications and Media
Dr. Stephan Steglich E-mail
360 degree video
volumetric video
 
Fraunhofer HHI
H
ead of Capture & Display Systems Group

Christian Weissig   E-mail

PROJECT PARTNERS