Virtual LiVe

The project Virtual LiVe aims at digitizing classic event formats that require physical presence via the use of new immersive media formats (e.g. 3D-Audio, 360°-Video, Light-fields, Volumetric Video). Virtual LiVe is funded within the framework of the KMU-akut program “Research for SMEs” of the Fraunhofer-Gesellschaft.

The three partner institutes of this project Fraunhofer FOKUS, Fraunhofer HHI and Fraunhofer IIS strive to find applicable high-end solutions for the event-streaming needs of various stakeholders from different industries.

PROJECT DETAILS

Goals and Motivation

A large number of requests show the massive increase in demand for immersive exchange formats due to COVID-19 and require a quick response.

Virtual LiVe – Virtualization of Live Events via Audiovisual Immersion strives to develop a pool of technological solutions where players from the industry can find a remedy for this problem.

The aim is to develop and evolve:

Platforms & Distribution
Identification, evaluation and implementation of technical and non-technical requirements and corresponding design of a platform for near-real-time and protected transmission of media content and interactions. Furthermore of marketing options (pay-per-use (tickets), subscription or advertising-supported models), and of necessary changes and adaptation along the distribution chain.

Interactive 3D Content
Identification, evaluation and implementation of workflows and tools for audiovisual interaction with realistic spatial perception through the combination of representation formats (i.e. 360°-Video, Light-fields, 3D-Audio), data reduction for immersive interactive content, efficient use of interaction possibilities and integration into communication platforms.

360°-Video & 3D-Audio
Identification, evaluation and implementation of requirements for high quality 360°-Video and 3D-Audio live-streaming for concerts that is compatible with various devices. Consideration of the requirements of cultural events, data protection and data security according to German/EU law, viewing direction and excerpt selection with object-based 3D audio and integration into communication platforms.

Current Problem

Due to the Corona crisis, face-to-face events such as concerts, trade fairs or workshops are no longer possible in the classical format with physical presence. This leads to cultural impoverishment and endangers business and cultural exchange in society. The cultural and creative industries are particularly dominated by SMEs and are the driving force behind such face-to-face events. They generate a turnover of more than EUR 170 billion in sales with 1.8 million employees and have been hit particularly hard as one of the most important sectors of the German economy.

At the same time, however, new immersive media formats (including 3D-Audio, 360°-Video, Light-fields, Volumetric Video) are opening up the possibility of digitizing classic event formats. This promises additional revenue streams and target audiences. Virtual LiVe therefore investigates how present-day solutions can be extended via high-quality virtual presence with immersive content to create new formats for virtual and hybrid cultural events as well as platforms for congresses and trade fairs.

 

Solution

Virtual LiVe will develop a tool box with individual elements from the technological expertise of the project partners. SMEs can put together a customized program from this tool box depending on the needs and requirements for their respective event needs and thus obtain a technology-based solution for the realization of their formats.

By combining competencies of the project partners, the entire production field of virtual events can be covered by a broad portfolio of immersive media formats (e.g. 3D-Audio, 360°-Video, Light-fields, Volumetric Video, interactive low-latency streaming and advanced stream analytics). The experience of the participating partner institutes in the respective core topics allows for a highly efficient and synergetic approach, enabling innovative and practical commercial applications and value creation opportunities for SMEs within a short processing time. Virtual LiVe offers the following benefits especially for SMEs:

  • Modular and scalable, thus both customer and application-specific optimized cost / benefit ratio
  • Complete solution including licensing and payment system from a single source, thus particularly attractive for SMEs
  • Higher degree of immersion compared to competitive solutions thanks to broad technology portfolio

 

Monthly Colloquium

The events will be held in German language.

Join our monthly colloquium session, keep up to date with the latest developments and also get in touch with the consortium of the project! Sign up for our newsletter in the section below and get all notifications!

6th Colloquium

Title: Spin-offs & Fraunhofer start-up collaborations – CoLab, AHEAD & FTTF support technology transfer from initial utilization ideas to financing.
Speaker: CoLab, AHEAD & FTTF

Date: 22 December 2021
Time: 10 am – 11 am
Location: Virtual

Abstract:
CoLab is the support program at Fraunhofer for initiating, devolping and implementing Fraunhofer startup collaborations in technology transfer. We bring Fraunhofer technologies and external founders/startups together, prepare them as tandems in a structured way for the AHEAD program and continue to accompany the teams in AHEAD.

AHEAD is the accelerator program of the Fraunhofer Gesellschaft. In addition to the pre-qualified teams from the CoLab, our offer supports in particular teams of Fraunhofer Intrapreneurs & Founders in founding startups or developing licensable products on the basis of Fraunhofer technology. In an iterative approach we work with the participants on challenges around market, product, team, IP and business model. In doing so, we provide the right tools, methods, networks and coaching to provide the necessary skills and contacts in the field of high-tech entrepreneurship. Through our work, we increase the chances of success of licensed products as well as Fraunhofer spin-offs and increase the attractiveness for venture capital to enter.

FTTF stands for “Fraunhofer Technology Transfer Fund” (FTTF) and is Germany’s first TechTransfer Fund. It finances start-ups using Fraunhofer technology with up to 250 k€ directly after the foundation or pre-seed phase. In subsequent financing rounds, up to an additional 5 m€ can be invested, but then together with other investors who often come from the company’s own network. The portfolio already includes a number of teams that have previously been accompanied by AHEAD and the CoLab. The team sees itself as co-founders and actively supports the establishment of the company.

In their collaboration, the three offerings from CoLab, AHEAD and FTTF pursue the goal of increasing both the number and success rate of spin-offs with Fraunhofer technologies.

_____________________________________________________________________

5th Colloquium:

Title: Insight into the digital stage: The project ‘The Digital Stage – digital-stage.org’ introduces itself
Speaker: Digital Stage

Date: 24 November 2021
Time: 10 am – 11 am
Location: Virtual

Abstract:
Many ensembles are currently looking for solutions to rehearse or even perform online. Common video conferencing solutions are usually speech-optimized, suppress quiet voices and are unsuitable for music and theater due to long time delays. The Digital Stage offers a remedy in the form of a modular audio-video conferencing service optimized for the needs of music, dance, theater and performing arts. For rehearsals, live performances and teaching – online on the Internet and distributed from home. The presentation gives both an overview on The Digital Stage project and an insight into the technical details.

_____________________________________________________________________

4th Colloquium:

Title: Social Media Platforms and Selected Use Cases from the Festival Landscape
Speaker: Sönke Kirchhof (Executive Producer / CEO, INVR.SPACE GmbH)

Date: 22 September 2021
Time: 10 am – 11 am
Location: Virtual

Abstract:
With the onset of the pandemic in 2020, the Berlinale festival season began, and suddenly both large institutions and small cultural events – as well as almost all industries – were forced to convert their offerings almost entirely to online. A number of platforms quickly emerged that allowed different levels of interaction with different numbers of people / their avatars. The development was so fast that meanwhile even first platforms are forming their own award category within festivals and conferences (like the VR Days Europe in Amsterdam) or the different builds are listed as a separate category within festivals like the Venice Biennale.

_____________________________________________________________________

3rd Colloquium:

18 August
Virtual

Title: Open Round Table on the Subject “Virtual and Hybrid Trade Fairs”

Abstract:
What could future trade fairs and conferences look like? Which technologies are necessary, in order to digitize such events or to realize them as hybrid events? Which challenges have to be resolved that such events are as immersive, intuitive and profitable as possible for all participants? We would like to try to answer those questions and numerous others conjointly with you. Moreover, we intend to create a first rough sketch of how a possible validation project or a feasibility study could look like.

______________________________________________________________________

2nd Colloquium:

21 July 2021
Virtual

Title: Connected Adventures – Six Projects of the Dresden Symphony Orchestra 2006-2024
Speaker: Markus Rindt (Intendant, Dresdner Sinfoniker)

Abstract:
The Dresden Symphony Orchestra leaves the traditional concert hall and looks for places where ideas and feelings, where the burning and relevant questions are to be found. They connect with people across continents, in real time, with technical means, but also with a wink. Their programs are musical syntheses of arts in which the levels of time, place, style and culture interpenetrate and influence each other. This intensifies and sharpens the perception of music towards an actively experienced, contemporary art form.
Six examples: In “Remote Conducting” from 2008, a conductor in concert dress stands on the banks of the Thames in London, where passersby mistake him and an accompanying barrel organ player for exalted street musicians. In the process, his conducting is transmitted by satellite to Dresden, where his orchestra follows him in a classical concert setting. In the “Hochhaussinfonie” a huge apartment block on Dresden’s history-steeped Prager Straße mutates into the Battleship Potemkin. The musicians of the orchestra are connected from 43 individual balconies and provide the soundtrack to the Eisenstein film. It is about the revolutions of the 20th century, about social upheavals and their effects on all of us… The project “Codex Dresdensis – Concert at the End of Time” creates the connection between a Mayan manuscript kept in a museum in Dresden and living indigenous culture in Mexico, concentrated in a joint, media-combined concert. With the crowdfunding project “Tear down this wall”, the Sinfonikers then in fact go to America and initiate a music festival on the border between Mexico and the USA, meant as a call to overcome everything that separates us in our minds. In 2019, “Himmel über Prohlis” transforms an entire Dresden district into a gigantic open-air concert hall. The instrumentalists spread out on high apartment blocks, whose residents and their guests on the surrounding streets will be enveloped in sound. The music of the evening was written especially for this location and is aimed decidedly at the residents of the prefabricated buildings. Finally, the “Europasinfonie”, a project in planning, takes a look at our common cultural area of Europe. The Dresden Symphony Orchestra will connect with eleven other top European orchestras via ultra-fast fiber optic cable: musical European understanding in real time and at eye level. A plea for Europe.

______________________________________________________________________

1st Colloquium:

23 June 2021
Virtual

Title: Future of Live Entertainment
Speaker: Sven Bliedung von der Heide (Volucap)

Abstract:
From mega events in Fortnite to musicians in the living room. New opportunities, formats and digital platforms for live events – Sven Bliedung von der Heide provides an overview and introduction to the status quo and latest trends.

Kick-Off Event: Virtual LiVe – Virtualization of Live Events via Audiovisual Immersion

26 May 2021
@3IT Berlin/ Hybrid Event

Shaping the future of virtual events together! Read the article about the event here.

Kick-Off Event Virtual LiVe 26 May 2021: General Project Overview

Presented by: Angela Raguse, Fraunhofer IIS // The event was held in German language.

Kick-Off Event Virtual LiVe 26 May 2021: Fraunhofer Technology Presentation

Presented by: AC Coppens, the Catalysts // Christian Weißig, Fraunhofer HHI // Stephan Steglich, Fraunhofer FOKUS // Siegfried Fößel, Fraunhofer IIS // The event was held in German language.

Virtual LiVe – Hybrid Live-Concert in New Format @Kesselhaus Berlin & @Planetarium Bochum with Fraunhofer Technology

With the help of interactive streaming technologies, visitors were able to experience a hybrid concert by the artist not only on site in Berlin, but also via the Internet, applaud the artist and exchange ideas with each other. Some of the musicians joined the performance via the Internet, even though they were playing in different locations. In addition, the concert was transmitted to the Planetarium Bochum as a 360° projection and with 3D-Audio sound.

Read the in-depth article about the event here.

The communication of the newsletter is in German.

If you’re already subscribed to the 3IT newsletter and also want to receive the Virtual LiVe newsletter, please contact us.

General Contact

Name
Expertise
Position
Contact
Maria Ott Coordination & Communication Fraunhofer HHI
Project Manager 3IT
E-Mail
Jennifer Chyla Coordination &
Communication
Fraunhofer HHI
Deputy Manager 3IT
E-Mail

Technical Contact: Immersive A/V Streaming & Platform Capabilities

Real-Time Immersive AV Streaming // DRM, ad-based Models, Interaction and Social Media Integration // Scalable Playback (Smartphone, Web, VR / AR, Dome)

Name
Expertise
Position
Contact
Dr. Stephan Steglich Transmission and playback of immersive media, interaction Fraunhofer FOKUS
Head of Department Future Applications and Media
E-Mail
Dr. Louay Bassbouss Web Technologies, Standardization (W3C, HbbTV, CTA), Media Streaming Fraunhofer FOKUS
Senior Project Manager, Business Area FAME
E-Mail

Technical Contact: Immersive Video Recording and Encoding

360°-Video // Volumetric Video // Video Coding (H.26x)

Name
Expertise
Position
Contact
Christian Weissig 360 degree video,
volumetric video
Fraunhofer HHI
Head of Capture & Display Systems Group
E-Mail
Thomas Koch 360 degree video,
volumetric video
Fraunhofer HHI
Project Manager at the Capture & Display Systems Group
E-Mail

Technical Contact: Immersive Audio Technologies and 6DoF Light-Field Rendering

3D-Audio (MPEG-H) // High-Quality Audio Communication (EVS) // Immersive Image-Based Rendering (Light-field)

 

Name
Expertise
Position
Contact
Dr. Michel Bätz Light-field, MPEG-I Fraunhofer IIS
Sub-project leader Virtual LiVe IIS
E-Mail
Dr. Ulli Scuda Immersive and object based audio, binaural audio, MPEG-H, interactive audio Fraunhofer IIS
Head of Group, SoundLab
E-Mail

PROJECT PARTNERS