New Project Launched @3IT: VoluProf – Volumetric Mixed Reality Professor for Omnipresent and User-Optimized Teaching

In September 2021, 3IT’s long-time partners Deutsche Telekom AG and Fraunhofer HHI, in collaboration with the project partners Aristech GmbH, German Research Center for Artificial Intelligence (DFKI), University of Rostock and Volucap GmbH have launched a new project at 3IT: VoluProf, which is funded by the Federal Ministry of Education and Research, BMBF.

The goal of the project is to develop a Mixed Reality application that is suitable for everyday use within the context of knowledge transfer. The idea is to create a novel and more efficient way of direct, interactive and user-oriented teaching via photo- and audiorealistic avatars of real lecturers for remote learning. 

Since the beginning of the pandemic in 2020, online teaching has taken on a completely new meaning and relevance. At the same time the weaknesses and limitations of the technological tools that have been hitherto used in the context of education have been exposed: traditional online teaching, where content is delivered via static video and audio scripts, is not sufficient to provide an interactive and engaging learning experience for users.

This is the point where VoluProf comes into play: the project partners want to create a space of future learning where both traditional lecturing and online learning environments are combined in a way, where lecturers can be experienced not only on video but also virtually and physically. In other words, the user participates in a lesson with a virtual professor by means of Augmented Reality glasses for example in their living room, while also being able to interact with them via eye contact and verbal communication.

Which criteria will play a role in the project realization?

  • Generating a photorealistic and animatable volumetric representation of the lecturer.
  • Location-independent applicability through transmission via 5G networks and direct integration into mixed reality devices on the market, as well as subjective optimization of image quality at minimal bit rate.
  • Natural and fast synthesis of the lecturer’s original voice.
  • Answering ethical questions about the implications of the new way of knowledge transfer.
  • Media reception and effects research for the analysis and optimization of user acceptance.
  • Educational research on the optimal and user-adaptive use of the new way of knowledge transfer

VoluProf strives for a ubiquitous and democratic use of technology, enabling accessibility for everyone to photo- and audiorealistic avatars of lecturers in particular by using 5G networks and advanced streaming technologies. By creating an immersive knowledge space, an improved learning experience is made possible, in which the limits of human-to-human interactions in digital learning are transcended.

About the project partners

The different in-depth knowledge of the project partners, ranging from the fields of language technology software (Aristech GmbH), 5G networks (Deutsche Telekom AG), Artificial Intelligence and educational technology (DFKI), Video, Image Coding and Processing of volumetric data (Fraunhofer HHI), empirical user research (University of Rostock) as well as production and post-production of Volumetric Video data (Volucap GmbH) guarantee a very interesting collaboration with exciting outcomes.

Are you interested in the future of Virtual and Mixed Reality Learning? Or are you interested in becoming a partner of the 3IT? Do not hesitate to contact the 3IT’s team here.

Stay tuned for more information on VoluProf here.  

Header: © /Theltern/Ljupco/klikk edit by Fraunhofer HHI