Duration: 36 months
Kickoff: September 2021
Sponsorship: Bundesministerium für Bildung und Forschung (BMBF)
Typical virtual teaching is not really interactive usually, as all content is recorded with static video and audio scripts and users can hardly provide feedback. Mixed Reality (MR), combined with animated volumetric avatars, makes immersed and interactive learning possible, taking the quality of teaching to a new level.
In this project, Aristech is responsible for synthesizing the teacher’s script with text-to-speech. The teacher’s voice is replicated accurately by using a new machine learning method. Facial expressions, lip movement and body language are automatically adjusted to the avatar accordingly.
Speech synthesis, speech recognition methods, and a chatbot system are being developed additionally. This gives the animated avatar a realistic voice when reading a teacher’s script, and also allows users to add input via speech or text.
This project is the first to combine the advantages of traditional lectures and virtual teaching respectively by letting the teachers be virtually/physically available and experienceable despite being on video.
We are working on VoluProf with several project partners:
• Fraunhofer Heinrich-Hertz-Institut (HHI) - Verbundkoordinator
• Deutsche Telekom AG
• Volucap GmbH
• Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI)
• Universität Rostock