The paper 'A 3D Tele-Immersion System based on Live Captured Mesh Geometry' by CWI researchers Rufael Mekuria, Dick Bulterman and Pablo Cesar (Distributed & Interactive Systems group) was selected as Best Paper on 3D Technology and Multimedia at the ACM Multimedia Systems Conference (MMsys) 2013 in Oslo (Norway). The article, authored with colleagues Michele Sanna, Stefano Asioli and Ebroul Izquierdo from QMUL, reports on the on-going collaboration between researchers at CWI and QMUL in the context of the EU-funded REVERIE project, which aims at enabling “3D-based Skype” between remote users.
3D Tele-Immersion enables participants in remote locations to share an activity in real-time. The article describes a complete media pipeline that can efficiently stream live-reconstructed 3D representations (triangle meshes). The pipeline includes both a novel fast local compression algorithm and a rateless packet protection scheme. First, it highlights the drawbacks of existing triangle mesh codecs and exploits the properties of live 3D reconstruction to achieve real-time encoding. Second, an implementation of a rateless code, inspired on network coding, achieves robust real-time transmission. These components have been successful integrated into a larger tele-Immersion environment that includes beyond the state of the art 3D reconstruction and rendering modules. This resulted in a prototype that can capture, compress, transmit, and render triangle mesh geometry in real-time over the Internet.
About the EU-funded REVERIE project:
REVERIE (REal and Virtual Engagement in Realistic Immersive Environments) believes that the time is ripe to push social networking towards the next logical step in its evolution: immersive collaborative environments that support realistic inter‐personal communication. Recent scientific advances in a variety of different research fields represented within REVERIE means that it is now possible to integrate research outputs towards technologies that support real‐time realistic interaction between humans in online virtual & immersive environments. In this way, the ambitious ideas outlined above will come within reach of the next generation of Internet users.