Van Damme won the Award for his article “From Individual QoE to Shared Mental Models: A Novel Evaluation Paradigm for Collaborative XR”. The paper is the result of the scientific sabbatical of three months at CWI, Distributed and Interactive Systems group, of the postdoctoral researcher Sam Van Damme from Ghent University. The article is co-authored by Jack Jansen (CWI), Silvia Rossi (CWI), and Pablo Cesar (CWI and TU Delft).
Focus on group dynamics in XR
To truly measure the success of an XR team application, the research focuses less on individual technical satisfaction and more on the group's collective understanding and connection. To do so, Van Damme explores a novel approach for evaluating collaborative and social eXtended Reality (XR) experiences. Beyond the more traditional individual metrics used by the community, such as Quality of Experience (QoE) or Sense of Presence (SoP), this article proposes a metric based on group level dynamics: Shared Mental Model (SMM). This metric, coming from Psychology, measures the strength and congruence of the dynamics of a group. A high SMM is already known in team psychology to be crucial for effective collaboration and better performance.
Results: a strong shared understanding is possible in XR
To better understand how to explore this construct, we ran a prototypical experiment where two users worked together on a block-building task. The influence of asymmetric latency (delay) was manipulated, meaning one collaborator might experience a different amount of lag than the other.
The results showed that even in an XR context, a strong SMM can still form among collaborators, even when those users have very different roles and behaviors, and are working under the stress of unequal technical conditions (like asymmetric lag).
This work is one further step of the DIS group on the exploration of metrics and protocols to adequately evaluate Social XR experiences, mixing subjective and objective methods. Apart from award-winning articles, the work has resulted in open source software and datasets, and in standardization activities in ITU.
About DIS
CWI's Distributed and Interactive Systems (DIS) group focuses on facilitating and improving the way people use interactive systems and how people communicate with each other. We combine data science with a strong human-centric, empirical approach to understand the experience of users. This enables us to design and develop next generation intelligent and empathic systems. We base our results on realistic testing grounds and data sets, and embrace areas such as ubiquitous computing, human-centered multimedia systems, and languages. Group leader Pablo Cesar won the Dutch prize for ICT research in 2020 and is also a professor of Human-Centered Multimedia at TU Delft.
More information
CWI's DIS group hosts VQEG Standardization Meeting on Immersive Communication Systems
