The creation of virtual environments is an important topic that has been addressed in recent years. Our team has undertaken a number of internationally leading projects in the field, in particular, using novel procedural modelling techniques to build 3D urban environments [PTDC/EIA-EIA/108982/2008] and recreate ancient cities [PTDC/EIA-EIA/114868/2009]. The main efforts, in these projects, were focused on the creation of the 3D models and thus the visual component of the virtual environments. Now we want to significantly improve their realism and consequently make them substantially more immersive. We define such a virtual environment, which a user might “admit to be real or true “ as “acknowledgeable”.
Our world is multisensory. How human beings perceive a physical environment depends on the interaction established by their senses, the cultural context, and their personal backgrounds and values. Currently, virtual environments (VEs) are playing an increasingly important role in user-centric applications, including training, design, entertainment etc. Yet the digital systems currently used still rely on visual expression and presentation only, with perhaps sound occasionally included as an additional sensory input.
Recent findings suggest that the more human’s senses that are engaged in a VE the more immersive is the experience and better is the subjects´ performance (as in real environments). The multisensory integration seems to be based on the principles of Bayesian decision theory incorporating Maximum-Likelihood Estimation (MLE; e.g., [Deneve 2004]; [Ernst 2002]; [Ernst 2004]; [Knill 1996]). There is also evidence showing that by adding a supplementary sensory modality to a display may only affect the way a participant responds to a given stimulus but not his/her actual experience. [Gallace 2012]
MASSIVE will investigate in detail how we can achieve an acknowledgeable virtual environment (AVE) in which a user’s five senses are stimulated to achieve a high immersivity. Thus, we will study, at a fundamental level, how perceptual issues affect the delivery of the five senses: visuals, audio, smell, taste and feel (temperature, wind-flow, humidity and motion). In order to measure the impact that each sense has in the VE experience, the assessment will be carried out by means of objective evaluation of data such as: heart rate, skin humidity as well as other more subjective methodologies, for example questionnaires, that test whether the experience is immersive or not.
The novelty of MASSIVE consists of the research and development of:
• Fundamental understanding of how individual human beings perceive virtual environments;
• New modelling and selectively deliver techniques that exploit cross-modal effects and personal knowledge to deliver high-fidelity to the perceptually important areas;
• Multi-sensory AVEs that are perceptually equivalent to the real world related scene which is being depicted;
• A Methodology that validates the efficacy of the resultant system against the target application.
The foundations of the project are expected to present and progressively assure the AVEs as reliable, safe, controlled and cost effective means to recreate real environments.
The conclusion of the project will gather and deliver the following results:
• MASSIVE: Research Lab. Facilities Prototype and MASSIVE: multisensory Engine – will enable the group to participate in key European projects and be one of the leading groups in multisensory research in Immersive VEs.
o Development of perceptual models
o Methodologies for multisensory delivery
o Multimodal interaction
• MASSIVE Acknowledgeable Experience:
The developed technology will be used to create a pilot case study in order to explore the potential of the MASSIVE system:
• Training and certification: within cities Civil Protection needs to conduct various simulations to certificate for example fire fighters.
This application will help to overcome problems that limit the number and quality of training opportunities, including, work requirements, limited training facilities, and budgetary.