Candidate:
Alexandre Resende Clément
Date, Time and Location:
5th of June 2025, 14:30, Sala de Atos, Faculdade de Engenharia, Universidade do Porto
President of the Jury:
António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with Habilitation, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto
Members:
Marcelo Mortensen Wanderley, PhD, Full Professor, Department of Music Research, Schulich School of Music, McGill University, Canadá;
Damián Keller, PhD, Associate Professor, Centro de Educação, Letras e Artes da Universidade Federal do Acre, Brasil;
Sofia Carmen Faria Maia Cavaco, PhD, Assistant Professor, Informatics Department, Faculdade de Ciências e Tecnologia, Universidade NOVA de Lisboa;
Rui Pedro Amaral Rodrigues, PhD, Associate Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto;
Gilberto Bernardes de Almeida, PhD, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto (Supervisor).
This thesis was cosupervised by Rui Luis Nogueira Penha, PhD, Coordinator Professor, Instituto Politécnico do Porto Escola Superior de Música e Artes do Espetáculo, Instituto Politénico do Porto.
Abstract:
“This thesis explores the potential of mobile handheld devices as tools for digital musical instrument interaction and participatory performance. Guided by the principles of ubiquitous music and intuitive interaction, the research investigates how mobile handheld devices can address challenges and unlock opportunities in contemporary music-making through participatory frameworks, gesture mappings, and multimodal feedback. Three experiments form the foundation of this study. The first describes and evaluates a system that enables large-scale audience participation in multimedia performances. It highlights the ability of mobile handheld devices to engage users and foster collaboration but reveals challenges in designing intuitive interactions for untrained participants. The second experiment examines how users instinctively map gestures to core musical parameters, such as pitch, duration, and amplitude, identifying natural trends – and the influence of musical training and experience on interaction strategies. The third focuses on evaluating the impact of multimodal feedback, combining auditory, visual, and haptic modalities, in note pitch tuning tasks.
The findings underscore the importance of designing standardised interaction guidelines and integrating multimodal feedback to make digital musical instruments more accessible and intuitive. Experiment 1 showed that the lack of a unified interaction model limited intuitive engagement, highlighting the need for standards that balance individual creativity with group intent. Experiment 2 found clear user preferences for gesture mappings of onset, pitch, and duration, shaped by cultural familiarity, and supporting context-aware design. Experiment 3 showed that while multimodal feedback had little immediate effect on accuracy, it improved user confidence and may aid long-term learning. This research advances the understanding of how mobile handheld devices can support participatory and creative music-making, contributing to the development of inclusive, user-friendly, and versatile musical tools.”