Candidate:
Jorge Federico Forero Rodríguez

Date, Time and Location:
8 July 2026, at 14:30, Sala de Atos of the Faculdade de Engenharia da Universidade do Porto

President of the Jury:
João Carlos Pascoal Faria (PhD), Full Professor of the Department of Informatics Engineering of the Faculdade de Engenharia da Universidade do Porto.

Members:
Nuno António do Nascimento Correia (PhD), Associate Professor at University of Tallinn, Estonia;
Hugo Gonçalo Oliveira (PhD), Associate Professor of the Department of Informatics Engineering, Faculdade de Ciências e Tecnologia da Universidade de Coimbra;
Nuno Manuel Robalo Correia (PhD), Full Professor of the Department of Informatics, Faculdade de Ciência e Tecnologia da Universidade Nova de Lisboa;
Mónica Sofia Santos Mendes (PhD), Assistant Professor with Habilitation, Faculdade de Belas-Artes da Universidade de Lisboa (Supervisor);
Tiago Barbedo Assis (PhD), Assistant Professor of the Department of Design, Faculdade de Belas-Artes da Universidade do Porto;
António Fernando Vasconcelos Cunha Castro Coelho (PhD), Associate Professor with Habilitation of the Department of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto.

The thesis was co supervised by Gilberto Bernardes de Almeida, Assistant Professor of the Department of Informatics Engineering of the Faculdade de Engenharia da Universidade do Porto.

Abstract:

The Ironic Machine[s] is a practice-based research project that explores the use of a bimodal speech emotion recognition system for the generation of affective virtual environments and creative
computational artifacts. Situated at the intersection of media art, affective computing, and natural language processing, the study investigates how rhetorical figures such as kind irony and sarcasm emerge through the divergence between semantic and acoustic emotional predictions, and how this interaction can inform the construction of Affective Virtual Environments. The primary contribution of this thesis is the formulation and empirical evaluation of a method for analyzing illocutionary irony using multimodal speech emotion recognition datasets. Building on sentiment analysis and speech emotion recognition techniques, the research introduces a statistical inference framework that integrates perceptual evaluation with hypothesis-driven analysis to identify prosodic features that significantly differentiate sincere and ironic speech. For posed American English samples, the results demonstrate that divergences between semantic and acoustic emotional predictions can be systematically quantified, supporting the hypothesis that prosodic irony can be understood as a measurable multimodal phenomenon. A secondary objective investigates whether these findings can be translated into a generative strategy for constructing Affective Virtual Environments. To this end, a three-level prompt engineering framework is proposed, mapping semantic and acoustic emotional predictions onto audiovisual features informed by statistical analyses of musical and visual datasets. Perceptual evaluation indicates that feature-based prompting shows a consistent advantage over high-level emotional labels, although the results do not reach conventional levels of statistical significance. The investigation is realized through a series of nine technological and artistic iterations, developed as autonomous yet interconnected works. These projects function as experimental platforms for exploring emotional ambiguity, disorientation, and the instability of meaning in computational environments within a techno-poetic narrative. Collectively, this work proposes both a methodological framework for the analysis of irony and a speculative approach to affective environment generation. It positions divergence between modalities as an operational principle in multimodal systems and demonstrates how practice-based research can serve as a rigorous mode of inquiry into the relationship between human emotion and machine-mediated representation.

Keywords: Speech Emotion Recognition, Prosodic Irony, Affective Virtual Environments, Multimodal Analysis, Practice-based Research, Media Art.

Candidate:
Nádia de Sousa Varela de Carvalho

Date, Time and Location:
6 July 2026, at 14:30, Room Professor Vasco de Sá (L119) at DEMec, Faculdade de Engenharia da Universidade do Porto

President of the Jury:
António Fernando Vasconcelos Cunha Castro Coelho (PhD), Associate Professor with Habilitation, Department of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto

Members:
Diemo Schwarz (PhD), Researcher of the Institute for Research and Coordination in Acoustics/Music (IRCAM), France;
Rui Luís Nogueira Penha (PhD), Coordinating Professor, Escola Superior de Música e Artes do Espetáculo do Instituto Politécnico do Porto;
Sofia Carmen Faria Maia Cavaco (PhD), Assistant Professor, Informatics Departament, Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa;
António Humberto Sá Pinto (PhD), Invited Assistant Professor, Departament of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto and Affiliated Researcher at the Institute of Systems and Computer Engineering, Technology and Science (INESC TEC);
Gilberto Bernardes de Almeida (PhD), Assistant Professor, Department of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto (Supervisor).

Abstract:

This thesis investigates the ontological and performative shift of the musical work. In this sense, we propose using intelligent computational mediation to reconfigure it from a static artefact into a mutable interface. Historically, the Western musical tradition has been framed by the work-concept (werktreue), which crystallizes the composition as an immutable object preserved by the rigidity of the score or the phonographic medium. However, contemporary digital and electroacoustic practices demand new paradigmas that account for the fluid boundaries between composition, performance, and technological agency. Acknowledging that, this research proposes a transition from the static score to the navigable topological manifold. Rather than utilizing fixed musical notation, these spaces treat the musical work as a dynamic environment. In this environment, identity is preserved through structural fluidity, leading to a paradigm change to a contextual navigation model. In a symbiotic dialogue of real-time structural discovery, the machine provides a restricted, high-dimensional map of the work’s universe, which the performer navigates using their gestural intuition.

The methodology follows a mixed approach between experimental and practice-based research. This methodology operates over a reflective-iterative cycle, integrating software development with musical performance. The study is divided into two main phases: symbolic (tonal) and sub-symbolic (timbre) representation and navigation. In the first phase, variational autoencoders (VAEs) are used to compress symbolic polyphony. A critical milestone was the empirical alignment found between unsupervised VAE latent spaces and theoretical Discrete Fourier Transform (DFT) phase spaces. This alignment confirms that black box latent space models naturally discover structural principles consistent with established music theory, such as the circle of fifths. The second phase employs Neural Audio Synthesis (RAVE) to map the timbre landscape of the tenor saxophone into high-dimensional latent spaces. To navigate these spaces, the research introduces a taxonomy of musical motions (parallel, oblique, and contrary). These motions propose to connect abstract data and musical intent, enabling gestural navigation that honours performer agency.

The practical applicability of this research is materialized in BroadcastJSB and Aethra. The former, a web-based topological instrument, uses a radio metaphor for real-time navigation within J.S. Bach’s chorales’ latent space. The latter consists of a co-creative interface for mixed music that synthesizes multidimensional complexity into a single axis of performative control. The primary contribution of this work lies in the formalization of the mutable work as a space of possibilities. We also demonstrate musical intelligibility within deep learning models and create open-access tools and data repositories for the computer music research and contemporary performance communities. In summary, the thesis argues that the future of musical practice lies in improving human intuition through intelligent mediation. By doing this, we position the mutable interface as a key place for collaboration and creative exploration in today’s digital world.

Candidate:
Pinto Francisco Impito

Date, Time and Location:
19 June 2026, at 14:00, Sala de Atos, Faculdade de Engenharia da Universidade do Porto

Chair of the Examination Committee:
Pedro Nuno Ferreira da Rosa da Cruz Diniz (PhD), Full Professor in the Department of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto.

Members:
Felisbela Maria Carvalho Lopes (PhD), Full Professor in the Department of Communication Sciences, Instituto de Ciências Sociais da Universidade do Minho;
Ivone Marília Carinhas Ferreira (PhD), Assistant Professor in the Department of Communication Sciences at the Faculdade de Ciências Sociais e Humanas da Universidade Nova de Lisboa;
Carla Susana Moiteiro Ganito Afonso (PhD), Associate Professor at the Faculdade de Ciências Humanas da Universidade Católica Portuguesa;
José Manuel Pereira Azevedo (PhD), Full Professor in the Department of Communication and Information Sciences at the Faculdade de Letras da Universidade do Porto (Supervisor);
Ricardo José Pinheiro Fernandes Morais (PhD), Assistant Professor in the Department of Communication and Information Sciences at the Faculdade de Letras da Universidade do Porto.

The thesis was co-supervised by Dr Vasco Francisco Japissane Cumbe, a lecturer at the Faculdade de Medicina da Universidade Católica de Moçambique.

Abstract:

Introduction:
The increasing complexity of health systems, combined with rapid technological advances, has transformed the ways digital tools are produced, disseminated, and used in healthcare. At the same time, the unprecedented availability of mobile devices and audiovisual resources has expanded access to technologies but also introduced new challenges, such as usage limitations due to structural and infrastructural issues, as well as inequalities in access.
Objective:
This study aimed to evaluate the implementation of digital technologies in health communication in Mozambique, focusing on the use of videos, with particular attention to doctor–patient interaction and medical training. The study implemented and assessed digital resources based on interactive simulation, education, and modeling, proposing a Mozambican digital innovation model adapted to the local context.
Methodology:
The research was conducted through three studies: (1) Interactive Simulation Video (ISV) in Medical Training, (2) Augmented Education Video (AEV) in Pre-ART Counseling for HIV-positive patients, and (3) Modeling Video (MV) in Rehabilitation Strategies. Study 1 evaluated the usability, educational value, and interactive experience of the ISV with medical students (N = 93), using an exploratory sequential approach. Study 2 employed a quasi-experimental design with an intervention group (IG = 23) and a comparison group (CG = 23), involving HIV-positive patients, using the Mann–Whitney U test to compare dimensions related to counseling quality. Study 3 followed a longitudinal design, with pre- and post-intervention assessments, to evaluate the impact of the MV on caregivers’ execution of motor recovery exercises for patients with stroke.
Results:
The studies revealed (1) high levels of agreement regarding realism, clinical relevance, and pedagogical usefulness (ISV); (2) statistically significant differences between groups (IG, CG) concerning time spent on care, appropriateness of counseling format, and decision-making for ART adherence (AEV); and (3) meaningful improvements after the intervention in strengthening practical learning and confidence in performing motor recovery procedures (MV). Overall, the research demonstrated that the structured and adapted use of digital resources, as proposed in this thesis, has high potential to transform communication processes in resource-limited environments.
Conclusion:
The research highlights the importance of structured and adapted integration of digital media in resource-limited contexts. Future investigations should explore more robust experimental designs, with larger samples and longitudinal follow-up, as well as assess the impact of these interventions on real clinical practice and treatment adherence indicators. The consolidation of the Mozambican digital innovation model can advance sustainable health education policies and promote the digital transformation of medical training in Mozambique.

Keywords: Digital innovation; Video; Medical Education; doctor–patient interaction; Mozambique.

Candidate:
Maria Helena Sampaio de Mendonça Montenegro e Almeida

Date Time and Location:
16 June 2026, 14:30, Sala de Atos, Faculdade de Engenharia da Universidade do Porto

President of the Jury:
Carlos Miguel Ferraz Baquero-Moreno (PhD), Full Professor at the Department of Informatics Engineering of the Faculdade de Engenharia da Universidade do Porto

Members:
Peter Johannes Schüffler (PhD), Professor at the Technical University of Munich, Germany;
Carlos Jorge Andrade Mariz Santiago (PhD), Research Assistant at the Robotics and Engineering Systems Laboratory (LARSyS) and Invited Assistant Professor of the Department of Electrical and Computer Engineering of the Instituto Superior Técnico da Universidade de Lisboa;
Jaime dos Santos Cardoso (PhD), Full Professor of the Department of Electrical and Computer Engineering of the Faculdade de Engenharia da Universidade do Porto (Supervisor);
Luís Filipe Pinto de Almeida Teixeira (PhD), Associate Professor of the Department of Informatics Engineering of the Faculdade de Engenharia da Universidade do Porto.

Abstract:

Artificial Intelligence models have been extensively applied to medical image analysis tasks over the past years, achieving outstanding results. However, the obscure reasoning of the models and the lack of supportive evidence causes both clinicians and patients to distrust their predictions, hindering their adoption in clinical practice. In recent years, the research community has focused on developing explanations capable of revealing a model’s reasoning. Among the various types of explanations, case-based explanations emerged as particularly intuitive for medical practitioners. While these types of explanations have been widely researched, they still possess limitations that compromise their real-world application. The main goal of this thesis is to overcome the limitations of medical case-based explanations, enabling their deployment in clinical practice.

To identify the main weaknesses of case-based explanations, we conduct a literature review on models that provide such explanations in healthcare. Through the analysis of existing works, we verify that the explanations raise privacy concerns by sharing sensitive images of patients, and lack the interactiveness required for clinicians to engage with the explanation. Furthermore, most works do not evaluate nor clinically validate the explanations through user studies with clinicians. Targeting these limitations, we propose deep generative models to obtain privacy-preserving, controllable and interactive explanations to explain the decisions of deep learning models.

To propose a privacy-preserving system for safely sharing case-based explanations, we start by reviewing the literature on image anonymisation techniques and identifying their vulnerabilities. In particular, we identify vulnerabilities by proposing two privacy attacks: a membership inference attack targeting deep generative models that generate synthetic images, and a superimposed image decomposition model to reverse the common anonymisation strategy of image averaging. After outlining the requirements that a privacy-preserving system for medical case-based explanations must fulfil, we propose privacy-preserving models capable of simultaneously anonymising medical images and generating counterfactual explanations. The proposed models rely on disentangled representation learning to separate identity and clinical traits, enabling their individual manipulation. Through this strategy, we can also separate and manipulate causal factors related with the clinical task to obtain controllable counterfactual explanations. To obtain interactive counterfactual explanations, we propose models to manipulate regions of medical images according to a segmentation mask, measuring their impact on a prediction. To evaluate the proposed models, we perform experiments on medical datasets like chest radiographs, verifying the models’ capacity to manipulate medical images for the purposes of generating explanations.

Finally, to show the wide applicability of the proposed models beyond their original purpose of generating case-based explanations, we apply them as decision-support systems in healthcare. More specifically, we adapt the models for predicting the aesthetic outcomes of breast cancer treatment, to facilitate the patients’ choice of treatment. Furthermore, we conduct a clinical study with breast surgeons from various health institutes to assess the predictions of the proposed models.

To conclude, through various contributions on explainable Artificial Intelligence and decision-support systems, this thesis contributes towards the safe use of trustworthy and privacy-preserving Artificial Intelligence models in healthcare.

Candidate: Samuel Filipe da Silva Pereira Oliveira