Candidate:
Jorge Federico Forero Rodríguez
Date, Time and Location:
8 July 2026, at 14:30, Sala de Atos of the Faculdade de Engenharia da Universidade do Porto
President of the Jury:
João Carlos Pascoal Faria (PhD), Full Professor of the Department of Informatics Engineering of the Faculdade de Engenharia da Universidade do Porto.
Members:
Nuno António do Nascimento Correia (PhD), Associate Professor at University of Tallinn, Estonia;
Hugo Gonçalo Oliveira (PhD), Associate Professor of the Department of Informatics Engineering, Faculdade de Ciências e Tecnologia da Universidade de Coimbra;
Nuno Manuel Robalo Correia (PhD), Full Professor of the Department of Informatics, Faculdade de Ciência e Tecnologia da Universidade Nova de Lisboa;
Mónica Sofia Santos Mendes (PhD), Assistant Professor with Habilitation, Faculdade de Belas-Artes da Universidade de Lisboa (Supervisor);
Tiago Barbedo Assis (PhD), Assistant Professor of the Department of Design, Faculdade de Belas-Artes da Universidade do Porto;
António Fernando Vasconcelos Cunha Castro Coelho (PhD), Associate Professor with Habilitation of the Department of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto.
The thesis was co supervised by Gilberto Bernardes de Almeida, Assistant Professor of the Department of Informatics Engineering of the Faculdade de Engenharia da Universidade do Porto.
Abstract:
The Ironic Machine[s] is a practice-based research project that explores the use of a bimodal speech emotion recognition system for the generation of affective virtual environments and creative
computational artifacts. Situated at the intersection of media art, affective computing, and natural language processing, the study investigates how rhetorical figures such as kind irony and sarcasm emerge through the divergence between semantic and acoustic emotional predictions, and how this interaction can inform the construction of Affective Virtual Environments. The primary contribution of this thesis is the formulation and empirical evaluation of a method for analyzing illocutionary irony using multimodal speech emotion recognition datasets. Building on sentiment analysis and speech emotion recognition techniques, the research introduces a statistical inference framework that integrates perceptual evaluation with hypothesis-driven analysis to identify prosodic features that significantly differentiate sincere and ironic speech. For posed American English samples, the results demonstrate that divergences between semantic and acoustic emotional predictions can be systematically quantified, supporting the hypothesis that prosodic irony can be understood as a measurable multimodal phenomenon. A secondary objective investigates whether these findings can be translated into a generative strategy for constructing Affective Virtual Environments. To this end, a three-level prompt engineering framework is proposed, mapping semantic and acoustic emotional predictions onto audiovisual features informed by statistical analyses of musical and visual datasets. Perceptual evaluation indicates that feature-based prompting shows a consistent advantage over high-level emotional labels, although the results do not reach conventional levels of statistical significance. The investigation is realized through a series of nine technological and artistic iterations, developed as autonomous yet interconnected works. These projects function as experimental platforms for exploring emotional ambiguity, disorientation, and the instability of meaning in computational environments within a techno-poetic narrative. Collectively, this work proposes both a methodological framework for the analysis of irony and a speculative approach to affective environment generation. It positions divergence between modalities as an operational principle in multimodal systems and demonstrates how practice-based research can serve as a rigorous mode of inquiry into the relationship between human emotion and machine-mediated representation.
Keywords: Speech Emotion Recognition, Prosodic Irony, Affective Virtual Environments, Multimodal Analysis, Practice-based Research, Media Art.







