Category Archives: Highlights

LEIC DNA in the 2024 Nobel Prize in Chemistry

Posted on

Hugo Penedones, the Portuguese who was involved in a project that ended up being awarded a Nobel Prize and who has been talked about a lot, is not an unknown Portuguese to us.

Born in Chaves and a graduate of Dr Júlio Martins Secondary School, Hugo spent five years (2000-2005) at LEIC – Bachelor in Informatics and Computing Engineering (recently celebrated 30 years since its creation), where he graduated as the best student in his year.

Before embarking on this adventure, the LEIC graduate spent four years at Google DeepMind (London, UK), during which time he was part of the Alphafold project’s initial team, contributing to the first artificial intelligence model to predict the structure of proteins, which led to the article “Improved protein structure prediction using potentials from deep learning“, published in Nature in January 2020, co-authored with the now Nobel Prize-winning chemists John M. Jumper and Demis Hassabis (among others).

The Alphafold model has been developed over the past few years ‘and has been able to predict the structure of virtually all of the 200 million proteins that researchers have identified. Since its discovery, it has been used by more than two million people in 190 countries. Among its many scientific applications, researchers can now better understand antibiotic resistance and create images of enzymes that can break down plastic. Without proteins, life could not exist. The fact that we can now predict protein structures and design our own proteins is of the greatest benefit to mankind,’ can be read at the Nobel Prize press release.

The 2024 Nobel Prize in Chemistry was awarded on 9 October to David Baker for the near-impossible feat of building entirely new types of proteins, and to Demis Hassabis and John Jumper jointly for developing an AI model to solve a 50-year-old problem: predicting the complex structures of proteins.

In the speeches of Demis Hassabis and John Jumper, transcribed below in this order, it is clear that this achievement is due to all the team members, past and present, whose exceptional work has made this result possible. The enthusiasm and curiosity of Hugo Penedones, recalled here in his talk at Commit 2018 (an annual event organised by AlumniEI), is proof that this recognition is also his.

“Receiving the Nobel Prize is the honour of a lifetime. Thank you to the Royal Swedish Academy of Sciences, to John Jumper and the AlphaFold team, the wider DeepMind and Google teams, and to all my colleagues past and present that made this moment possible. I’ve dedicated my career to advancing AI because of its unparalleled potential to improve the lives of billions of people. AlphaFold has already been used by more than two million researchers to advance critical work, from enzyme design to drug discovery. I hope we’ll look back on AlphaFold as the first proof point of AI’s incredible potential to accelerate scientific discovery.”

“Thank you to the Royal Swedish Academy of Sciences for this extraordinary honor. We are so honored to be recognized for delivering on the long promise of computational biology to help us understand the protein world and to inform the incredible work of experimental biologists. It is a key demonstration that AI will make science faster and ultimately help to understand disease and develop therapeutics. This is the work of an exceptional team at Google DeepMind and this award recognizes their amazing work.

Computational biology has long held tremendous promise for creating practical insights that could be put to use in real-world experiments. AlphaFold delivered on this promise. Ahead of us are a universe of new insights and scientific discoveries made possible by the use of AI as a scientific tool. Thank you to my colleagues over the years, for making possible this moment of recognition, as well as the many moments of discovery that lie ahead.”

Hugo Penedones is currently the co-founder and Chief Technology Officer (CTO) of Inductiva.AI, a Portuguese start-up that provides an application development interface (API) that allows research labs and companies to use the start-up’s AI algorithms to run simulations. The applications for Inductiva’s technology are myriad, ranging from drug design to aerospace parts, mechanical engineering, route optimisation and more. At the end of last year, the startup – which, in addition to Hugo Penedones, has the other two co-founders on its board of directors, Luís Sarmento (ProDEI alumnus) as CEO, and Clara Gonçalves as COO – received an investment of 2.25 million euros, which has allowed it to invest in new technology and expand its customer base.

DEI Faculty ranked among the world’s most cited scientists

Posted on

The latest data of the “World’s Top 2% Scientists list“, which annually ranks the world’s most cited scientists according to SCOPUS, the world’s largest database of abstracts and citations of scientific publications, is now available.

U.Porto has 182 researchers on this list, including two DEI faculty, João Cardoso and Rui Maranhão (publishes as Rui Abreu).

The full article is available at Notícias U.Porto.

30 years of Informatics and Computing Engineering at FEUP – 30 years of Engineering with Soul

Posted on

In 2024 we celebrate 30 years since the creation of LEIC – the Bachelor’s Degree in Informatics and Computing Engineering at FEUP, a course that began as a pre-Bologna 5-year Bachelor’s Degree (LEIC) in 1994/95, went on to become a 5-year Integrated Master’s Degree (MIEIC) from 2006/07 and separate 3-year Bachelor’s Degree (L.EIC – joint course with FCUP) and 2-year Master’s Degree (M.EIC) courses from 2021/22, a format that continues today.

This is a good time to recall the innovative characteristics of LEIC when it was launched in 1994, namely that it was the first interdepartmental and interdisciplinary degree course at FEUP, with specific management bodies composed of teachers from various departments, a requirement presented in the creation proposal as essential and decisive by its creators.

The proposal was born in 1993, following a weekend of reflection at the Hotel Boega, Vila Nova de Cerveira, at the beginning of January, attended by Ademar Aguiar, João Falcão e Cunha, João Pascoal Faria, João Vasco Ranito, José Correia, José Manuel Moreira and Raul Moreira Vidal, at the time researchers at INESC Porto. The proposal to create LEIC, signed by Professors João Falcão e Cunha and Raul Moreira Vidal, was presented to the then Director of FEUP, Prof. José Marques dos Santos, and received his full support, namely the assumption of the position of LEIC’s Director between 1994 to 2001.

These have been years of hard work, disruption and winning bets, as evidenced by the evaluation of the External Commission appointed by the Foundation of Portuguese Universities (FUP), which in 2001 ranked LEIC 1st among all similar courses in Portugal, in what was the first evaluation of the course. The report of this evaluation states that “… it should be clarified that the teaching of Informatics in Portugal is in good health and that this health has improved even more since the 1st cycle of evaluations was carried out”, which leads us to conclude that LEIC has made a very significant contribution to the teaching of Informatics and Computing Engineering in our country.

And it is this quality of teaching that has always distinguished the course over the last 30 years, providing students not only with an excellent technical training, but also with transversal training that is highly valued by employers, who have always recognised the quality stamp of FEUP’s EIC graduates. Raul Moreira Vidal, co-founder of LEIC and its director from 2001 to 2008, the year in which he became director of the Department of Informatics Engineering (DEI), created in 2008, is certain that “some of the main reasons for the great successes recorded in the evolution of the course are essentially due to the quality and care taken in the supervision of the course, the quality of the direction/coordination of the course and the quality of the teaching staff and secretariat“.

Looking to the future and all the challenges associated with rapid technological evolution, João Paiva Cardoso, the current director of DEI, says: “30 years strengthens our commitment and increases our responsibility. It’s a time to celebrate and also to wish that we continue to improve our capacity to provide a unique and complete EIC education, which will allow us to continue to train the Informatics engineers of the future.”

 

The celebration

After 30 years and more than 2600 graduates, it’s time to celebrate the many achievements and look to the future with optimism.

And it was in a festive atmosphere that this milestone of 30 years of EIC was celebrated on the 27th and 28th September at the José Marques dos Santos Auditorium (streamed online), bringing together the large EIC family – alumni, teachers, staff, current students, industrial partners and other institutions, in a total of more than 300 participants, in an event co-organised with AlumniEI (FEUP’s Informatics Engineering Alumni Association, officially created in 2005). 

The programme was opened by Renato Natal Jorge (Deputy Director of FEUP) and João Paiva Cardoso (Director of DEI), followed by a presentation of the programme by the President of the Organising Committee, João Pascoal Faria, a presentation on the evolution and current situation of the course by Rosaldo Rossetti, Co-Director of L.EIC, and a presentation on the alumni community around the world by the President of AlumniEI, José Coutinho.

At this point in the programme, the two projects developed as part of this celebration were presented. The ‘EIC Timeline‘ project, developed by the LGP ‘OpenTabs’ team, where we can explore the historical moments that have shaped Informatics Engineering at the University of Porto, and the ‘AlumniEI World‘ project, a joint DEI-FEUP/AlumniEI initiative, developed by Jénifer Constantino (MESW 2024) and José Coutinho and Rui Neves as consultants, a tool based on LinkedIn data that allows us to find the current location/company of EIC alumni and search by country, course and year of graduation. It’s an ongoing project that aims to involve more and more former students and that will continue with two master’s theses.

Throughout the day, there were two round tables of alumni testimonials, where we were able to witness the diversity of paths and options for those who start from the same point.

Frederico Câmara (Alts Digital), Felipe Ávila da Costa (Infraspeak), Tiago Azevedo (University of Cambridge), Margarida Silva (AWS), Sofia Sá (Alumna L.EIC), Tiago Fernandes (Coverflex), Pedro Côrte-Real (MC Sonae), Sónia Liquito (Spotify), Rui Gonçalves (Meta) and Mariana Salvaterra (Zühlke Group) took to the stage to share their experiences, their individual stories, their difficulties and their successes in two panels moderated by Rui Rodrigues (Director of M. EIC) in the morning and Carla Teixeira Lopes (Director of MCI and one of the first LEIC graduates) in the afternoon.

Later in the afternoon, Clara Gonçalves (Inductiva.AI) brought together Inês Lynce (IST/UL & INESC-ID), José Nuno Oliveira (UMinho & ENSICO), João Ranito (Hands-on Investor) and Gonçalo Quadros (Critical Software) for a round table discussion on what the “next 30 years of EIC” might look like.

Two musical moments livened up the programme and delighted the audience. Sofia Sá, who took part in the first panel of the day with her testimony, included in the repertoire she brought to the auditorium a song of her own, composed especially for this celebration. “Engenharia Informática” is the result of three years of meaningful experiences that, according to Sofia, ‘will last a lifetime’. TUNAFE closed the programme with a joyful performance, something that they have accustomed us to over the years. There were many Informatics familiar faces, who, in addition to their computers, carry their love for music and TUNAFE with them on a daily basis.

The closing speech was given by Raul Moreira Vidal, Emeritus Professor of DEI/FEUP, and Alberto Silva, Secretary of State for Modernisation and Digitalisation.

From the auditorium to the lawn (where the tree planted to celebrate the EIC’s 20th anniversary has been growing for 10 years), the participants ended the evening in the best possible way, with an informal dinner party where conversations and memories brought smiles and promised new reunions.

This was accompanied by a performance by the band Júpiter, with disco funk in the style of Porto, and TECHUMAN, an audiovisual installation designed by Openfield Creative Lab that explores the symbiotic and ever-changing relationship between technology and humanity.

And of course there was plenty of birthday cake, sparkling wine and toasts to the long life of FEUP’s Informatics and Computing Engineering programme.

The party continued on Saturday 28th with a ‘Meet&Grill’ event organized by the AlumniEI Board, with the aim of bringing together former students from the various Informatics and Computing Engineering courses for a time of activity and socializing between several generations of EIC, an opportunity to catch up, meet new people and make plans for the future.

 

And speaking of the future, we invite you to read ‘Clarke’s Lunette‘, a text written by Fátima Vieira, Vice-Rector of Culture and Museums at the University of Porto, as part of this celebration.

This event was sponsored and supported by various partners listed on the event’s webpage.

The full recording of the event is available here.

DEI Talks | “Accelerating Implicit Mechanics” by Robert F. Lucas

Posted on

“Historically, the run time of implicit mechanics has been dominated by the time required to solve a large sparse linear system. The default solver is a multifrontal sparse matrix factorization, which will reliably solve ill conditioned, indefinite problems. The multifrontal method turns a sparse matrix factorization into a directed acyclic graph of smaller, dense “frontal” matrix factorizations, and these can be accelerated using Graphics Processing Units. As the number of processors used grows into the thousands, reordering the sparse matrix to reduce the storage and operations required to factor it, is the emerging computational bottleneck. Reordering is NP-complete, and in computational mechanics the preferred heuristic is nested dissection, i.e., recursive graph partitioning. Finding a balanced min cut is NP-hard, and classical codes such as ParMetis have limited parallel scaling. This talk will also discuss on-going work to explore a new generation of specialized devices for solving optimization problems. These include the D-Wave adiabatic quantum annealer, so called Silicon annealers produced by Fujitsu and Toshiba, the LightSolver Laser Processing Unit. The Digital Annealer is a dedicated chip that uses non-von Neumann architecture to minimize data movement in solving combinatorial optimization problems.”

“Accelerating Implicit Mechanics” will be presented October 10, at 15:00, room Vasco Sá (L119) – Sala de Atos do Departamento de Engenharia Mecânica.

“Dr. Robert F. Lucas received his BSc, MSc, and PhD degrees in Electrical Engineering from Stanford University in 1980, 1983, and 1988 respectively. He is currently an Ansys Fellow where he is responsible for the default multifrontal linear solver used in LS-DYNA and MAPDL. Previously, he was the Operational Director of the University of Southern California (USC) – Lockheed Martin Quantum Computing Center. Before joining USC, he was the Head of the High-Performance Computing Research Department in the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory and before that the Deputy Director of DARPA’s Information Technology Office. From 1988 to 1998 he was a member of the research staff of the Institute for Defense Analyses’s Center for Computing Sciences. From 1979 to 1984 he was a member of the Technical Staff of the Hughes Aircraft Company.”

Note: This talk is preceded by another talk, geared towards Mechanical Engineering and focusing on the use of ANSYS/LS-DYNA for modeling and simulation, by the same speaker at 14:00, in the same room, entitled “An Industrial Grand Challenge”. You are all welcome.

PhD Defense in Informatics Engineering: ”Enhanced multiview experiences through remote content selection and dynamic quality adaptation”

Posted on

Candidate
Tiago André Queiroz Soares da Costa

Date, Time and Location:
September 16, 14:30, Sala de Atos da Faculdade de Engenharia da Universidade do Porto

President of the Jury:
Doutor Carlos Miguel Ferraz Baquero-Moreno, Professor Catedrático da Faculdade de Engenharia da Universidade do Porto

Members:
Klara Nahrstedt, PhD, Full Professor, Department of Computer Science, University of Illinois at Urbana-Champaign, United States of America;
Pedro António Amado de Assunção, PhD, Coordinator Professor, Departamento de Engenharia Eletrotécnica, Escola Superior de Tecnologia e Gestão do Instituto Politécnico de Leiria;
Luís António Pereira de Meneses Corte-Real, PhD, Associate Professor, Departamento de Engenharia Eletrotécnica e de Computadores, Faculdade de Engenharia da Universidade do Porto;
Maria Teresa Magalhães da Silva Pinto de Andrade, PhD, Assistant Professor, Departamento de Engenharia Eletrotécnica e de Computadores, Faculdade de Engenharia da Universidade do Porto (Supervisor).

Abstract:
This thesis proposes a novel approach to immersive, multiview media distribution that uses Deep Learning models and user-centric data to predict user interest in the near future while multimediacontent is being presented. The main objective of this thesis is to give the user a truly ubiquitousmultimedia immersive experience without the need for expensive equipment, while also allowing him or her to see the scene being presented on the screen from almost any angle, as if they were actually there when the scene was shot. A methodological approach was envisioned based on the literature review and the identification of gaps in immersive streaming scenarios, which resulted in the conceptualization of a brand new architecture that was coined Smooth Multiview (SmoothMV). This architecture is capable of analysing user behaviour data in real-time and preemptively preparing content delivery accordingly based on viewing interests demonstrated by users while visualising a particular scene. Users effortlessly provide behaviour data without equiring intrusive equipment, which is then processed using the novel concept of the Hot&Cold matrix, which this thesis describes. With the use of this concept, the screen is divided into nine separate regions, each of which is connected to a neighbouring view that the SmoothMV architecture can prepare and choose to present. Separate queues designated for playback and buffering of upcoming content segments are introduced to provide minimal delay without compromising the user experience, since content adaptation is closely linked to user inputs. The number of views that are available and the approach employed for analysing behaviour while viewing content and selecting which view should be processed in the following moment affect how these queues are managed. This thesis developed from a purely reactive approach to a sophisticated, twofold Deep Learning architecture that can accurately identify the view that best fits the interests of the user with a high degree of accuracy. The development of a new dataset was needed in order to achieve this level of performance, as the data provided by existing datasets was not suitable for the scenario that was proposed. After a series of 128 experiments were conducted to collect visual attention data from 45 participants while viewing multi-perspective content, the Data2MV dataset was created and made available to the public. This thesis’ fundamental concepts and practical outputs are considered to be of significant importance to the body of knowledge currently available in the field of research, while also offering relevant tools for the general enhancement of current content distribution architectures.

Keywords: Multimedia, Streaming, Multiview, Focus-of-Attention, Deep Learning

DEI Talks | The Limitations of Data, Machine Learning & Us by Prof. Ricardo Baeza-Yates

Posted on

“Machine learning (ML), particularly deep learning, is being used everywhere. However, not always is used well, ethically and scientifically. In this talk we first do a deep dive in the limitations of supervised ML and data, its key component. We cover small data, datification, bias, predictive optimization issues, evaluating success instead of harm, and pseudoscience, among other problems.  The last part is about our own limitations using ML, including different types of human incompetence: cognitive biases, unethical applications, no administrative competence, copyright violations, misinformation, and the impact on mental health. In the final part we discuss regulation on the use of AI and responsible AI principles, that can mitigate the problems outlined above.”

“The Limitations of Data, Machine Learning & Us” will be presented September 10, at 11:00, room B032. Free entry but registration required here.

“Ricardo Baeza-Yates is the Director of Research at the Institute for Experiential AI of Northeastern University, as well as part-time professor at the Dept. of Computer Science of University of Chile. Before, he was VP of Research at Yahoo Labs, based in Barcelona, Spain, and later in Sunnyvale, California, from 2006 to 2016. He is co-author of the best-seller Modern Information Retrieval textbook published by Addison-Wesley in 1999 and 2011 (2nd ed), that won the ASIST 2012 Book of the Year award. From 2002 to 2004 he was elected to the Board of Governors of the IEEE Computer Society and between 2012 and 2016 was elected for the ACM Council. In 2009 he was named ACM Fellow and in 2011 IEEE Fellow, among other awards and distinctions. He obtained a Ph.D. in CS from the University of Waterloo, Canada, and his areas of expertise are responsible AI, web search and data mining plus data science and algorithms in general.”

Applications for the Prof. Doutor Raul Vidal/Deloitte Prize are now open

Posted on

The period for submitting applications for the Prof. Doutor Raul Vidal/Deloitte Prize has been running since July 19th. Students from the Master in Informatics and Computing Engineering and the Master in Software Engineering can apply until August 31st for the prize which is now in its third edition.

This award is intended to honour a recent graduate from one of these FEUP courses who has distinguished themselves in curricular activities, for the quality and innovation of the work carried out within the scope of Software Engineering, for their involvement in activities to support other students, namely in activities associated with FEUP’s student groups, and also for their involvement in social and solidarity initiatives.

The award was proposed by Deloitte, with the support of FEUP, through DEI, with the aim of honouring the Professor Emeritus of U.Porto in recognition of all his work at FEUP in the area of Informatics Engineering which has resulted in FEUP’s projection at national and international level and in the high-quality preparation of its students for the labour market, making FEUP unquestionably one of the leading schools with excellent technological teaching.

All the information on the application process and regulation can be found at: Prémio Prof. Doutor Raul Vidal – DEI – Departamento de Engenharia Informática (up.pt)

FC Portugal the big winner at RoboCUP 2024

Posted on

The Genneper Parken in Eindhoven hosted the 2024 edition of RoboCUP, the most exciting event in the world of autonomous robots. The joint team from the Universities of Porto and Aveiro, FC Portugal, was crowned Robotic Simulated Football champion for the third time in this competition. The final against the German team MagmaOffenburg saw FC Portugal win 5-3, a result that underlines the quality of the highly effective and cohesive Portuguese team.

A team of finalists from the L:IACD – Degree in Artificial Intelligence and Data Science, a joint course between the Faculties of Sciences (FCUP) and Engineering (FEUP) of the University of Porto, contributed to this victory. Tomás Azevedo and Francisco Gomes da Silva were present at this final of the 3D Simulation League – Humanoids.

Their victory at RoboCup 2024 is the result of Machine Learning (ML) and Deep Reinforcement Learning (DRL) methodologies used to train robotic agents, which are part of the themes covered at L:IACD. ML allows robots to learn and improve their performance based on data and previous experiences, while DRL focuses on teaching robots to make optimal decisions through interaction with the simulation environment.

In the context of the 3D Simulation League, these techniques were essential for developing agents capable of performing complex tasks such as team coordination, game strategy, advanced individual skills (such as shooting, sprinting and dribbling). The FC Portugal robots were trained to learn autonomously and optimise their behaviour on an ongoing basis, resulting in a winning team, which also won the Scientific Challenge (best innovative contribution to the progress of this league) and the ‘FatProxy’ Challenge (simulation of humanoids using high-level ‘move’ and ‘kick’ commands instead of direct control of the various degrees of freedom).

The team’s success is evidenced by the 30 international awards achieved and the publication of more than 100 scientific articles in indexed journals and conferences.

Brasil will be the next destination of RoboCUP which will bring the world’s competition enthusiasts to Salvador from 15 to 21 July 2025.

All the information at: https://2024.robocup.org/

Photo: Bart van Overbeeke

PhD Defense in Digital Media: “Narrative in Interactive Documentary: a Categorisation Framework”

Posted on

Candidate
Ana Sofia Airosa Coelho de Passos Baptista

Date, Time and Location
July 23, 4:30, Sala de Atos FEUP

President of the Jury
António Fernando Vasconcelos Cunha Castro Coelho (PhD), Associate Professor with Habilitation, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Members
Paulo Filipe Gouveia Monteiro (PhD), Full Professor, Departamento de Ciências da Comunicação, Faculdade de Ciências Sociais e Humanas, Universidade Nova de Lisboa;
Manuela Maria Fernandes Penafria (PhD), Associate Professor, Departamento de Artes da Faculdade de Artes e Letras, Universidade da Beira Interior;
Patrícia Nogueira da Silva (PhD), Assistant Professor, Departamento de Artes da Faculdade de Artes e Letras, Universidade da Beira Interior;
José Manuel Pereira Azevedo (PhD), Full Professor, Departamento de Ciências da Comunicação e da Informação, Faculdade de Letras, Universidade do Porto (Supervisor);
Hugo Daniel da Silva Barreira (PhD), Assistant Professor, Departamento de Ciências e Técnicas do Património, Faculdade de Letras, Universidade do Porto.

Abstract
Interactive Documentary offers innovative ways of telling reality-based stories, it reaches wider audiences more easily, and its availability, granted by the internet, seems durable. Nevertheless, we soon realised that their many possibilities came with as many obstacles. In addition to an even trickier production system, involving professionals with technological skills, a revenue model adapted to the web distribution, and the problem of technological obsolescence, interactive storytelling is a complex challenge. The openness inherent to interactivity affects how the story is constructed and experienced by the audience. Some interactive documentaries focus more on navigation rather than on the story. Narrative and storytelling are often neglected in research, as it tends to focus on the new affordances of interactivity. This journey aimed at understanding how interactive documentaries can balance the need for meaningful coherent stories with the advantage of interactivity and potential non-linearity and collaboration, through new narrative structures. We aim to support creators and researchers in the development and study of interactive documentaries, by identifying strategies and best practices regarding narrative and storytelling, based on literature review, case studies and interviews. For practical application, we propose a Categorisation Framework, illustrated with the case studies, which allows us to typify i-docs from the perspective of narrative and storytelling. Finally, we suggest a hands-on guide, comprising twelve tactics, for creators who aim to develop interactive documentaries with more meaningful and coherent narratives.

Keywords: Interactive Documentary; Narrative; Storytelling; Interactivity; Linearity; Categorisation.

PhD Defense in Digital Media (PDMD): ”Emotion-driven Physiological Actor Dynamics For Interactive Theatre Sound”

Posted on

Candidate
Luís Alberto Teixeira Aly

Date, Time and Location
July 22, 14:00, Sala de Atos, FEUP

President of the Jury
António Fernando Vasconcelos Cunha Castro Coelho (PhD), Associate Professor with Habilitation, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto.

Members
Javier Enrique Jaimovich Fernández (PhD), Associate Professor, Departamento de Sonido da Facultad de Artes, Universidad de Chile, Chile;
William Ruddock Primett (PhD), Postdoctoral Researcher, School of Digital Technologies, Tallinn University, Estónia;
Carla Maria de Jesus Montez Fernandes (PhD), Main Researcher, Instituto de Comunicação (ICNOVA), Faculdade de Ciências Sociais e Humanas, Universidade Nova de Lisboa;
Rui Pedro Amaral Rodrigues (PhD), Associate Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto;
Gilberto Bernardes de Almeida (PhD), Assistant Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto (Supervisor).

The thesis was co-supervised by Hugo Plácido da Silva (PhD), from Instituto Superior Técnico.

Abstract

This thesis, titled ’Emotion-driven Physiological Actor Dynamics For Interactive Theatre Sound,’ embarks on an exploratory journey into the innovative integration of physiological responses with emotional expression and sound design within theatre. This research investigates the intricate relationship between actors’ emotional states and physiological signals, delves into the impact of sound generated from physiological data on the actors’ emotional expression and agency, and examines how this novel integration can redefine traditional theatrical narratives and storytelling techniques. The study examines actors’ experiences and perceptions using qualitative and quantitative methodologies. It utilizes focus groups, observational studies, and sophisticated physiological sensors and surveys to capture and analyze the physiological signals and the feedback from actors. This approach allows for a nuanced understanding of the interplay between the physiological and emotional aspects of acting, shedding light on how actors embody and convey complex emotions through their performances. A key empirical contribution of this research is the DECEIVER dataset, which comprises extensive physiological recordings. These recordings provide valuable insights into the consistency and variability of emotional expression in performance settings. This dataset is a treasure trove for researchers and practitioners in the field, offering unprecedented detail and depth in understanding the physiological underpinnings of theatrical performance. Furthermore, the thesis presents a comprehensive historical analysis of the use of physiological sensors in interactive music, spanning the period from 1965 to 2023. This historical overview not only charts the technological evolution in this domain but also sets the stage for understanding the current trends and potential future developments. It contextualizes the research within the broader trajectory of technological advancements, highlighting the incremental and sometimes revolutionary changes that have shaped the current state of interactive music systems. The thesis introduces an empirical and functional taxonomy for Interactive Music Systems driven by physiological signals. This taxonomy represents a significant contribution to designing and applying physiological signals to interactive musical systems, providing a structured framework that can guide future developments in the field. It categorizes different approaches and methodologies in integrating physiological data into sound design, offering a comprehensive understanding of the potential and limitations of these systems. The research also involves the development of an extensive experimental protocol designed to analyze the physiological correlates of emotional valence and arousal in acting. A sophisticated software toolbox for data processing complements this protocol. The protocol’s design underscores the effectiveness of mental imagery in eliciting specific emotional states and highlights the complexity of emotional expression in theatre actors. This aspect of the research provides a methodological blueprint for future studies aiming to explore similar themes and questions. The Biosignal Processing Toolbox, a software tool for real-time operations integrating physiological signals with sound, is central to the study. The Biosignal Processing Toolbox enables the creation of dynamic, responsive soundscapes that interact with actors, enhancing the storytelling and engagement of the audience in the theatre. The toolbox is equipped to handle various physiological signals such as electromyography, electrocardiography, and electrodermal activity, each offering unique opportunities and challenges for sonification. The versatility of BarT lies in its ability to adapt and respond to different physiological inputs, making it an effective tool for sound designers in the theatre. A significant part of the research was a collaborative techno-artistic project, which utilized Samuel Beckett’s theatre as a backdrop. This project led to developing a prototype for an Interactive Music System driven by physiological sensors. This project explored the transformative possibilities of integrating physiological sensors and gesture typologies into theatre, providing fresh perspectives on character development and narrative construction. The project demonstrated the potential of this technology to bring a new dimension to theatrical performances, allowing for a more immersive and interactive experience for both actors and audiences. Despite its groundbreaking nature, the research acknowledges the challenges and limitations of such technological integrations. These include issues such as the need for real-time data processing, the necessity of actor-specific system calibration, technical and financial constraints, training requirements for actors and production teams, ensuring the comfort and unobtrusiveness of sensors during performances, ethical considerations related to the use of physiological data, and the subjective interpretation of such data in artistic contexts. In conclusion, this thesis contributes to theatre and interactive media art. Exploring the integration of physiological sensors in theatre sound design opens up new avenues for artistic expression and audience engagement. The development of the Biosignal Processing Toolbox and the DECEIVER dataset represent significant advancements in the field, paving the way for more immersive, interactive, and expressive forms of storytelling. This research provides novel perspectives for sound design and actor training and contributes to the broader discourse on the intersection of technology and art.