Candidate
Yassine Baghoussi

Date, Time and Place
May 29, at 09:30, Sala de Atos FEUP

President of the Jury
Pedro Nuno Ferreira da Rosa da Cruz Diniz, PhD, Full Professor, Department of Informatics Engineering, Faculty of Engineering, University of Porto.

Members
Joydeep Chandra, Phd, Associate Professor, Department of Computer Science and Engineering, Indian Institute of Technology, Patna, Índia;
Mykola Pechenizkiy, PhD, Full Professor, Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlands;
Luís Filipe Pinto de Almeida Teixeira, PhD, Associate Professor, Department of Informatics Engineering, Faculty of Engineering, University of Porto;
João Pedro Carvalho Leal Mendes Moreira, PhD, Associate Professor, Department of Informatics Engineering, University of Porto (Supervisor).

The thesis was co-supervised by Professor Carlos Manuel Milheiro de Oliveira Pinto Soares, Associate Professor at the Department of Informatics Engineering, University of Porto.

Abstract

“Machine Learning (ML) relies on both data and algorithms for optimal functioning. While conventional ML research often emphasizes algorithmic improvements, the significance of data processing is frequently overlooked. In contrast, preprocessing data stands as a distinct task, executed before feeding it into algorithms. The diversity of preprocessing methods tailored for various ML algorithms underscores its importance. However, the feedback loop between algorithms and data is often neglected. Data-related issues pose significant challenges for predictive ML algorithms, adversely affecting forecasting accuracy. These challenges arise because problems in data are inherently unpredictable, lacking a discernible pattern. In this doctoral thesis, we introduce Read and Write Machine Learning (RW-ML), an innovative paradigm enhancing time series forecasting accuracy by integrating data modification techniques into the learning process. The RW-LSTM, an adaptation of the backpropagation algorithm, unifies preprocessing with recurrent neural networks (RNNs), significantly outperforming traditional models such as LSTM. RW-LSTM enables the transition from read-only RNNs, which merely learn from data, to RW-ML, allowing direct alterations for improved predictions. Expanding the framework, the Corrector Long Short-Term Memory (cLSTM) addresses the limitations of read-only RNNs, demonstrating enhanced forecasting accuracy through empirical verification and extensive experiments. The final chapter provides a real-world evaluation, highlighting the competitive advantage of cLSTM over LSTM models in various scenarios.”

This research was carried out as part of SonaeIM.Lab@FEUP, involving Inovretail.

Candidate
Simão Paulo Rato Alves Reis

Date, Time and Place
January 11, 14:00, Sala de Atos FEUP

President of the Jury
Carlos Miguel Ferraz Baquero-Moreno, PhD, Full Professor, Department of Informatics Engineering, Faculdade de Engenharia, University of Porto

Members
João Alberto Fabro, PhD, Associate Professor, Departamento Acadêmico de Informática, Universidade Tecnológica Federal do Paraná, Brasil;
Rui Filipe Fernandes Prada, PhD, Associate  Professor, Instituto Superior Técnico, Universidade de Lisboa;
Pétia Georgieva Georgieva, PhD, Associate Professor with Habilitation, Department of Electronics, Telecommunications and Informatics, Universidade de Aveiro (representative of the MAP-i Scientific Committee);
Luís Paulo Gonçalves dos Reis, PhD, Associate Professor with Habilitation, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto (Supervisor);
Henrique Daniel de Avelar Lopes Cardoso, PhD, Associate Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto.

The thesis was co-supervised by Doutor Nuno Lau, Associate Professor at the University of Aveiro.

Abstract:
“This thesis studies the balance problem in game development, notably in two-player games. Specifically, we aim to study the viability of Artificial Intelligence (AI) as an assisting tool to fix game properties. We split our research into two paths: Power Balance, where the aim is to adjust game strategies, so they become effective as winning tools; Difficulty Balance, where the objective is to adjust game attributes on the fly so that weaker players or players at a disadvantage can compete with stronger players or players in advantage. Both domains require tuning the game, but they mainly differ in the timing and in their aim, one deals with the imbalance in game design, while the other deals with inequality in player skills. For Power Balance, our methodology was to define a full meta-game balance ecosystem based on the Pokémon video game franchise and develop an AI competition where the multiple associated tasks (battling, team prediction and assembly, and meta-game balance) are present and can be tested in a common ground. To balance the meta-game, we follow an adversarial model where team builders aim to narrow the use of optimal Pokémon while balancer agents aim to incentive the maximum of Pokémon as possible to be selected by team builders. This results in agents being able to play, build effective teams, and being able to tune the Pokémon roster over time. We discuss how our models can be extended to other video game domains. For Difficulty Balance, we propose a Multiplayer Dynamic Difficulty Adjustment framework where a Game Master (GM) agent is trained and embedded into a game, depending on the game state it will deploy handicap mechanisms. The training regime follows a specific pipeline. To generalize advantage situations, parameterized perturbations on the actions of a reference player are used to emulate several degrees of playing skill, and the advantage for each player is used to draw curves which are evaluated as a reward for the GM. This results in the GM being able to optimize game design criteria and create opportunities for the player behind to recover. We show there are suited AI tools for each task, and it is reasonable to think of power balance and difficulty as separate problems, where both can be automatically assisted and eased. Both further augment our overall comprehension of the automated game balance field.”

Candidate:
Gil Filipe da Rocha

Date, Time and Place
October 2, 14:00, Room Professor Joaquim Sarmento (G129), DEC, FEUP

President of the Jury
Rui Filipe Lima Maranhão de Abreu, PhD, Full Professor, Faculdade de Engenharia, Universidade do Porto

Members
Hugo Ricardo Gonçalo Oliveira, PhD, Associate Professor, Department of Computer Science, Faculdade de Ciências e Tecnologia, Universidade de Coimbra;
Bruno Emanuel da Graça Martins, PhD, Associate Professor, Department of Electrical and Computer Engineering, Instituto Superior Técnico, Universidade de Lisboa;
Eugénio da Costa Oliveira, PhD, Emeritus Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto;
Sérgio Sobral Nunes, PhD, Associate Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto;
Henrique Daniel de Avelar Lopes Cardoso, PhD, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto (Supervisor).

 Abstract
“The aim of argumentation mining is to automatically detect, identify and extract arguments from natural language text. The end goal is to provide a structured representation of the arguments (argument diagrams) that can be automated and analyzed in many different ways. Argumentation is a rhetorical act that has been studied for centuries and has been influenced by different research fields such as philosophy, linguistics, computer science, and artificial intelligence. In general, arguments are justifiable positions where pieces of evidence (premises) are offered in support of a claim (conclusion). Some characteristics of natural language text and, more specifically, of argumentation exposition make argumentation mining a complex task. Indeed, the ambiguity of natural language text, the assumption of commonsense reasoning and implicit knowledge, different writing styles, and the inherent complexity of argument diagrams are some of the challenges that argumentation mining systems have to overcome. Addressing these challenges, especially across different languages and text genres, demands robust argumentation mining systems. In this thesis, we conduct research toward the development of a robust computational system that can be employed to detect, identify, and extract argumentative content across different languages and text genres. Our vision is to deploy such a system to address argumentation mining in less-resourced languages (such as Portuguese) and in text genres that feature high-variability of argument exposition profiles (such as opinion articles). To this end, we combine techniques from computational linguistics and machine learning with knowledge from argument structure and rhetoric theories to automatically identify argumentative reasoning in natural language texts. To study argumentation mining in a less-resourced language and a challenging text genre, we conduct an annotation study to create a corpus annotated with arguments from opinion articles written in Portuguese. To address the challenging argumentation mining task, we propose a relation-based approach and context-aware models motivated by argumentation theory foundations and tailored to overcome some of the challenges of argument exposition. To tackle this task in a less-resourced language, we investigate how cross-language learning techniques can be employed to explore annotated resources from different languages and improve the performance of machine learning models in a target language. Finally, to improve the robustness of argumentation mining systems across different genres, we leverage recent advancements in language modeling capabilities and propose a cross-genre approach for argumentation mining.”

Candidate:
António Humberto e Sá Pinto

Date, Time and Place:
September 8, 14:30, Sala de Atos FEUP

President of the Jury:
António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with Habilitation, Faculdade de Engenharia da Universidade do Porto;

Members:
Magdalena Fuentes, PhD, Assistant Professor, Music and Audio Research Lab (MARL) and Integrated Design & Media (IDM), New York University (NYU);
Jason Hockman, PhD, Associate Professor, School of Computing and Digital Technology (DMT), Birmingham City University (UK);
Matthew Edward Price Davies, PhD, Senior Scientist,  SiriusXM/Pandora (USA) – (Supervisor);
Rui Pedro da Silva Nóbrega, PhD, Assistant Professor, Departamento de Informática, Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa;
Aníbal João de Sousa Ferreira, Associate Professor, Departamento de Engenharia Eletrotécnica e de Computadores, Faculdade de Engenharia da Universidade do Porto.

The thesis was co-supervised by Prof Rui Luís Nogueira Penha, Coordinating Professor at ESMAE, and Prof Gilberto Bernardes de Almeida, Assistant Professor at FEUP.

Abstract:

“Music Information Retrieval (MIR) is an interdisciplinary field focused on the extraction, analysis, and processing of information from various musical representations.
Grounded on the automatic analysis of musical facets such as rhythm, melody, harmony, and timbre, MIR enables applications in areas like music recommendation, automated music transcription, and intelligent music composition tools. Rhythm, an integral element of music, provides a foundation for decoding music’s complex relational structures and layered depth. Computational rhythm analysis is thus central to MIR research. It encompasses a wide range of tasks, such as the pivotal beat tracking, which unlocks the use of musical time across many MIR systems. However, conventional beat-tracking methods have struggled when dealing with complex musical features, such as expressive timing or intricate rhythmic patterns. While specialized approaches demonstrate some degree of adaptation, they do not generalise to diverse scenarios. Deep learning methods, while promising in addressing these issues, depend heavily on the availability of substantial annotated data. In scenarios requiring adaptation to user subjectivity, or where acquiring annotated data is challenging, the efficacy of beat-tracking methods lowers, thus leaving a gap in the applicability of computational rhythm analysis methods. This thesis investigates how user-provided information can enhance computational rhythm analysis in challenging musical conditions. It initiates the exploration of human-in-the-loop strategies with the aim of fostering adaptability of current MIR techniques. By focusing on beat tracking, due to its fundamental role in rhythm analysis, our goal is to develop streamlined solutions for cases where even the most advanced methods fall short. This is achieved by utilising both high-level and low-level user inputs —- namely, the user’s judgement regarding the expressiveness of the musical piece and annotations of a brief excerpt —- to adapt the state of the art to abstract particularly demanding signals. In an exploratory study, we validate the shared perception of rhythmic complexity among users as a proxy for musical expressiveness, and consequently as a key performance enhancer for beat tracking. Building upon this, we examine how highlevel user information can reparameterise a leading-edge beat-tracker, augmenting its performance to highly expressive music. We then propose a transfer learning method that finetunes the current state of the art, hereafter referred to as the baseline, to a concise user-annotated region. This method exhibits versatility across varied musical styles and offers potential solutions to the inherent limitations of previous approaches. Incorporating both user-guided contextualisation and transfer learning into a human-in-the-loop workflow, we undertake a comprehensive evaluation of our adaptive techniques. This includes examining the key customisation options available to users and their effect on performance enhancement. Our approach outperforms the current state of the art, particularly in the challenging musical content of the SMC dataset, with an improvement over the baseline F-measure of almost 10 percentage points (corresponding to over 16%). However, these quantitative improvements require further interpretation due to the inherent differences between our file-specific, human-in-the-loop technique and traditional dataset-wide methods, which operate without prior exposure to specific file characteristics. With the aim of advancing towards a user-centric evaluation framework for beat tracking, we introduce two novel metrics: the E-Measure and Annotation Efficiency. These metrics account for the user perspective regarding the annotation and finetuning process. The E-Measure is a variant of the F-measure focused on the annotation correction workflow and includes a shifting operation over a larger tolerance window. The Ae is defined as the relative (to the baseline) decrease in correction operations enabled by the fine-tuning process, normalised by the number of user annotations. Specifically, we probe the theoretical upper bound of beat tracking accuracy improvement over the SMC dataset. Our results show that the correct beat estimates provided by our approach surpass those of the state of the art by more than 20%. When considering the full length of the files, we can further frame this improvement in terms of gain per unit of user effort, quantifying the annotation efficiency of our approach. This is reflected in the substantial reduction of required corrections, with nearly 2/3 fewer corrections per user annotation compared to the baseline. In the final phase, we evaluate our human-in-the-loop strategy’s adaptability across a range of musical genres and instances presenting significant challenges. Our exploration extends to various rhythm tasks, including beat tracking, onset detection, and (indirectly) metre analysis. We apply this user-driven strategy to three unique genres with complex rhythm structures, such as polyrhythms, polymetres, and polytempi. Our approach exhibits swift adaptability, enabling efficient utilisation of the state-of-the-art method while bypassing the need for extensive retraining. This results in a balanced integration of data-driven and user-centric methods into a practical and streamlined solution.”

Keywords: Music Information Retrieval; User-centric; Transfer Learning; Beat Tracking.

Requested by:
Joana Rodrigues da Silva

Date, time and place
July 19, 14h30, room L119 DEMEC (FEUP)

President of the Jury:
António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with Habilitation, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Members:
Teresa Isabel Lopes Romão, PhD, Associate Professor, Departamento de Informática, Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa;
Luís António Santos, PhD, Assistant Professor, Departamento de Ciências de Comunicação, Instituto de Ciências Sociais da Universidade do Minho;
Miguel Ângelo Rodrigues Midões, PhD, Invited Adjunct Professor, Departamento de Comunicação e Arte, Escola Superior de Educação do Instituto Politécnico de Viseu;
Helena Laura Dias de Lima, PhD, Associate Professor, Departamento de Ciências da Comunicação e da Informação, Faculdade de Letras da Universidade do Porto (Supervisor);
Alexandre Miguel Barbosa Valle de Carvalho, PhD, Assistant Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Abstract:

“After the COVID-19 epidemic and the consequent humanitarian crises that devastated the planet, there is a need to claim the role of investigative journalism as a watchdog and permeation system of social justice and democracy through public exposure. We are witnessing a sharp decrease in investment in this journalism specialty, either because of its impertinence in dealing with public management issues or because of the time spent on this type of investigation, which fundamentally takes longer than current journalism to produce results. In this sense, we perceive the influence of automation and artificial intelligence in information production processes to highlight all human tasks with the possibility of being carried out in less time by technological systems. Considering this possibility, there was an interest in studying, in-depth, how robotics and the application of artificial intelligence through platforms that support the usual procedure of journalism can help and even improve the global state of investigative journalism practice, nowadays. The Connect-the-Dots platform and the DODO assistant emerge as a hypothetical digital solution for some of the problems that investigative journalism currently faces. It could be a way to practically implement investigative journalism in its scope of innovation by integrating tools and open-source practices in a Design-Based-Research approach to knowledge archaeology.”

Keywords: Artificial Intelligence, Automation, Investigative Journalism, Design-Based-Research, Digital Media.