Candidate:
Tiago Filipe Mendes Neves

Date, Time and Location:
16 September 2025, 15h30, Sala de Atos, Faculdade de Engenharia da Universidade do Porto

President of the Jury:
Pedro Nuno Ferreira da Rosa da Cruz Diniz (PhD), Full Professor, Department of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto

Members:
Keisuke Fujii (PhD), Associate Professor, Department of Intelligent Systems, Graduate School of Informatics of the Nagoya University, Japan;
Jesse Jon Davis (PhD), Full Professor, Department of Computer Science, Faculty of Engineering Science, Katholieke Universiteit Leuven, Belgium;
Luís Paulo Gonçalves dos Reis (PhD), Associate Professor with Habilitation, Departament of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto;
João Pedro Carvalho Leal Mendes Moreira (PhD), Associate Professor, Departament of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto (Supervisor).

The thesis was co-supervised by Luís Jorge Machado da Cunha Meireles (PhD), Senior Psychologist & Data Scientist, FC Porto.

Abstract:

Self-supervised large models that disrupt domains such as language, vision, and biology are transforming the world. However, these generative models that learn the underlying data distribution do not perform at the same level on all tasks. For example, Large Language Models (LLMs) do not yet have concrete applicability in soccer analytics. The models lack reasoning capabilities to provide concrete and actionable insights that can compete with the wide range of case-specific metrics within soccer analytics. While there have been some studies exploring the applicability of generative models in soccer, no study aimed for the moonshot of building a complete self-supervised learning model for soccer event data. Let’s consider the individual events (each shot, pass, tackle, …) in a soccer match the “words” that describe what is happening. We can consider each possession a “sentence,” each game an “essay,” and event data as a whole a “language.” By working within this framework, we have all the tools to build a self-supervised model in the same image as LLMs. The goal of this thesis is to build a foundation self-supervised model for soccer event data – termed Large Events Model (LEM) – and demonstrate its real-world applicability and generality in solving a wide range of tasks, such as simulation and modeling, that would otherwise require multiple different approaches. We propose three approaches to building LEMs: a chain of classifiers, causal mask modeling, and sequential language modeling with transformers. First, the chain of classifiers provides the first generative model that models all aspects of event data without posing restrictions on event types, reaching a level of performance that allows large-scale simulation of soccer matches. Then, we investigate two alternative approaches to remove some of the constraints of the first approach. The causal mask modeling approach using multilayer perceptrons reaches the state-of-the-art performance of several of our proposed benchmarks, providing a set of application-ready models to solve a wide range of soccer analytics tasks. We explore a wide range of applications, from automated strategy search with reinforcement learning to risk-reward behaviors of soccer players. More than a dozen use cases for LEMs are present in this thesis. The implications of our work are far-reaching. LEMs have the potential to become the operating system for event data in soccer analytics. They will transform the way clubs work, with easier access to machine learning models that would otherwise require tremendous modeling effort. With LEMs, the barrier to entry will lower significantly as any club in the world can access a model capable of solving its most relevant problems.

Keywords: generative models; foundation models; sports analytics; deep learning applications; simulation; soccer.

Candidate:
Gabriel de Jesus

Date, Time and Location:
1 September 2025, 14:30, Sala de Atos, Faculdade de Engenharia da Universidade do Porto

President of the Jury:
Rui Filipe Lima Maranhão de Abreu (PhD), Full Professor, Departament of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto

Members:
Arjen P. de Vries (PhD), Full Professor at the Institute for Computing and Information Sciences of the Radboud Universiteit, Nimega, The Netherlands;
Bruno Emanuel da Graça Martins (PhD), Associate Professor, Departament of Electrical and Computer Engineering, Instituto Superior Técnico da Universidade de Lisboa;
Henrique Daniel de Avelar Lopes Cardoso (PhD), Associate Professor, Departament of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto;
Sérgio Sobral Nunes (PhD), Associate Professor, Departament of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto (Supervisor).

Abstract:

Ensuring access to information in all languages is crucial for bridging disparities in communities’ participation in the digital age and fostering a more inclusive and equitable society, particularly for speakers of low-resource languages. However, enabling such access remains a significant challenge for many of these communities. Tetun, a language that transitioned from a dialect to one of Timor-Leste’s official languages when the country restored its independence in 2002, faces similar challenges. According to the 2015 census, Tetun is spoken by approximately 79% of the country’s 1.18 million population. Despite its official status, Tetun remains underserved in language technology. Specifically, information retrieval-based solutions for the language do not exist, making it challenging to find relevant information on the internet and digital platforms for text-based search in Tetun.
This work tackles these challenges by investigating retrieval strategies for text-based search that can enable the application of information retrieval techniques to develop search solutions for Tetun, with a specific focus on the ad-hoc text retrieval task. Given that language-specific algorithms, tools, and document collections for Tetun were previously unavailable, this work began by creating these foundational resources, which serve as contributions relevant to information retrieval and natural language processing domains. These resources include a tokenizer, a language identification model, a stemmer, a stopword list, a document collection, a test collection, baselines for the ad-hoc text retrieval task, and a search log dataset. The contributions to information retrieval for low-resource languages include: (1) A data collection pipeline tailored for low-resource languages to streamline the construction of textual data from the web; (2) A human-in-the-loop methodology for annotating, processing, and constructing a dataset well-suited for a variety of information retrieval and natural language processing tasks; (3) A novel network-based approach for stopword detection; (4) Methodologies for developing a stemmer, designed for a language heavily influenced by loanwords, and the construction of a ground truth set for evaluating stemmer performance; (5) A detailed approach for constructing a test collection to evaluate the effectiveness of retrieval systems; (6) A methodology for establishing a robust baseline for the ad-hoc text retrieval task; and (7) Document contextualization and dual-parameter tuning strategies for hybrid text retrieval. The results from this work contribute to the development of technologies associated with the computational processing of Tetun, address gaps in its linguistic resources, and achieve impactful outcomes that elevate Tetun’s status. These advancements open new opportunities for future research and innovation. Moreover, this work introduces promising methodologies that can be adapted to other languages facing similar challenges, thereby contributing to the broader advancement of information retrieval for low-resource languages.

Candidate: Martin Joseph Aubard

Date, time and location:
25 July 2025, 14:00, Sala de Atos DEEC – I-105, Faculty of Engineering, University of Porto

President of the Jury:
Pedro Nuno Ferreira da Rosa da Cruz Diniz (PhD), Full Professor, Department of Informatics Engineering, Faculty of Engineering, University of Porto

Members:
Bilal Wehbe (PhD), Senior Researcher at the German Research Center for Artificial Intelligence, Germany;
Catarina Helena Branco Simões da Silva (PhD), Associate Professor, Department of Computer Engineering, Faculty of Science and Technology, University of Coimbra;
Andry Maykol Gomes Pinto (PhD), Associate Professor, Department of Electrical and Computer Engineering, Faculty of Engineering, University of Porto;
Ana Maria Dias Madureira Pereira (PhD), Coordinating Professor with Aggregation, Department of Computer Engineering, Instituto Superior de Engenharia do Porto, Instituto Polítécnico do Porto (Supervisor).

The thesis was co-supervised by Luís Filipe Pinto de Almeida Teixeira (PhD), Associate Professor in the Department of Informatics Engineering at the Faculty of Engineering of the University of Porto.

Abstract:

This thesis addresses the challenge of improving Autonomous Underwater Vehicles (AUVs) onboard detection and interaction capabilities using Side-Scan Sonar (SSS) data. Traditionally, underwater missions relied on pre-defined plans where data are analyzed post-mission by operators or experts. This workflow is time-consuming, often requiring multiple missions to identify and localize underwater targets. The need for repeated missions increases operational costs and complexity, highlighting the inefficiency of current methodologies. Moreover, such approaches do not allow the AUV to interact with detected targets in real time, limiting the scope of mission adaptation and real-time decision-making. To overcome these limitations, this thesis presents a novel framework integrating deep learning models for object detection directly onboard AUVs. This integration enables the vehicle to detect, localize, and interact with underwater targets in real time, offering significant improvements over traditional post-mission analysis. The framework builds upon the LSTS toolchain, which is responsible for AUV motion control and communication, and introduces enhanced real-time data processing capabilities. However, implementing such a model into an embedded system suffers from computational limitations affecting the model’s performance. Thus, the knowledge distillation methods have been implemented, ensuring smaller, more efficient models to perform onboard detection without sacrificing accuracy. Additionally, to improve the model’s robustness against underwater noise, a novel adversarial retraining framework, ROSAR, is introduced, ensuring reliable operation even in noisy sonar environments. Following the onboard detection and localization enhancement, we focused on onboard interaction with the detected object. This is realized by extending the previous onboard framework and validating it through a customized simulator, enhancing interaction with the detected objects, and validating through a pipeline inspection use case, which reduces mission time by combining sonar detection and camera data collection in a single mission, utilizing behavior trees and safety-assessed models. Given the lack of open-source sonar datasets in the field, this thesis contributes to two novel publicly available side-scan sonar datasets, SWDD and Subpipe, which include field-collected data on walls and pipelines and are manually annotated for object detection. By shifting from post-mission analysis to real-time detection and interaction, this thesis significantly improves the operational efficiency of AUV missions. The proposed framework streamlines underwater operations and enhances AUVs’ autonomous behavior, relying on efficient, accurate, and robust object detection model for efficient underwater exploration and monitoring applications.

Candidate:
Filipa Marília Monteiro Ramos Ferreira

Date, time and location:
23 July 2025, 14:30, Sala de Atos, Faculty of Engineering, University of Porto

President of the Jury:
Carlos Miguel Ferraz Baquero-Moreno (PhD), Full Professor, Department of Informatics Engineering, Faculty of Engineering, University of Porto

Members:
Tiago Manuel Lourenço Azevedo (PhD), Associate Researcher, Department of Computer Science and Technology, University of Cambridge, United Kingdom;
Marco António Morais Veloso (PhD), Coordinating Professor, Department of Science and Technology, Oliveira do Hospital School of Technology and Management, Polytechnic Institute of Coimbra;
Luís Filipe Pinto de Almeida Teixeira (PhD), Associate Professor, Department of Informatics Engineering, Faculty of Engineering, University of Porto;
Rosaldo José Fernandes Rossetti (PhD), Full Professor, Department of Informatics Engineering, Faculty of Engineering, University of Porto (Supervisor).

Abstract:

Ensuring the reliability and robustness of deep learning remains a pressing challenge, particularly as neural networks gain traction in safety-critical applications. While extensive research has focused on improving accuracy across datasets, generalisation, interpretability and robustness in the deployment domain remain poorly understood. In fact, in real-world scenarios, models often underperform without clear explanations. Addressing these concerns, uncertainty quantification has emerged as a key research direction, offering deeper insight into neural networks and enhancing confidence, interpretability, and robustness. Among critical applications, self-driving vehicles stand out, where uncertainty-aware object detection can significantly improve perception and decision-making. This thesis explores interpretations of uncertainty tailored to object detection in the context of self-driving vehicles. In this sense, two novel methods to estimate the aleatoric component and one approach to modelling the epistemic uncertainty are proposed. Through the utilisation of anchor distributions readily available in any anchor-based object detector, uncertainty is estimated holistically while avoiding costly sampling procedures. Further, the concept of existence is introduced, a probability measure that indicates whether an object truly exists in the real-world, regardless of classification. Building upon these ideas, three applications of uncertainty and existence are explored, namely the Existence Map, the Uncertainty Map and the Existence Probability. Whilst the aforementioned maps encode the existence measure and the aleatoric uncertainty over the space of input samples, the Existence Probability merges the information provided by the Existence Map with the standard detections, supplementing model outputs. Evaluation showcases the coherence of uncertainty estimates and demonstrates the usefulness of the Existence and Uncertainty Map in supporting the standard model, providing open-set capabilities and giving a degree of confidence to true positives, false positives and false negatives. The merging strategy of the Existence Probability reports a considerable improvement in the performance of the object detector both in validation and perturbation, while detecting all classes of the dataset despite being trained only on cars, pedestrians and cyclists. The second part of this thesis features a study of the underspecification distribution and its connection with the epistemic uncertainty. Underspecification, recently coined, greatly endangers deep learning deployment in safety-critical systems as it depicts the variability of predictors generated by a single architecture with increasingly diverging performance in the application domain. The analysis performed showcases that, if the uncertainty estimates are correctly calibrated, a single predictor is sufficient to predict the spread of the underspecification distribution, avoiding running repeated costly training sessions. All proposed methods are designed to be model-agnostic, real-time compatible, and seamlessly applicable to deployed models without requiring retraining, underscoring their significance for robust and interpretable object detection in autonomous driving.