Category Archives: PhD Defenses

PhD Defense in Digital Media: ”Narratives of Our Age: Intergenerational Digital Storytelling and Cultural Identity”

Posted on

Candidate
Juliana Carolina Campos Monteiro

Date, Time and Location 
June 4, 14h30, Room Professor Joaquim Sarmento (G129), DEC, FEUP

President of the Jury
António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with Habilitation, Faculdade de Engenharia, Universidade do Porto.

Members
Paulo Nuno Gouveia Vicente, PhD, Associate Professor, Departamento de Ciências da Comunicação, Faculdade de Ciências Sociais e Humanas, Universidade Nova de Lisboa;
Daniel da Cruz Brandão, PhD, Assistant Professor, Departamento de Ciências da Comunicação, Instituto de Ciências Sociais, Universidade do Minho;
Teresa Margarida Loureiro Cardoso, PhD, Assistant Professor, Departamento de Educação e Ensino à Distância, Universidade Aberta;
Dinis Miguel de Almeida Cayolla Ribeiro, PhD, Assistant Professor, Departamento de Ciências da Arte e do Design, Faculdade de Belas Artes, Universidade do Porto;
Carla Susana Lopes Morais, PhD, Assistant Professor, Departamento de Química e Bioquímica, Faculdade de Ciências, Universidade do Porto (Supervisor).

Abstract
The expansion of digital tools and participatory media created unprecedented possibilities for the maintenance and sharing of social memory and cultural identity knowledge. These possibilities are expanding in parallel with a context of an increasingly aged society, where elders are privileged keepers of regional cultural knowledge but often don’t have the opportunity to pass it on to future generations, making it prone to shortly disappear. This research approaches digital storytelling during intergenerational exchanges as a stage for a participatory contribution to the maintenance of cultural identity. We sought to determine how intergenerational dynamics can give place to cultural identity narratives, as well as how digital media can support the maintenance of cultural identity. For that purpose, we promoted the project NOOA: Narratives Of Our Age, with the premise of bringing generations together in sharing stories on topics such as Memories, Crafts, Myths, and Traditions, to endorse the continuity of the cultural identity of the Vale do Sousa region, using the potential of digital media in this process. With an action research design and an ethnographic approach, from 2018 to 2021, this project reunited three intergenerational groups, composed of participants aged 16 to 85 years old, in a set of activities designed to foster intergenerational dynamics. We sought to gather qualitative perceptions about the phenomenon of storytelling in the participatory maintenance of cultural identity through intergenerational dynamics. To this end, our research was based on data collected through detailed observations and through the application of semi-structured interviews and group discussions. We developed a five-step intergenerational storytelling framework to encourage the exchange of cultural identity knowledge, including digital registration, dissemination, and discussion in-person and online. This framework was submitted to a cyclical evaluation and its adjustments, in accordance with the action-research method, and it was implemented in three phases: phase one corresponded to a group dynamic component, through brainstorming and story planning; phase two corresponded to the creation of digital stories; phase three corresponded to dissemination, with the publication and sharing of these stories in the project’s digital spaces. These steps were conducted through in-person digital storytelling activities, in which cultural identity knowledge was shared and discussed in intergenerational partnerships, resulting in the participatory creation of 35 digital stories, that were afterwards shared and also discussed on the Project’s digital spaces. The outcomes of the activities, as well the enthusiasm and the reflections that occurred during the process, highlight the investment of participants in the transmission and maintenance of knowledge related to their experiences and knowledge acquired throughout life, in a family and community context. The observations of intergenerational dynamics, as well as the results of the semi-structured interviews, suggested a growing curiosity, involvement and overall understanding and retention of the stories of cultural identity, by both seniors and juniors, as well as a growing curiosity of senior participants regarding the pervasive digital reality presented to them by the juniors. Likewise, we observed the importance of empowering groups with spaces of sharing, both face-to-face and online, to enhance intergenerational exchanges around the knowledge of cultural identity. By promoting opportunities for intergenerational dialogue and empowering groups with face-to-face and online sharing spaces, we contributed with the development and application of a storytelling framework and digital spaces to safeguard, discuss and disseminate some of the specific cultural knowledge of the region of Vale do Sousa in Portugal. We observed the potential of using digital storytelling dynamics to break through barriers of communication between generations and to perpetuate and value cultural identity knowledge, acquired throughout life. This research goes beyond the aim of documenting the envisaged cultural identity knowledge, by combining creativity during the digital storytelling processes with connectivity among the participants on a community level, by sparking the dialogue between generations and enhancing the social impact of cultural identity knowledge and the self-knowledge value that generations are able to share through stories. We examined the opportunities and challenges of digital media as a platform and a catalyst for cultural identity maintenance, situating the problematic of cultural literacy in a contemporary setting. We conducted a thorough assessment of the stakeholders involved in participatory cultural identity maintenance in our present context, adding the observation of their synergies in a real context. This allowed us to examine the diversity of outcomes and the multiplicity of variables that contribute to it, as well as to grasp the impact that this new information flow paradigm may have on how we currently approach cultural identity maintenance.

Keywords: Intergenerational Digital Storytelling, Cultural Identity, Cultural Literacy, Participatory Action-Research.

PhD Defense in Digital Media: ”Computing by going back in time: Composing video sequences through multimodal generative coordination”

Posted on

Candidate:
Luís Henrique Pinto Arandas

Date, Time and Place:
June 03, 14:30, Sala de Atos FEUP

President of The Jury:
António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with Habilitation, Faculdade de Engenharia, Universidade do Porto.

Members:
Luísa Maria Lopes Ribas, PhD, Assistant Professor, Departamento de Design de Comunicação, Faculdade de Belas-Artes, Universidade de Lisboa;
David Fernandes Semedo, PhD, Assistant Professor, Departamento de Informática, Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa;
André Sier, PhD, Invited Assistant Professor, Departamento de Artes Visuais e Design, Universidade de Évora;
José Miguel Santos Araújo Carvalhais Fonseca, PhD, Full Professor, Faculdade de Belas Artes, Universidade do Porto (Supervisor);
Gilberto Bernardes de Almeida, PhD, Assistant Professor, Departamento de Engenharia Informática, Faculdade de Engenharia, Universidade do Porto.

The thesis was co-supervised by Professor Mick Grierson, Professor in Computing, and research leader at the Institute of Creative Computing at the University of the Arts, London.

Abstract
This project proposes a set of methods, inspired by the human experience of vision and time, for developing video sequences using trained generative models. The methods serve the production of video sequences, with patterns derived from trained models found in literature on metacreation and in the artworld. This project defines possible futures where the now pervasive generative models can be reused in computer simulations that focus on the human experience of video and mental images; models which, because of how they are trained through archives and records representing both the human and the physical world, can capture media themselves and represent specific moments in time.

The research outputs are in film and audiovisual installation, proposing that practice can further benefit from self-reference, using deep generative models as synthesisers of video, sound, and text. The methods produced take advantage of natural language guidance and deep generative models in ways that can be understood as sampling, sequencing, and translation, following computing literature and AI design. Each result can be understood in larger domains such as: 1) short films from text inputs, in the film Irreplaceable Biography; 2) discursive installations from video datasets, in the installation Time as meaning; and 3) short films from video inputs, in the film Man lost in the convergence of time and the collaboration all YIN no YANG. This research extends on the use of generative practice following a construct of language in the human mind, behaviour, and visual experience as inspiration for the experience of video. These projects further define what can be a broader understanding of directionality and representation of the past using systems of memory that learn, are networked and produced following structure found in nature and human experience.

Keywords: Video composition; Deep generative models; Time-travelling; Human visual experience; Predictive representations.

PhD Defense in Informatics Engineering: ”Enhancing Forecasting using Read & Write Recurrent Neural Networks”

Posted on

Candidate
Yassine Baghoussi

Date, Time and Place
May 29, at 09:30, Sala de Atos FEUP

President of the Jury
Pedro Nuno Ferreira da Rosa da Cruz Diniz, PhD, Full Professor, Department of Informatics Engineering, Faculty of Engineering, University of Porto.

Members
Joydeep Chandra, Phd, Associate Professor, Department of Computer Science and Engineering, Indian Institute of Technology, Patna, Índia;
Mykola Pechenizkiy, PhD, Full Professor, Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlands;
Luís Filipe Pinto de Almeida Teixeira, PhD, Associate Professor, Department of Informatics Engineering, Faculty of Engineering, University of Porto;
João Pedro Carvalho Leal Mendes Moreira, PhD, Associate Professor, Department of Informatics Engineering, University of Porto (Supervisor).

The thesis was co-supervised by Professor Carlos Manuel Milheiro de Oliveira Pinto Soares, Associate Professor at the Department of Informatics Engineering, University of Porto.

Abstract

“Machine Learning (ML) relies on both data and algorithms for optimal functioning. While conventional ML research often emphasizes algorithmic improvements, the significance of data processing is frequently overlooked. In contrast, preprocessing data stands as a distinct task, executed before feeding it into algorithms. The diversity of preprocessing methods tailored for various ML algorithms underscores its importance. However, the feedback loop between algorithms and data is often neglected. Data-related issues pose significant challenges for predictive ML algorithms, adversely affecting forecasting accuracy. These challenges arise because problems in data are inherently unpredictable, lacking a discernible pattern. In this doctoral thesis, we introduce Read and Write Machine Learning (RW-ML), an innovative paradigm enhancing time series forecasting accuracy by integrating data modification techniques into the learning process. The RW-LSTM, an adaptation of the backpropagation algorithm, unifies preprocessing with recurrent neural networks (RNNs), significantly outperforming traditional models such as LSTM. RW-LSTM enables the transition from read-only RNNs, which merely learn from data, to RW-ML, allowing direct alterations for improved predictions. Expanding the framework, the Corrector Long Short-Term Memory (cLSTM) addresses the limitations of read-only RNNs, demonstrating enhanced forecasting accuracy through empirical verification and extensive experiments. The final chapter provides a real-world evaluation, highlighting the competitive advantage of cLSTM over LSTM models in various scenarios.”

This research was carried out as part of SonaeIM.Lab@FEUP, involving Inovretail.

PhD Defense in Informatics Engineering: ”Enhancing Research Data Lifecycle: Solving Observation-centric and Reproducibility Challenge”

Posted on

Candidate
Artur Jorge da Silva Rocha

Date, Time and Place
May 17, 14:30, Sala de Atos FEUP

President of the Jury
Pedro Nuno Ferreira da Rosa da Cruz Diniz, PhD, Full Professor, Faculdade de Engenharia da Universidade do Porto

Members
José Luís Brinquete Borbinha, PhD, Full Professor, Department of Informatics Engineering, Instituto Superior Técnico da Universidade de Lisboa;
Irene Pimenta Rodrigues, PhD, Associate Professor, Informatics Department, Universidade de Évora;
Rosaldo José Fernandes Rossetti, PhD, Associate Professor, Department of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto;
Ademar Manuel Teixeira de Aguiar, PhD, Associate Professor, Department of Informatics Engineering, Faculdade de Engenharia da Universidade do Porto (Supervisor).

Abstract
“Observation is central to most research methods independent of the paradigms used. From the observational roots of positivism to the subjective individual experiences of interpretivism, most, if not all, scientific studies require repeatedly observing features over time. Documenting observations and the processes used to acquire them, capturing the context of observations, and identifying relationships between observed features in a structured and meaningful manner is extremely important for the interpretability and reproducibility of research results. It is common that the effort preceding data processing, including steps such as data cleansing and structuring, vastly outweighs the time spent writing and tailoring the algorithms for processing and analysis. Therefore, having a conceptual framework instantiated in models, methods, and tools to effectively record observations in a structured way, independently of their nature, along with associated features and context, would help reduce the effort of preprocessing and potentially contribute to increasing the quality of results. Analogously, derived features resulting from the processing of the original observations can benefit from a similar approach, thus making the workflows on research data more explicit and reproducible. This research work is focused on the data collection and analysis stages of the research data lifecycle. It has been done in the context of diverse research projects, over several years, contributing to structure knowledge across different research domains. The research encompassed the identification of new needs, the design and prototyping of novel solutions, and their application to concrete high-demand environments to test, refine, and validate the proposed solutions, which are collectively consolidated as an enhanced approach for the lifecycle of research data. As a result, this work provided contributions of different types, of which we highlight observation templates and observation framework. These main contributions were validated in the context of the respective research projects and scientific publications.”

PhD Defense in Informatics Engineering : ”Highly reconfigurable smart component system”

Posted on

Candidate
Luís Carlos de Sousa Moreira Neto

Date, Time and Place 
January 31, 14:15, Sala de Atos FEUP

President of the Jury
Carlos Miguel Ferraz Baquero-Moreno, PhD, Full Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto.

Members
Julio Luis Medina Pasaje, PhD, Associate Professor, Departamento de Ingeniería Informática y Electrónica, Facultad de Ciencias, Universidad de Cantabria, Espanha;
António Eduardo Vitória do Espírito Santo, PhD, Assistant Professor, Department of Mechanical Engineering, Faculdade de Engenharia, Universidade da Beira Interior;
Pedro Nuno Ferreira da Rosa da Cruz Diniz, PhD, Full Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto;
Luis Miguel Pinho de Almeida, Associate Professor with Habilitation, Department of Electrical and Computer Engineering, Faculdade de Engenharia, Universidade do Porto;
Gil Manuel Magalhães de Andrade Gonçalves, PhD, Assistant Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto (Supervisor).

Abstract:
“Across all sectors of our society, efficiency is an increasingly paramount concern for a sustainable world. While the significance of efficiency spans all levels, it is at a large scale where the impacts of efficient practices are most prominently noticed. Industrial activities are an example of how efficiency traduces in visible results. It doesn’t require extensive reasoning to recognize that everyday increasingly affordable goods we consume are a direct outcome of these efficiency demands. The market is demanding new services and business models that center the end user in the product design. In the near future, consumers will be able to customize a product on-line, place a production order, and see it delivered, all in the same day. This remarkable possibility arises from of a combination of efficiency and flexibility within the production processes. Several names have been used to describe the same fundamental paradigm in both academic and industrial contexts: Factories of the Future, Smart Manufacturing and Industry 4.0, all remounting to the same technological advent. This concept has far-reaching implications, extending its influence across multiple technological domains, presenting a wealth of research opportunities and driving the need for innovative technologies. This thesis delves into two technological domains related with this new paradigm and tackles one key problem in either domain. Within the Cyber-Physical Production Systems (CPPS) domain, it addresses the problem of establishing a unified network of industrial assets where software and its connections to other assets are clearly discernible and recognized. On the Reconfigurable Manufacturing Systems (RMS) domain, it addresses the fast pace at which the production lines will have to reconfigure, in particular, how software will have to reconfigure in parallel with the production lines and the ease with which new software can be developed and deployed to meet emerging challenges. A solution to both problems emerges from the field of Component-Based Software Engineering (CBSE), where this thesis drew inspiration to develop an innovative Smart Component with enhanced software reconfiguration and deployment capabilities. The proposed system exploits using Linux, a general-purpose operating system, as the component runtime environment (RTE). A combination of shared memory for efficient component communication and parallel and reconfigurable computing properties for enhanced throughput allows the proposed system to meet established application performance standards while maintaining a high degree of flexibility and reusability. The Smart Component’s flexibility is demonstrated through the implementation of two component models. The IEC 61499 component model, designed to model event-driven distributed applications for industrial system monitoring and control, and the Smart Object Self-Description (SOSD), developed by the author to describe software components, their interconnections, and their associations with industrial assets. The IEC 61499 implementation was directly compared to existing RTEs, outperforming them in real-world use cases and equaling the performance of one RTE in a literature benchmark. Additional benchmarks to assess the Smart Component’s reconfiguration performance and simplified software component development method were proposed in this thesis. The effectiveness of the SOSD implementation was validated through its application in a real-world use case, furnishing other CPPS nodes with context regarding the origin of the collected data and the software components responsible for its processing. By using Linux as the RTE, a software layer traditionally dedicated to manage components was deemed unnecessary, due to the system’s ability to execute applications conforming to relevant performance standards, while showing superior software flexibility, and even outperforming existing RTEs which employ the traditional approach. Many runtime environments for software components exist, but few allow the deployment of components built in more than one programming language, and none – to the best of the author’s knowledge – allow the development of components in any language – provided that language is at least able to read and write to files. The simplicity of developing regular software program for Linux and converting it into a software component is a promising feature that should benefit the development of industrial control and monitoring applications by bringing along the benefits of multiple high-level programming languages.”

PhD Defense in Informatics (MAP-i): ”Artificial Intelligence Methods for Automated Difficulty and Power Balance in Games”

Posted on

Candidate
Simão Paulo Rato Alves Reis

Date, Time and Place
January 11, 14:00, Sala de Atos FEUP

President of the Jury
Carlos Miguel Ferraz Baquero-Moreno, PhD, Full Professor, Department of Informatics Engineering, Faculdade de Engenharia, University of Porto

Members
João Alberto Fabro, PhD, Associate Professor, Departamento Acadêmico de Informática, Universidade Tecnológica Federal do Paraná, Brasil;
Rui Filipe Fernandes Prada, PhD, Associate  Professor, Instituto Superior Técnico, Universidade de Lisboa;
Pétia Georgieva Georgieva, PhD, Associate Professor with Habilitation, Department of Electronics, Telecommunications and Informatics, Universidade de Aveiro (representative of the MAP-i Scientific Committee);
Luís Paulo Gonçalves dos Reis, PhD, Associate Professor with Habilitation, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto (Supervisor);
Henrique Daniel de Avelar Lopes Cardoso, PhD, Associate Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto.

The thesis was co-supervised by Doutor Nuno Lau, Associate Professor at the University of Aveiro.

Abstract:
“This thesis studies the balance problem in game development, notably in two-player games. Specifically, we aim to study the viability of Artificial Intelligence (AI) as an assisting tool to fix game properties. We split our research into two paths: Power Balance, where the aim is to adjust game strategies, so they become effective as winning tools; Difficulty Balance, where the objective is to adjust game attributes on the fly so that weaker players or players at a disadvantage can compete with stronger players or players in advantage. Both domains require tuning the game, but they mainly differ in the timing and in their aim, one deals with the imbalance in game design, while the other deals with inequality in player skills. For Power Balance, our methodology was to define a full meta-game balance ecosystem based on the Pokémon video game franchise and develop an AI competition where the multiple associated tasks (battling, team prediction and assembly, and meta-game balance) are present and can be tested in a common ground. To balance the meta-game, we follow an adversarial model where team builders aim to narrow the use of optimal Pokémon while balancer agents aim to incentive the maximum of Pokémon as possible to be selected by team builders. This results in agents being able to play, build effective teams, and being able to tune the Pokémon roster over time. We discuss how our models can be extended to other video game domains. For Difficulty Balance, we propose a Multiplayer Dynamic Difficulty Adjustment framework where a Game Master (GM) agent is trained and embedded into a game, depending on the game state it will deploy handicap mechanisms. The training regime follows a specific pipeline. To generalize advantage situations, parameterized perturbations on the actions of a reference player are used to emulate several degrees of playing skill, and the advantage for each player is used to draw curves which are evaluated as a reward for the GM. This results in the GM being able to optimize game design criteria and create opportunities for the player behind to recover. We show there are suited AI tools for each task, and it is reasonable to think of power balance and difficulty as separate problems, where both can be automatically assisted and eased. Both further augment our overall comprehension of the automated game balance field.”

PhD Defense in Informatics Engineering: ”Argumentation mining from text using semantic approaches”

Posted on

Candidate:
Gil Filipe da Rocha

Date, Time and Place
October 2, 14:00, Room Professor Joaquim Sarmento (G129), DEC, FEUP

President of the Jury
Rui Filipe Lima Maranhão de Abreu, PhD, Full Professor, Faculdade de Engenharia, Universidade do Porto

Members
Hugo Ricardo Gonçalo Oliveira, PhD, Associate Professor, Department of Computer Science, Faculdade de Ciências e Tecnologia, Universidade de Coimbra;
Bruno Emanuel da Graça Martins, PhD, Associate Professor, Department of Electrical and Computer Engineering, Instituto Superior Técnico, Universidade de Lisboa;
Eugénio da Costa Oliveira, PhD, Emeritus Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto;
Sérgio Sobral Nunes, PhD, Associate Professor, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto;
Henrique Daniel de Avelar Lopes Cardoso, PhD, Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto (Supervisor).

 Abstract
“The aim of argumentation mining is to automatically detect, identify and extract arguments from natural language text. The end goal is to provide a structured representation of the arguments (argument diagrams) that can be automated and analyzed in many different ways. Argumentation is a rhetorical act that has been studied for centuries and has been influenced by different research fields such as philosophy, linguistics, computer science, and artificial intelligence. In general, arguments are justifiable positions where pieces of evidence (premises) are offered in support of a claim (conclusion). Some characteristics of natural language text and, more specifically, of argumentation exposition make argumentation mining a complex task. Indeed, the ambiguity of natural language text, the assumption of commonsense reasoning and implicit knowledge, different writing styles, and the inherent complexity of argument diagrams are some of the challenges that argumentation mining systems have to overcome. Addressing these challenges, especially across different languages and text genres, demands robust argumentation mining systems. In this thesis, we conduct research toward the development of a robust computational system that can be employed to detect, identify, and extract argumentative content across different languages and text genres. Our vision is to deploy such a system to address argumentation mining in less-resourced languages (such as Portuguese) and in text genres that feature high-variability of argument exposition profiles (such as opinion articles). To this end, we combine techniques from computational linguistics and machine learning with knowledge from argument structure and rhetoric theories to automatically identify argumentative reasoning in natural language texts. To study argumentation mining in a less-resourced language and a challenging text genre, we conduct an annotation study to create a corpus annotated with arguments from opinion articles written in Portuguese. To address the challenging argumentation mining task, we propose a relation-based approach and context-aware models motivated by argumentation theory foundations and tailored to overcome some of the challenges of argument exposition. To tackle this task in a less-resourced language, we investigate how cross-language learning techniques can be employed to explore annotated resources from different languages and improve the performance of machine learning models in a target language. Finally, to improve the robustness of argumentation mining systems across different genres, we leverage recent advancements in language modeling capabilities and propose a cross-genre approach for argumentation mining.”

PhD Defense in Digital Media: ”Towards Human-in-the-Loop Computational Rhythm Analysis in Challenging Musical Conditions”

Posted on

Candidate:
António Humberto e Sá Pinto

Date, Time and Place:
September 8, 14:30, Sala de Atos FEUP

President of the Jury:
António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with Habilitation, Faculdade de Engenharia da Universidade do Porto;

Members:
Magdalena Fuentes, PhD, Assistant Professor, Music and Audio Research Lab (MARL) and Integrated Design & Media (IDM), New York University (NYU);
Jason Hockman, PhD, Associate Professor, School of Computing and Digital Technology (DMT), Birmingham City University (UK);
Matthew Edward Price Davies, PhD, Senior Scientist,  SiriusXM/Pandora (USA) – (Supervisor);
Rui Pedro da Silva Nóbrega, PhD, Assistant Professor, Departamento de Informática, Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa;
Aníbal João de Sousa Ferreira, Associate Professor, Departamento de Engenharia Eletrotécnica e de Computadores, Faculdade de Engenharia da Universidade do Porto.

The thesis was co-supervised by Prof Rui Luís Nogueira Penha, Coordinating Professor at ESMAE, and Prof Gilberto Bernardes de Almeida, Assistant Professor at FEUP.

Abstract:

“Music Information Retrieval (MIR) is an interdisciplinary field focused on the extraction, analysis, and processing of information from various musical representations.
Grounded on the automatic analysis of musical facets such as rhythm, melody, harmony, and timbre, MIR enables applications in areas like music recommendation, automated music transcription, and intelligent music composition tools. Rhythm, an integral element of music, provides a foundation for decoding music’s complex relational structures and layered depth. Computational rhythm analysis is thus central to MIR research. It encompasses a wide range of tasks, such as the pivotal beat tracking, which unlocks the use of musical time across many MIR systems. However, conventional beat-tracking methods have struggled when dealing with complex musical features, such as expressive timing or intricate rhythmic patterns. While specialized approaches demonstrate some degree of adaptation, they do not generalise to diverse scenarios. Deep learning methods, while promising in addressing these issues, depend heavily on the availability of substantial annotated data. In scenarios requiring adaptation to user subjectivity, or where acquiring annotated data is challenging, the efficacy of beat-tracking methods lowers, thus leaving a gap in the applicability of computational rhythm analysis methods. This thesis investigates how user-provided information can enhance computational rhythm analysis in challenging musical conditions. It initiates the exploration of human-in-the-loop strategies with the aim of fostering adaptability of current MIR techniques. By focusing on beat tracking, due to its fundamental role in rhythm analysis, our goal is to develop streamlined solutions for cases where even the most advanced methods fall short. This is achieved by utilising both high-level and low-level user inputs —- namely, the user’s judgement regarding the expressiveness of the musical piece and annotations of a brief excerpt —- to adapt the state of the art to abstract particularly demanding signals. In an exploratory study, we validate the shared perception of rhythmic complexity among users as a proxy for musical expressiveness, and consequently as a key performance enhancer for beat tracking. Building upon this, we examine how highlevel user information can reparameterise a leading-edge beat-tracker, augmenting its performance to highly expressive music. We then propose a transfer learning method that finetunes the current state of the art, hereafter referred to as the baseline, to a concise user-annotated region. This method exhibits versatility across varied musical styles and offers potential solutions to the inherent limitations of previous approaches. Incorporating both user-guided contextualisation and transfer learning into a human-in-the-loop workflow, we undertake a comprehensive evaluation of our adaptive techniques. This includes examining the key customisation options available to users and their effect on performance enhancement. Our approach outperforms the current state of the art, particularly in the challenging musical content of the SMC dataset, with an improvement over the baseline F-measure of almost 10 percentage points (corresponding to over 16%). However, these quantitative improvements require further interpretation due to the inherent differences between our file-specific, human-in-the-loop technique and traditional dataset-wide methods, which operate without prior exposure to specific file characteristics. With the aim of advancing towards a user-centric evaluation framework for beat tracking, we introduce two novel metrics: the E-Measure and Annotation Efficiency. These metrics account for the user perspective regarding the annotation and finetuning process. The E-Measure is a variant of the F-measure focused on the annotation correction workflow and includes a shifting operation over a larger tolerance window. The Ae is defined as the relative (to the baseline) decrease in correction operations enabled by the fine-tuning process, normalised by the number of user annotations. Specifically, we probe the theoretical upper bound of beat tracking accuracy improvement over the SMC dataset. Our results show that the correct beat estimates provided by our approach surpass those of the state of the art by more than 20%. When considering the full length of the files, we can further frame this improvement in terms of gain per unit of user effort, quantifying the annotation efficiency of our approach. This is reflected in the substantial reduction of required corrections, with nearly 2/3 fewer corrections per user annotation compared to the baseline. In the final phase, we evaluate our human-in-the-loop strategy’s adaptability across a range of musical genres and instances presenting significant challenges. Our exploration extends to various rhythm tasks, including beat tracking, onset detection, and (indirectly) metre analysis. We apply this user-driven strategy to three unique genres with complex rhythm structures, such as polyrhythms, polymetres, and polytempi. Our approach exhibits swift adaptability, enabling efficient utilisation of the state-of-the-art method while bypassing the need for extensive retraining. This results in a balanced integration of data-driven and user-centric methods into a practical and streamlined solution.”

Keywords: Music Information Retrieval; User-centric; Transfer Learning; Beat Tracking.

PhD Defense in Informatics Engineering: ”Scaling-up organization of document sets to facilitate their analysis”

Posted on

Candidate:
Rui Portocarrero Macedo de Morais Sarmento

Date, Time and Place:
July 24, 14:00, Sala de Atos DEGI (L202A), FEUP

President of the Jury:
Carlos Manuel Milheiro de Oliveira Pinto Soares, PhD, Associate Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Members:
José Fernando Ferreira Mendes, PhD, Full Professor, Departamento de Física, Universidade de Aveiro;
Bruno Emanuel da Graça Martins, PhD, Associate Professor, Departamento de Engenharia Electrotécnica e de Computadores, Instituto Superior Técnico da Universidade de Lisboa;
Pavel Bernard Brazdil, PhD, Emeritus Professor, Faculdade de Economia, Universidade do Porto (Co-Supervisor);
Henrique Daniel de Avelar Lopes Cardoso, PhD, Associate Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto;
Sérgio Sobral Nunes, PhD, Associate Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

The thesis was supervised by João Manuel Portela da Gama, PhD, Full Professor at Faculdade de Economia da Universidade do Porto.

Abstract:

“The summarization and organization of document production of an organization in an intuitive and scalable way for massive amounts of data is of great importance in supporting decision-making.

This thesis intends to develop a theoretical and practical study to solve these challenges. The contents of this thesis were born after developing a static software prototype to analyze and provide decision support from text documents and a network of authors of scientific documentation. Several advantages were proved from the use of this mentioned prototype. Nonetheless, there were some concerns regarding the prototype’s ability to cope with higher dimensional networks and also a massive amount of documents. The development case study considers the affinity between authors on a large scale and constantly evolving. The first challenge is to scale the representation methods of documents of the authors. The second challenge is to capture the temporal development of the organization. Considering this context, we developed and implemented streaming techniques for the characterization of each author and other sub-units of the organization. Thus, by integrating into affinity groups identified by keywords and relevance measures that characterize them. We have finished this work by testing several developed algorithms to minor the disadvantages of the original prototype and gathering a panoply of solutions for problems related to text streaming techniques, considering a large-scale approach for the corresponding analysis. Information Retrieval techniques were used, and the analysis of social networks and streaming data was necessary. We solved several associated issues with efficient text streams analysis, using several techniques from pure streams analysis techniques to evolving complex networks techniques. These techniques that served as a base to innovation and contribution with more than ten new algorithms proved to improve the prototype and solve the issues that initially drove us to improve and contribute to several related areas of text analysis and streams.”

keywords: Streaming; Text Mining; Social network Analysis; Social network Visualization.

PhD Defense Digital Media: ”Connect-the-Dots: Artificial Intelligence and Automation in Investigative Journalism”

Posted on

Requested by:
Joana Rodrigues da Silva

Date, time and place
July 19, 14h30, room L119 DEMEC (FEUP)

President of the Jury:
António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with Habilitation, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Members:
Teresa Isabel Lopes Romão, PhD, Associate Professor, Departamento de Informática, Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa;
Luís António Santos, PhD, Assistant Professor, Departamento de Ciências de Comunicação, Instituto de Ciências Sociais da Universidade do Minho;
Miguel Ângelo Rodrigues Midões, PhD, Invited Adjunct Professor, Departamento de Comunicação e Arte, Escola Superior de Educação do Instituto Politécnico de Viseu;
Helena Laura Dias de Lima, PhD, Associate Professor, Departamento de Ciências da Comunicação e da Informação, Faculdade de Letras da Universidade do Porto (Supervisor);
Alexandre Miguel Barbosa Valle de Carvalho, PhD, Assistant Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Abstract:

“After the COVID-19 epidemic and the consequent humanitarian crises that devastated the planet, there is a need to claim the role of investigative journalism as a watchdog and permeation system of social justice and democracy through public exposure. We are witnessing a sharp decrease in investment in this journalism specialty, either because of its impertinence in dealing with public management issues or because of the time spent on this type of investigation, which fundamentally takes longer than current journalism to produce results. In this sense, we perceive the influence of automation and artificial intelligence in information production processes to highlight all human tasks with the possibility of being carried out in less time by technological systems. Considering this possibility, there was an interest in studying, in-depth, how robotics and the application of artificial intelligence through platforms that support the usual procedure of journalism can help and even improve the global state of investigative journalism practice, nowadays. The Connect-the-Dots platform and the DODO assistant emerge as a hypothetical digital solution for some of the problems that investigative journalism currently faces. It could be a way to practically implement investigative journalism in its scope of innovation by integrating tools and open-source practices in a Design-Based-Research approach to knowledge archaeology.”

Keywords: Artificial Intelligence, Automation, Investigative Journalism, Design-Based-Research, Digital Media.