Candidate:
Bernardo José Coelho Leite
Date, Time and Location:
17 November 2025, 14:0, na Sala de Atos da Faculdade de Engenharia da Universidade do Porto
President of the Jury:
Pedro Nuno Ferreira da Rosa da Cruz Diniz (PhD), Full Professor in the Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto.
Members:
Hugo Ricardo Gonçalo Oliveira (PhD), Associate Professor in the Department of Informatics Engineering, Faculdade de Ciências e Tecnologia, Universidade de Coimbra;
Maria Luísa Torres Ribeiro Marques da Silva Coheur (PhD), Associate Professor in the Department of Informatics Engineering, Instituto Superior Técnico, Universidade de Lisboa;
Luís Paulo Gonçalves dos Reis (PhD), Associate Professor with Habilitation in the Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto;
Henrique Daniel de Avelar Lopes Cardoso (PhD), Associate Professor in the Department of Informatics Engineering, Faculdade de Engenharia, Universidade do Porto (Supervisor).
Abstract:
Humans pose questions all the time, and efforts to create AI systems to do the same have been developed. This task, known as Question Generation (QG), is a subfield of natural language generation that aims to automatically produce relevant and grammatically correct questions from a given input, such as text. A key motivation for QG is to support time-consuming tasks like the manual creation of educational questions by teachers. While QG systems have significantly improved, grammatical accuracy alone does not ensure educational value. Consequently, the adoption of QG tools in educational contexts remains limited.
This thesis is driven by three key challenges in QG: (1) the trustworthiness of AI-generated questions; (2) the limited controllability; (3) restricted applicability in less-resourced languages. To address these challenges, we focus on generating open-ended and multiple-choice reading comprehension questions from narrative texts for elementary school students. For challenge 1, we analyze and report the quality of generated questions, identifying both successful and failed cases. For challenge 2, we enhance controllable generation mechanisms by incorporating multiple attributes, such as narrative elements, explicitness, and difficulty, into the generated questions. Challenge 3 is addressed through a special focus on Portuguese, a morphologically rich language that remains underrepresented in QG research.
Our methodology spans from early rule-based and neural approaches to more advanced controllable QG techniques, including fine-tuning, zero- and few-shot prompting with both small and large language models. This offers a comprehensive view of the evolution and performance of QG systems across different stages. We contribute by systematically applying and adapting current QG techniques. We develop case studies that explore controllability and educational relevance, providing comprehensive analyses of question quality, and releasing new QG models and datasets tailored to less-resourced languages such as Portuguese. Evaluation combines automatic metrics with human-centered assessments involving experts, teachers, and students, whose input provides critical insights into the usefulness and effectiveness of the generated questions.
The results show that it is possible to generate well-formulated and answerable questions with controllable attributes. Although machine-generated questions approach the quality of humanauthored ones, semantic issues still arise. In addition, generating MCQs with answer options that are effective for students remains a challenge. These findings highlight the ongoing need for research in educational QG, especially in supporting less-resourced languages and enhancing the reliability of automated generation systems.
