PhD Defense in Digital Media: ”Modelo para utilização da prosódia e da interacção no acesso às expressões matemáticas através da fala sintetizada para pessoas com deficiência visual”

Candidate:
Adriana Silva Souza

Date, time and place:
July 10, 10:30,  Sala de Atos FEUP

President of the Jury:
António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with Habilitation, Faculdade de Engenharia da Universidade do Porto.

Members:
Vitor Manuel Pereira Duarte dos Santos, PhD, Assistant Professor, NOVA Information Management School, Universidade Nova de Lisboa;
João Manuel Pereira Barroso, PhD, Associate Professor with Habilitation, Vice-Reitor para a Inovação, Transferência de Tecnologia e Universidade Digital, Universidade de Trás-os-Montes e Alto Douro;
João Paulo Ramos Teixeira, PhD, Coordinator Professor, Departamento de Eletrotecnia, Escola Superior de Tecnologia e Gestão do Instituto Politécnico de Bragança;
Maria Selene Henriques da Graça Vicente, PhD, Assistant Professor, Departamento de Psicologia, Faculdade de Psicologia e de Ciências da Educação da Universidade do Porto;
Maria do Rosário Marques Fernandes Teixeira de Pinho, PhD, Associate Professor, Departamento de Engenharia Eletrotécnica e de Computadores, Faculdade de Engenharia da Universidade do Porto;
Diamantino Rui da Silva Freitas, PhD, Associate Professor, Departamento de Engenharia Eletrotécnica e de Computadores, Faculdade de Engenharia da Universidade do Porto (supervisor).

Abstract:

“The synthesized speech of mathematical contents still presents several challenges. For Mathematics to be understood by people with visual impairment, it needs to be verbalized in detail, which generates long outputs and causes cognitive overload; in addition, Mathematics has quite peculiar rules. Therefore, most of the time, prosodic limits such as pauses, and intonation are not adequately synthesized. This investigation proposes a model that uses prosody and interaction to access mathematical expressions to minimize the problems mentioned. We relied on the Design-Based Research methodology to develop the model and divided the study into four stages. In the first stage, a systematic literature review was carried out. We conducted an initial exploration investigation with interviews with students with visual impairments and braille teachers and analyzed the mathematics spoken by speech synthesizers. In the second stage of the investigation, a corpus of mathematical expressions spoken by math professors was created to support prosody research. Intonation and pauses were the investigated prosodic components. Although the studies have yet to go into intonation in-depth, we did some tests of prosodic modulation of the fundamental frequency, highlighting stretches of mathematical expressions according to the level in the MathML tree. Concerning pauses, we identified their main patterns in mathematical expressions. We also performed an eye-tracking experiment with sighted people to understand the cognitive processes surrounding mathematical expressions’ observation, analysis, and processing. In the third stage, a linear regression model which calculates the pauses for mathematical expressions dynamically was created and evaluated with visually impaired students. The results showed advances regarding the solutions found, perceived mainly when the mathematical expressions are unfamiliar to the students. The results of the eye tracking experiment showed that in addition to the complexity of the mathematical expression, it was necessary to propose a new formal concept that was called diversity, quantifying this subjective property of the structures of expressions because it was found that it also impacts during the cognitive processing of expressions. Data analysis provided clues for creating the interaction model that uses diversity to control the cognitive load in accessing mathematical expressions during the process. The evaluation of the model with visually impaired people showed an advance concerning existing works since students performed better when accessing mathematical expressions with the model. In the fourth stage, we made the final proposition of the model based on the assessment of people with visual impairments. The results achieved in this investigation allow greater autonomy in the reading of mathematical expressions; people with visual impairment can govern the interaction in the auditory access according to the need to reinforce their memory; in addition, it can reduce the time in the manipulation of mathematical expressions compared to traditional tools, improve the writing process, since reading is linked to writing and relieve the student’s memory. In addition to these contributions, we can also highlight the discovery of the new diversity parameter, which is strongly related to the cognitive processing of expressions. In general, these contributions make it possible to improve and develop mathematics education, particularly in the teaching-learning process of visually impaired people, making them more autonomous beings, which, in addition to scientific contributions, can also generate social and economic impacts arising from accessibility.”

Keywords: Synthesized Speech, Mathematics, Accessibility, Visual Impairment, Complexity, Diversity.

PhD Defense in Digital Media: “Location-Based Serious Games for Science Communication of Natural Heritage”

PhD Defense in Digital Media

Candidate:
Liliana Andreia da Rocha Santos

Date, time and place:
June 29, 14:00, Sala de Atos FEUP

President of the Jury:
Pedro Nuno Ferreira da Rosa da Cruz Diniz, PhD, Full Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Members:
Teresa Isabel Lopes Romão, PhD, Associate Professor, Departamento de Informática, Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa;
João Paulo Fonseca da Costa Moura, PhD, Assistant Professor, Universidade de Trás-os-Montes e Alto Douro;
Paulo Simeão de Oliveira Ferreira de Carvalho, PhD, Assistant Professor, Departamento de Física e Astronomia, Faculdade de Ciências da Universidade do Porto;
António Fernando Vasconcelos Cunha Castro Coelho, Associate Professor with Habilitation, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto (Supervisor);
Rui Pedro Amaral Rodrigues, PhD, Associate Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Abstract:

“Science Communication assumes increasing importance to encourage people to become more conscious about natural heritage, which is most important for a sustainable society.

This thesis proposes the use of location-based serious games as an effective digital media tool for science communication of natural heritage. We hypothesise that serious games can be an effective tool for science communication of natural heritage and that location-based serious games can further improve science communication of natural heritage by allowing that science communication message to occur in its respective physical context (in loco). With our research questions, we set out to find the role of serious games for science communication of natural heritage, how can location-based serious games contribute to science communication of natural heritage, and how we can help others design location-based serious games for science communication of natural heritage.

A systematic literature review helped to understand the possible roles of serious games in science communication. It revealed that serious games can raise awareness, enjoyment, interest, opinions and understanding of our natural heritage. We also identified different types of technologies, game mechanics and science communication contents that these games integrate.

This research follows the methodology of design science research. We developed three game prototypes to understand how location-based serious games can contribute to this area, thus answering the second research question. These games included a map with points of interest, the location from a Global Navigation Satellite System, geotagged information, contextualised mini-games, a collectables photo gallery, and content for the general public. They were developed within the context of a specific natural park, however, the concept can be adapted and applied to different parks, botanical gardens, and other green spaces.

The knowledge and experience obtained with the literature review and the creation of the prototypes helped develop a framework and guidelines for creating location-based serious games for science communication of natural heritage, thus answering the third research question. The framework includes the context, objectives, how to do it, the product and the responses. The guidelines further assist in the analysis, design, development and evaluation. A co-creation workshop for designing location-based serious games for an urban park was created to validate the framework and the guidelines.

Location-based serious games are an effective digital media tool for science communication of natural heritage, allowing communicating in a more contextualised way and further enhancing the visitor experience. Our main contributions are the framework and guidelines, for designing location-based serious games for science communication of natural heritage.”

Keywords: Serious Games, Location-Based Games, Science Communication, Natural Heritage.

PhD Defense in Digital Media: ”Research data description in multiple domains: supporting researchers with data management plans”

Candidate:
Yulia Karimova

Date, time and place
June 2, 14:00, Sala de Atos FEUP

President of the Jury
António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with habilitation, Faculdade de Engenharia da Universidade do Porto.

Members
Maria Manuel Lopes de Figueiredo Costa Marques Borges, PhD, Associate Professor, Departamento Filosofia, Comunicação e Informação, Faculdade de Letras da Universidade de Coimbra;
José Luis Brinquete Borbinha, PhD, Full Professor, Instituto Superior Técnico da Universidade de Lisboa;
Irene Pimenta Rodrigues, PhD, Associate Professor, Departamento de Informática, Escola de Ciências e Tecnologia da Universidade de Évora;
Maria Cristina de Carvalho Alves Ribeiro, PhD, Associate Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto (Supervisor);
Carla Alexandra Teixeira Lopes, PhD, Assistant Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Abstract
The growing amount of research data and the recognition of their value in the scientific community has led to the development of policies, infrastructures, tools and services. Researchers are encouraged to open their data and make them FAIR (Findable, Accessible, Interoperable and Reusable) to ensure the verifiability of results and transparency of research, while increasing their value. In this context, Research Data Management (RDM) becomes a vital component of high-quality research and a regular activity for researchers. The Data Management Plan (DMP), provision of quality metadata, repository selection, and data publication are some of the activities that researchers need to undertake when following the Open Science policies. However, these activities are time-consuming and not obvious to researchers, requiring specific knowledge, effort and experience. To support researchers, institutions are looking for solutions and developing workflows, tools, systems and services that aim to meet RDM requirements of both researchers and funding agencies. Implementing any information system, tool or service at a research institution requires their development, testing and evaluation, as well as adaptation to existing regulations. In this work, we present the development of a DMP support system that meets RDM requirements and researcher needs. We also propose its implementation in the institutional infrastructure and workflow. We explore various aspects related to the DMP support system development, namely RDM and DMP requirements and legislation at the international and national levels, institutional project management and RDM workflows, existing initiatives, recommendations and tools. We introduce a collaborative DMP-building method based on collaboration between researchers and a data steward. We describe its application to case studies from different scientific domains, analyze the results and propose its systematic evaluation. We also propose a path that simplifies the DMP creation process, to make plans more standard, high-quality, and detailed, by implementing controlled vocabularies in the DMP support system. To this end, we apply the collaborative method on projects from the same scientific domain, identifying specific aspects for this domain and proposing terms that can be used with controlled vocabularies and possibly contributing to the design of Domain Data Protocols. The proposed DMP support system also considers the existing mechanisms of automation and interoperability, namely the machine-actionable DMP (maDMP) standard. We analyze the maDMP standard and existing tools created according to this standard and propose a DMP structure and a DMP workflow that can help to make the DMP machine-actionable. Finally, we suggest the implementation of the DMP support system in the institutional RDM workflow, and the interface to support researchers in DMP creation and RDM activities in general. Moreover, this work concludes that the DMP support system can be implemented in any institution, and used for projects in different scientific domains. The results constitute a guide for institutions that are still at the beginning of the development of the RDM infrastructure. The feedback from researchers shows that they need support in DMP creation and monitoring, not only at the planning stage of the project but also during its course and sometimes even after. They are very positive about the implementation of the proposed system, highlighting its importance and necessity. Along with researchers, institutions also benefit from the implementation of such a system by providing support to researchers in RDM issues, reducing the time to create plans, improving the quality of DMPs and making projects comply with the RDM and funders’ requirements, thereby contributing to the prestige of the institution.

PhD Defense in Informatics Engineering: “Incentive Mechanisms and Policy Evaluation on Open Multi-Agent Systems: Towards Social-Aware Transportation Systems”

Candidate:
Zafeiris Kokkinogenis

Date, time and place
23rd September, 14:30, Sala de Atos FEUP

 

President of the Jury:

Carlos Manuel Milheiro de Oliveira Pinto Soares, PhD, Associate Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto

 

Members:

Alberto Fernandez Gil, PhD,  Associate Professor, Departamento de Ciencias de la Computación, Arquitectura de Computadores, Lenguajes y Sistemas Informáticos y Estadística e Investigación Operativa, Universidad Rey Juan Carlos;

Sandra Maria Monteiro de Melo, PhD, Main Researcher of the Business Unit INTELI – Policy & Intelligence for Sustainability, CEiiA – Centre of Engineering and Product Development;

Pedro José Ramos Moreira de Campos, PhD, Assistant Professor, Agrupamento Científico de Matemática e Sistemas de Informação, Faculdade de Economia da Universidade do Porto;

Rosaldo José Fernandes Rossetti, PhD, Associate Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto (Supervisor);

Ana Paula Cunha da Rocha, PhD, Associate Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

 

Abstract

Generally speaking, a large-scale socio-technical system is formed up of individual entities that are distributed along with the system’s space and act asynchronously in their decision making processes. Each of the individuals bears its own goals and tends to behave rather “selfishly” and “greedily” to maximise self-welfare utilities. However, this characteristic will generally affect negatively the global efficiency and the designed (expected) emergent behavior of the system. Indeed, private road transport imposes negative externalities on society, such as road capacity restrictions, accidents, congestion, etc. An efficient mobility model must take into account the real costs of transport, and its regulatory framework will need to produce the conditions for people to make sustainable transport choices. Economic theories offer two types of instruments for addressing the problem of transport externalities: command-and-control and incentive-based policies.

Command-and-control policies are government regulations which force users to change their behavior. In that sense, recent approaches to optimise the traffic network throughput and reduce traffic congestion basically rely on “road pricing”. However, this approach ends up penalising the user and creating social inequalities as it imposes a tax to be paid. Only those who are insensitive to the price will benefit. Also, a population may not be responsive to the defined penalties, and thus, the regulation may not be efficient.

On the other hand, an approach that has gained the community’s attention is based on the implementation and design of incentive schemes in public policy. Incentives are seen as those external measures that try to motivate a behavior change towards the objective of the system. It appears to be a “fairer” vision, as it does not discriminate the user but rather tries to bring the society into equilibrium.

The domain area on which this PhD thesis is focused concerns open and competitive multiagent systems, such as the Intelligent Transportation Systems (ITS) and the electricity markets. This thesis intends to address the issue of whether or not incentive-centred designs can favour the emergence of social-aware behavior in agents that have selfish tendencies for a (global) optimal evolution of a socio-technical system. Traditional transport planning tools using the four-step model combined with standard economic appraisal methods are not able to provide such analysis. Instead, a multi-agent system (MAS) social simulations can be used as it is argued in the literature of complex systems.

Keywords: Multi-Agent Systems, Incentive Mechanisms, Resource Markets, Policy Evaluation, Traffic Simulation.

PhD Defense in Digital Media: ”Acoustic ecology in artistic creation: re-discovering underwater soundscapes”

Candidate:
Francisca Santos Silva Rocha Gonçalves

Date, time and place
22nd September, 10h00, Sala de Atos FEUP

President of the Jury
António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with Habilitation at Faculdade de Engenharia da Universidade do Porto

Members
Angus Carlyle, PhD, Professor at University of Arts London, UK;
Isabel Maria Antunes Pires, PhD, Invited Assistant Professor at Universidade Nova de Lisboa;
Raquel Maria Lemos e Castro, PhD, Researcher at Universidade Lusófona de Humanidades e Tecnologias;
Rui Luís Nogueira Penha, PhD, Assistant Professor at Instituto Politécnico do Porto (Supervisor);
Gilberto Bernardes de Almeida, PhD, Assistant Professor at Faculdade de Engenharia da Universidade do Porto.

Abstract

The aim of this thesis is to connect people with underwater environments while creating awareness for the problem of anthropogenic noise pollution through artistic creation. This dissertation addresses the lack of information on underwater noise discourse, especially on the vibrational aspect of underwater sound. By asking how to convey the experience of underwater soundscapes and their human disruption, it proposes two major directions that inform and complement each other. First, several underwater soundscape recordings were realised. They provide an understanding of the ambience of places and allow attunement with the natural world. Second, six practice-based research artworks based on the soundscape recordings and methods from acoustic ecology and ecoacoustics, focusing on anthropogenic sounds aimed at improving inclusion with nature while creating awareness of the noise pollution problem. The revised strategies and concepts included place attachment, emotional affinity, nature-connectedness and ecology of æffect. These artworks provide the setting for challenging audiences to bond with underwater environments by listening to their soundscapes while revealing a particular aspect of underwater sound. Moreover, they explore creative strategies for developing sonic experiences with the introduction, experience and performance of underwater sounds in artistic creation. Additionally, the artworks introduce and approach a natural phenomenon related to underwater sound (Luciferina – bioluminescence; SINØ III – cymatics; #switchoffandlisten – noise awareness campaign; DIS_turbation – bioturbation; Munda – cavitation; 2182kHz – auditory masking). The insights from the artistic experiences proved critical to continue producing innovative contributions and to create knowledge while influencing the communities. The significance of this study is that it informs the theoretical understanding of noise pollution impacts on aquatic fauna and underwater acoustics by introducing a focus on the vibrational aspect of underwater sound: particle motion. In addition, it informs the empirical understanding of the practice-based research on developing the proposed artworks. The novelty lies in addressing biological phenomena as a creative resource to intervene in underwater noise dialogue, reveal hidden qualities of vibroscape ecology (such as particle motion) and translate them in a more grasping way through the artistic artworks. As underwater soundscapes are increasingly a topic of concern globally, they become crucial to fostering an emotional affinity between people and natural environments, promoting inclusion with nature.

Keywords: ecological sound art, acoustic ecology, underwater soundscapes, anthropogenic noise, vibroscape ecology, nature-connectedness, ecology of æffect.

PhD Defense in Digital Media: “Migration as movement – repurposing the voice/interface to explore aspects of human migratory movement through artistic research”

Candidate:
Terhi Tuulikki Marttila

Date, Time and place:
June 27, 10:00, Sala de Atos of FEUP

 President:

António Fernando Vasconcelos Cunha Castro Coelho, PhD, Associate Professor with Habilitation, Faculdade de Engenharia da Universidade do Porto.

Members:

Anne Karhio, PhD, Associate Professor, Faculty of education of Inland Norway, University of Applied Sciences;

Manuel José de Freitas Portela, PhD, Full Professor, Faculdade de Letras da Universidade de Coimbra;

Vitor Manuel Pereira Duarte dos Santos, PhD, Assistant Professor, NOVA IMS da Universidade Nova de Lisboa;

José Miguel Santos Araújo Carvalhais Fonseca, Professor Auxiliar da Faculdade de Belas Artes da Universidade do Porto;

Diamantino Rui da Silva Freitas, PhD, Associate Professor, Faculdade de Engenharia da Universidade do Porto (Supervisor).

Abstract:

In this artistic research we start from the idea that digital works can be utilized for imagining and playing with yet impossible realities, for asking questions, and for portraying aspects of life such that we can explore these from a distance. This dissertation consists of three browser-based artworks programmed by myself in HTML, Javascript and CSS. These art experiences unfold on the screen, asking the person to interact with them in order to complete their meanings. The works incorporate, among other things, voice interaction through synthetic speech and speech input, the uses of which form an essential component of the works meanings. Each work is concerned with very different types of questions around migration, but overall, and as a triptych, the works explore what the movement brought about by migration implies from a human standpoint. The themes encompassed extend from exploring perspectives on language, where movement results in linguistic accent which provokes a notion of the Other through external difference (in We cannot take them all), or exploring the movement induced by the tension between our body and our personal needs and desires, but that is governed by (shifting) borders (in Speak out) and finally, at exploring migratory movement through the ambivalent relationship between the here and the there, the simultaneous processes of (re-)rooting and of living with homesickness (in Transplanted). While this research may not depart from clear research questions, it nevertheless arrives at observations about the artistic strategies that emerge when migration is explored through notions of voice/interaction. The works themselves open up a space of contemplation about the respective themes, inviting participation through voice.

Keywords: migration, voice interaction, electronic literature, digital art, artistic research

PhD Defense in Informatics Engineering: ”An Agent-based Approach to Design Decentralized Cloud-Edge Data Analysis in Cyber-Physical System”

Candidate:
Jonas Felipe Pereira de Queiroz

Date, Time and place:
June 14, 14:00, Sala de Atos of FEUP

President:

Carlos Manuel Milheiro de Oliveira Pinto Soares, PhD, Associate Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Members:

Luís Filipe Santos Gomes, PhD, Associate Professor with Habilitation, Departamento de Engenharia Electrotécnica e de Computadores, Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa;

Maria Goreti Carvalho Marreiros, PhD, Coordinating Professor with Habilitation, Departamento de Engenharia Informática, Instituto Superior de Engenharia do Porto;

Paulo Jorge Pinto Leitão, PhD, Full Professor, Departamento de Engenharia Eléctrica, Instituto Politécnico de Bragança (Supervisor);

Rosaldo José Fernandes Rossetti, PhD, Associate Professor, Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto.

Abstract:

The 4th industrial revolution is characterized by the digitization of industrial environments, based on the use of Internet of Things, Cloud/Edge Computing and Artificial Intelligence technologies. This technological advances are pushed by the ever-changing market that besides posing new business challenges, further increase the already existing complexity and related issues faced by industries at the operational environments. In this context, Cyber-Physical Systems (CPS) have been considered the new paradigm to develop the next generation of intelligent and distributed industrial automation systems that can achieve higher levels of flexibility and dynamic adaptation. Although data analysis has shown to be a key enabler of industrial CPS in the development of smart machines and products, the traditional Cloud-centric solutions are not suitable to attend the data and time-sensitive requirements. Aiming to cope with that, Edge Computing has been adopted to enable the data processing capabilities at or close to the physical components, complementing the Cloud solutions. However, defining what data analysis capabilities should be deployed along Cloud-Edge computational layers is not a straightforward task. Motivated by this challenge, this work proposes a conceptual framework that defines a modular agent-based architecture to design and develop cyber-physical components, together with a general guideline to support the system engineers to evaluate and determine the most suitable computing layer to deploy a given data analysis task. The proposed framework was implemented and evaluated considering some experiments based on a smart machine CPS testbed, and different scenarios that illustrate the benefits of distributing the data analysis capabilities along Cloud-Edge layers. The modular structure of the agent-based cyber-physical component architecture shown to be suitable, enabling to easily configure and distribute data analysis tasks along the CPS components and their deployment in the different computing layers, also providing a seamless and transparent interaction between the components. In addition, the proposed guideline and the Fuzzy Logic decision-making systems were used to recommend, in a less ad-hoc manner, the most suitable layer to deploy three data analysis tasks identified in the case study. 

Keywords: Cyber-Physical Systems, Data Analysis, Multi-Agent Systems, Edge Computing

PhD Defense in Digital Media: “Modular Architecture for 3D Game Environment Art with Photogrammetry”

Candidate:
Nataska Statham

 Date, Time and place:
June 22, 09:00, remotely with streaming at: https://www.youtube.com/channel/UCvsg2ymeaHLRLbsGt67JmKw

 President:

Doutor António Fernando Vasconcelos Cunha Castro Coelho, Associate Professor with Habilitation, Departamento de Engenharia Informática of Faculdade de Engenharia, Universidade do Porto.

Members:

PhD Richard A. Bartle, Senior Lecturer and Honorary Professor of Computer Game Design at the University of Essex, UK;

PhD Andrija Bernik, Assistant Professor at the Department of Multimedia, Design and Application, University North in Varazdin, Croácia;

PhD Rui Pedro da Silva Nóbrega, Assistant Professor, Departamento de Informática of Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa;

PhD Luís Filipe Pinto de Almeida Teixeira, Assistant Professor, Departamento de Engenharia Informática of Faculdade de Engenharia, Universidade do Porto;

PhD João Tiago Pinheiro Neto Jacob, Technical Product Owner at Mindera Gaming (Supervisor).

Abstract

 Photogrammetry is a technology increasingly used in games for modelling 3D characters and natural environments, with the potential of time-efficient generation of realistic meshes with a higher level of detail than is normally achieved via hand-modelling. In architectural environments, however, although textures are photoscanned, meshes are still hand-modelled. A key challenge is that architecture for games tends to rely on modular meshes, which require high precision and standardisation, whereas photoscans of real-world architecture tend to output irregular meshes that are hard to conform to modularity. This investigation aims to address this lack of integration with a focus on high granularity modular architecture for 1st and 3rd person 3D games. It starts with a review of 3D game graphics production and modular architectural assets for games, as well as the integration of photogrammetry in the creation of 3D assets for games and for 3D environment art in particular. This is followed by a theoretical contribution, consolidating the principles of modular architecture for games within the game development lifecycle. Next, it identifies the inherent incompatibilities preventing the integration of photogrammetry with modular architecture for games and postulates solutions, implemented and tested via a series of controlled experiments using commonly available software. It concludes that integration is possible, but that a manual process is presently required due to the lack of tools designed to conform detailed surfaces to the profile of standardised reference meshes. The investigation proposes a workflow to achieve sectional modularity of architectural photoscans via tileable borders to which mesh variations can be integrated via modular swapping. It proposes a modified game development lifecycle with longer pre-production and potentially multiple photoscanning sessions to incorporate architectural photoscans into modular game environment art production. Via this workflow, a wide range of types of photoscanned architectural surfaces and types of geometry were standardised into the requirements optimal for modular meshes for 3D environment art, indicating its viability as a workflow for game development in general.

 Keywords: 3D environment art, game development lifecycle, level design, modular architecture, photogrammetry.

PhD Defense in Digital Media: ”Literacia visual e comunicação”

Candidate:
Jorge Luís Pacheco Barcelos

Date, Time and place:
May 31,10:30, remotely with streaming at: https://www.youtube.com/channel/UCvsg2ymeaHLRLbsGt67JmKw

President of the jury:
PhD Jorge Manuel Pinho de Sousa, Full Professor at Faculdade de Engenharia of Universidade do Porto

Members:
PhD Paulo Nuno Vicente, Assistant Professor at Departamento de Ciências da Comunicação of Faculdade de Ciências Sociais e Humanas of Universidade Nova de Lisboa;

PhD Luís Francisco Mendes Gabriel Pedro, Assistant Professor at Departamento de Comunicação e Arte of Universidade de Aveiro;

PhD Maria da Conceição Gonçalves Costa, Associate Professor at Escola de Comunicação, Arquitetura, Artes e Tecnologias da Informação of Universidade Lusófona;

PhD José Manuel Pereira Azevedo, Associate Professor at Departamento de Ciências da Comunicação e da Informação of Faculdade de Letras of Universidade do Porto (Supervisor);

PhD António Fernando Vasconcelos Cunha Castro Coelho, Associate Professor at Departamento de Engenharia Informática of Faculdade de Engenharia of Universidade do Porto.

Abstract:

This thesis seeks to verify the degree of visual literacy that individuals in undergraduate and graduate studies at the Universities of Porto and Paraíba Federal, in the courses of design, multimedia, digital media, engineering, communication, have when interpreting an image. These interpretations present three important dimensions such as affective, compositional, and critical when analyzing an image, be it a painting, a photograph, a work of art, or an audiovisual piece, materialized in a verbal text. The theoretical framework of this research is given by reading a corpus of visual literacy such as Dondis (1974), Housen (1983), Braden and Bacca (1991), Arnheim (2001), Averignou and Pettersson (2011), Duchak (2014), Serafini (2014), Brumberger (2019), Kedra (2019), Supsakova (2020); and on image with Kress and Van Leeuwen (1996), Mirzoeff (1999), Cartwright & Sturken (2001), Villafañe (2006), and Mitchel (2009); and communication with Hall (2001), Berger (2000), Baylen&D’alba (2005), Horkenheimer & Adorno (2006), Santaella & Noth (2020), and in methodologies with Bloom (1956); Leung (2000); Amantes&Borges (2008), Hassan (2011), Biggs &Collins (2014), Rautiainen&Jappinen (2017), Arneson&Offerdahl (20018), Fernades (2019). This research has developed the Multimodal Assessment Method of Visual Literacy (MAMLV) that uses both quantitative and qualitative methodologies, with the qualitative aspect assisting in providing greater detail of the interpretive experience. The data collection instrument was a semi-structured interview that allows the interviewee to vent their interpretive thoughts, with minimal interference from the interviewer. These statements were collected and classified through the use of a coding manual that aids in classification. The statements were classified into categories of domains and subcategories, the topics. Depending on the number of statements, the individual belongs to a certain level of visual literacy. There are three levels of visual literacy, although the Multimodal Assessment Method of Visual Literacy (MAMLV) considers intermediate levels between each level. Thus, it can be observed that individuals despite being immersed in a visual world still present a low degree of visual literacy, as they do not have the skills and competencies fully developed to know the communicative interactions between the visual elements, their design principles and the visual grammar that are contained in an image. Although there are also individuals with a high degree of visual literacy. The findings suggest that this happens due to the lack of teaching curricula that contemplate visual and media literacy in the different levels of primary, secondary and university education, considering the curricular structures of the courses involved, and the results found, we can conclude that the inclusion of visual literacy in systematized teaching is of paramount importance for the formation of fully visually literate individuals.

Keywords: Visual Literacy; Image; Communication, Methodology

PhD Defense in Informatics Engineering: ”Increasing the Dependability of Internet-of-Things Systems in the context of End-User Development Environments”

Candidate:
João Pedro Matos Teixeira Dias

Date, Time e Place
1st of April, 09:00, remotely with streaming at: https://www.youtube.com/channel/UCvsg2ymeaHLRLbsGt67JmKw

President of the Jury
PhD Rui Filipe Lima Maranhão de Abreu, Full Professor at Faculdade de Engenharia da Universidade do Porto

Members
PhD Dariusz Mrozek, Associate Professor at the Department of Applied Informatics at Silesian University of Technology, Poland;
PhD Pedro Nicolau Faria da Fonseca, Assistant Professor at the Department of Electronics, Telecommunications and Informatics of Universidade de Aveiro;
PhD André Monteiro de Oliveira Restivo, Assistant Professor at the Department of Informatics Engineering of Universidade do Porto;
PhD Hugo José Sereno Lopes Ferreira, Assistant Professor at the Department of Informatics Engineering of Universidade do Porto (Supervisor).

Abstract:

The ubiquitousness of computing, known as Internet-of-Things (IoT), has reshaped the way people interact with the physical world. However, the scale, distribution — both logical and geographical –, density, heterogeneity, interdependence, and {quality-of-service} requirements of these systems make them complex, posing several challenges from both operational and development viewpoints.

While there is a consensus that the widely used software engineering practices are inadequate for IoT development, they remain the go-to solutions for most practitioners. This aspect has severely compromised their dependability, centralizing most of the computation of these (soft) real-time systems in cloud infrastructure. Likewise, as these systems scale in terms of devices and applications, it outreaches existing technical resources to manage and operate them, becoming of paramount importance, making them as most self-managed as possible while empowering the ability of system operators (including end-users) to configure and understand them — mainly using solutions that do not require high technical expertise, viz. low-code development solutions — including the configuration of fail-safe measures.

This dissertation’s primary focus is to research how to improve the current status quo on the dependability of IoT. However, this is a manifold endeavor: (1) what are the best practices for developing IoT dependably, and what is their scientific soundness, (2) do the current solutions give the fundamental building blocks that allow to design and construct dependable systems, and, if not, what contributions are needed to overcome the existing limitations, and, lastly, (3) giving that these systems are operated by humans with limited technical expertise, it is required that their users can use and configure them without compromising their correct operation. As we set ourselves to tackle these challenges, we claim that:

It is possible to enrich IoT-focused end-user development environments in such a way that the resulting systems have a higher dependability degree, with the lowest impact on the know-how of the (end-)users.

As preliminary research, to understand what end-users want to automate and how they wish to perform such automations, a study was carried to collect automation scenarios. These scenarios showcased the complexity of the automations that some end-users want to perform and the interdependencies between different information sources, devices, and persons. It also supported the view that some of the appliances that end-users want to automate can have nefarious effects if a malfunction happens or a misconfiguration is performed.

We followed extensive literature research and experimental process to mine a set of patterns that can be used to improve IoT systems by making them more dependable, documenting them as patlets, which summarily describe solutions that address some particular problem within a specific context. We further studied a subset of these patterns as a self-healing pattern language that contemplates the use of more than one pattern in tandem to address systems’ operational concerns autonomically.

Adopting these patterns depends on supporting foundations, which include architectural and functional aspects of the target systems. A key aspect is that most of the current solutions do not provide any features to readjust their intrinsic behaviors during runtime — with the software that runs on edge devices being mostly set on stone, delegating all the computational needs to cloud-based services. The research on fog and edge computing attempt to mitigate this by leveraging computational resources across architectural tiers, making the resulting systems more dependable and improving their scalability. Taking on these foundations, we explored and asserted the feasibility of using serverless functions in the IoT context, optimizing the choice of execution contexts according to a priori preferences, constraints, and latencies.

To understand how these paradigms can be leveraged in widely used solutions, we select the open-source Node-RED solution as the experimental base, given its popularity. It provides a visual programming interface that increases its target user base across different expertise levels. Like other available solutions, Node-RED does not provide any feature that allows it to orchestrate tasks across devices or deal with system parts’ failures, limiting the dependability of systems built with it. Nonetheless, given its open-source and extensible nature, we proceed to address some of its limitations. We proceed to evaluate empirically, both in virtual and physical setups, the feasibility of using Node-RED as an orchestrator, where computational tasks are allocated to the available resources, and failures are mitigated by re-orchestrating as devices fail and recover. We also implemented a set of extensions for Node-RED that allows one to enrich the existing programs (i.e., flows) with self-healing capabilities — allowing the detection errors of different parts during runtime, and readjust its behavior to keep delivering correct service by recovering to normal operation, or, at least, maintain its operation within acceptable Quality-of-Service levels.

As IoT users have different expertise levels, we also attempt to improve the interaction with these systems in a way that the users can understand what the configured automations are (viz. inspection), how it is behaving (viz. observability and feedback), and increase their capability to know what was the possible cause behind certain events (viz. causality). In the first study, we extended the visual notations and functionalities of Node-RED to improve the development process using it. We proceed to empirically evaluate the performance of our solution against a non-modified version of Node-RED, observing statistically significant improvements in the users’ ability to evolve existing IoT deploys. Lastly, we explored the use of voice assistants as an alternative way of configuring, understanding, and interacting with IoT-enriched environments, with a particular focus on the ability of a user to understand the cause behind some events. We assert the feasibility of our solution by covering all the different automation possibilities that Node-RED supports, with a considerable extension of the interaction possibilities due to multi-message dialogs support. We proceeded to empirically validate the feasibility of users using the voice assistant to complete different tasks, and all the users were able to finish the tasks. While some valid sentences were incorrectly recognized, forcing the user to repeat their intent, participants expressed a preference for voice interfaces over visual ones in terms of subjective perception.

These contributions materialize into a core set of building blocks that, in assemble, can be used to improve the dependability of IoT systems while leveraging abstractions that do not hinder the (end-)user capability to configure, use, and evolve them. The experimental counterparts of the contributions provide empirical supporting evidence for the plausibility of the hypothesis.