Banca de QUALIFICAÇÃO: DANIEL QUINTELA BERTUZZI

Uma banca de QUALIFICAÇÃO de MESTRADO foi cadastrada pelo programa.
STUDENT : DANIEL QUINTELA BERTUZZI
DATE: 19/09/2023
TIME: 14:00
LOCAL: https://us02web.zoom.us/j/87932654813?pwd=MkVPRFFNYnB1NzF1eS9GWnM4ZStkQT09
TITLE:

Exploring Interactions between Visual and Auditory Complex Stimuli through Crossmodal Associations and Pupil Responses

PAGES: 50
BIG AREA: Ciências Humanas
AREA: Psicologia
SUMMARY:

Crossmodal correspondences are natural tendencies for observers to make systematic associations between stimuli features from different sensory modalities. The scientific literature has documented the presence of crossmodal correspondences across different pairs of sensory modalities, including visual and tactile, auditory and tactile, gustatory and auditory, and olfactory and visual. Research has primarily centered on investigating simple auditory and visual stimuli, although there is also evidence of crossmodal associations between complex visual and auditory stimuli, but the focus has often been on abstract paintings and basic auditory features. However, the interactions between complex visual and auditory elements and their possible influence on pupil responses remain less understood. The current study aims to investigate the systematic intermodal associations between figurative visual stimuli (specifically, representations of the sun and the moon) and musical excerpts through both semantic differential and association judgment. The first phase seeks to explore whether these associations are consistent with expectations. Additionally, the second phase seeks to uncover how these crossmodal sensory interactions impact pupillary behavior, shedding light on the intricate relationships between complex sensory stimuli and their effects on both perception and physiological responses.

In the first phase, the study employed nine figurative paintings representing the sun, the moon, or diffused light themes, selected from previous work. Additionally, nine musical excerpts were chosen based on a pilot study, representing the sun, the moon, or neither theme. A final sample of 377 participants completed an online survey, consisting of two main tasks: semantic differential and direct association. In the semantic differential task, participants rated paintings and musical excerpts on 15 pairs of bipolar adjectives. In the direct association task, participants matched musical excerpts to paintings, rating the strength of association on a bipolar scale. The results of the study suggest a strong and meaningful relationship between visual artworks (paintings) and auditory stimuli (musical excerpts) in terms of semantic associations and direct associations. This indicates a cross-modal perceptual connection between visual and auditory stimuli. The findings reveal that participants consistently associated certain musical excerpts with specific paintings based on their perceived characteristics, aligning with the intended representation of either the 'sun' or the 'moon'. These associations may occur based on crossmodal characteristics such as semantic, structural, statistical, or emotional.

In the second phase, the study aims to investigate the effects of interactions between the crossmodal sensory stimuli on pupillary behavior; more specifically, we aim to investigate whether songs associated with sun and moon themes had a different impact on pupil responses when presented together with related paintings, and discuss how semantic and direct associations between the songs and the paintings may have influenced perception and pupil responses. Participants will be tested individually using an eye-tracking headset and specific equipment for visual and auditory stimulus presentation. The same nine figurative paintings will be employed, and two musical excerpts will be selected from the first phase for presentation as background music. One of these musical excerpts will be chosen for its higher congruence with moon-related paintings and greater incongruence with sun-related paintings, while the other will present the opposite pattern. Paintings will be presented in blocks with background music or silence. Data will undergo preprocessing, including baseline correction, trial exclusion, and participant exclusion based on data quality. Statistical analysis will be employed by linear mixed effects analysis, conducted to evaluate the effects of painting and music categories on pupil size, considering both pre-determined and perceived perspectives.

COMMITTEE MEMBERS:
Presidente - Interno ao Programa - 1714632 - PETER MAURICE ERNA CLAESSENS
Membro Titular - Examinador(a) Externo à Instituição - IVAN EIJI YAMAUCHI SIMURRA - UFAC
Membro Titular - Examinador(a) Externo à Instituição - MIRELLA GUALTIERI
Membro Suplente - Examinador(a) Interno ao Programa - 1955999 - ANDRE MASCIOLI CRAVO
Membro Suplente - Examinador(a) Externo ao Programa - 3271359 - THENILLE BRAUN JANZEN