ESI TALKS: Gemma Roig

We are reviving the ESI Lecture series!

At the bi-weekly event, we invite speakers from Neuroscience and related fields to share their current research in an informal and casual environment. The ESITALKS are on Fridays at 11 am CET (or Thursdays at 5 pm CET) and last around 45 minutes, followed by discussions.

We are very happy to invite you to the fifth ESITALK of this new series, which has the following title:

Task-specific vision DNN models and their relation for explaining different areas of the visual cortex

Deep Neural Networks (DNNs) are state-of-the-art models for many vision tasks. We propose an approach to assess the relationship between visual tasks and their task-specific models. Our method uses Representation Similarity Analysis (RSA), which is commonly used to find a correlation between neuronal responses from brain data and models. With RSA we obtain a similarity score among tasks by computing correlations between models trained on different tasks. We demonstrate the effectiveness and efficiency of our method to generating task taxonomy on the Taskonomy dataset. Also, results on PASCAL VOC suggest that initializing the models trained on tasks with higher similarity score show higher transfer learning performance. Finally, we explore the power of DNNs trained on 2D, 3D, and semantic visual tasks as a tool to gain insights into functions of visual brain areas (early visual cortex (EVC), OPA and PPA). We find that EVC representation is more similar to early layers of all DNNs and deeper layers of 2D-task DNNs. OPA representation is more similar to deeper layers of 3D DNNs, whereas PPA representation to deeper layers of semantic DNNs. We extend our study to performing searchlight analysis using such task specific DNN representations to generate task-specificity maps of the visual cortex. Our findings suggest that DNNs trained on a diverse set of visual tasks can be used to gain insights into functions of the visual cortex. This method has the potential to be applied beyond visual brain areas.

Recommended reading prior to the talk:
Unveiling functions of the visual cortex using task-specific deep neural networks (project page w. paper, code & data)
Unraveling representations in scene-selective brain regions using scene parsing deep neural networks
Duality diagram similarity: a generic framework for initialization selection in task transfer learning
Representation similarity analysis for efficient task taxonomy and transfer learning

To register for the upcoming session of ESITALKS please send an email to one of the organizers: christini.katsanevaki(at) or tim.naeher(at)
Please consider our privacy notice.