Skip to content. | Skip to navigation

Personal tools
Document Actions

Computational Embodied Neuroscience

Material of a course held at the Universiy of Rome, Faculty of Psychology, by Gianluca Baldassarre (held from 2009-2010). The info below refers to the academic year 2012-2013.

General Information

Note: We try our best to plan things in advance, but check this page frequently for changes.

Informal presentations of the course for students (in Italian)

Teachers:

The course is carried out by Prof. Gianluca Baldassarre with the help of his co-worker Francesco Mannella.

Structure of the course:

The course is formed by:

  • 8 theory lectures (3 hours each)
  • 8 hands-on laboratory lessons (3 hours each).

Start of the course:

When: Tuesday 5 Feb 2013, 15:00-18:00.

Where: Via dei Marsi 78, classroom 10.

Timetable

Tuesday: 15:00-18:00, classroom 10

Friday: 15:00-18:00, classroom 7.

Calendar of the lectures:

 

February: 5-8, 12-15, 19-22, 26

March: 1, 5-8, 12-15, 19-22, 28(substitues 26)-29

 

Location:
The course lecturesj and labs take place at the University of ''La Sapienza'', Psychology, Via dei Marsi 78 (San Lorenzo area), room 10 (on Tuesday) and room 7 (on Wednesday), Rome.

Credits, total hours:

6 formation credits: 8 frontal hours for each credit, 48 total hours for the whole course divided in the "theory" and "laboratory" modules (24+24 hours)

Legal status of course:

Universita': Universita' di Roma "La Sapienza"
Facolta': Medicina e Psicologia
Corso di Laurea: Magistrale
Corso di studio: Neuroscienze Cognitive e Riabilitazione Psicologica [LM ORDIN. 2011 - DM 270/04]
Insegnamento: Neuroscienze Computazionali
Tipo corso: non curriculare
Crediti formativi (CFU): 6
Settore disciplinare (SSD): M-PSI/01
Canale: nessuna canalizzazione
Codice: 1032755
Docente: Gianluca Baldassarre

 

LIST AND CONTENTS OF 8 THEORY LECTURES AND 8 LABS

Lecture 1, Computational research approaches
(Tuesday, 5 February 2013, Baldassarre)

Introduction. Goals of course.

What is: a computational model, a neural network, an embodied model.

Overview of the methodological principles of the main approaches to computational modelling: Artificial Intelligence, Cognitive Science, Machine Learning; Connectionism; Artificial Life, Adaptive Behaviour, Developmental Robotics/Epigenetic Robotics; Computational Neuroscience.

The approach followed in the course: Computational Embodied Neuroscience (CEN).
The goal of CEN (understanding the overall brain organisation underlying behaviour, and the type of computational models used to this purpose).
The 8 fundamental methodological principles of CEN.

Material for the exam related to the lecture:

  • Slides integrated with your notes taken at lesson: downloadable file
  • Web-sites cited in the slides, related to the different computational approaches
  • On the CEN method, initial part of paper (downloadable pdf file): Caligiore Daniele, Borghi Anna, Parisi Domenico, Baldassarre Gianluca (2010). TRoPICALS: A Computational Embodied Neuroscience Model of Compatibility Effects. Psycological Review vol. 117, Issue 4, pp. 1188-1228.
  •  Anastasio handbook: Chapters 1, 2, 3.

Lecture 2, Tools, learning paradigms, assoc. learning
(Friday, 22 February 2013, Baldassarre)

Basic mathematical concepts needed for computational modelling: variables,  functions, derivatives, vectors, matrix.

The biological neuron.
Modelling one neuron: sigmoid neuron, leaky neuron, integrate and fire neuron.

Main neural network architectures: feed-forward networks, recurrent networks.

Three (plus 1) learning paradigms and their location in the brain:
(0/3) associative learning; (1/3) unsupervised learning;
(2/3) supervised learning; (3/3) reinforcement learning.

0/3: Associative learning: Hebb rule and its variants.

Models of memory based on Hebb rule: the example of the Hopfield network.

Material for the exam related to the lecture:

  • Slides integrated with your notes taken at lesson: downloadable file
  • Anastasio handbook: Chapters 4.

Lecture 3, Unsupervised learning, brain overall archit.
(Friday, 1 March 2013, Baldassarre)

1/3: Unsupervised learning: the Kohonen learning algorithm.

The overall brain cortical architecture: sensory, motor, and decision making brain;
two dorsal and the ventral neural routes of what and how; the prefrontal decision making areas

Case study on associative and unsupervised learning: modelling compatibility effects in psychological experiments with a focus on cortex

Material for the exam related to the lecture:

  • Slides integrated with your notes taken at lesson: downloadable file
  • Anastasio handbook: Chapters 5.
  • On the overall organisation of cortical brain, see the paper (downlodable file): Cisek, P. & Kalaska, J. F.. 2010. Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci, Vol. 33. pp. 269-298.
  • On the case study, see the paper (downloadable pdf file): Caligiore Daniele, Borghi Anna, Parisi Domenico, Baldassarre Gianluca (2010). TRoPICALS: A Computational Embodied Neuroscience Model of Compatibility Effects. Psycological Review vol. 117, Issue 4, pp. 1188-1228.

Lecture 4, Trial-and-error learning, basal ganglia
(Friday, 8 March 2013, Baldassarre)

2/3: Supervised learning: the delta rule for two layer networks and for multi-layer networks.

The origin of trial-and-error learning (or reinforcement learning):
the Rescorla-Wagner model of conditioning experiments.

3/3: Most used bio-inspired model of RL: Actor-Critic reinforcement-learning model.

Biological correspodents of the actor-critic model: basal ganglia for action learning and selection and dopamine as a learning signal.

Case study on reinforcement learning: A reinforcement-learning model of navigation in chicks (and rats).

Material for the exam related to the lecture:

  • Slides integrated with your notes taken at lesson: downloadable file
  • Anastasio handbook: Chapters 6, 11.
  • On the case study, see paper (downloadable file): Mannella F., Baldassarre G. (2007). A neural-network reinforcement-learning model of domestic chicks that learn to localise the centre of closed arenas. Philosophical Transactions of the Royal Society B – Biological Sciences. Vol. 362 (No. 1479), pp. 383-401.

Lecture 5, Hierarchical actions and basal ganglia
(Friday, 15 March 2013, Baldassarre)

Introduction to the problem of hierarchical actions: biology and theory of basal ganglia-cortical loops, and the hierarchical organisation of action in brain.

The GPR model of basal ganglia.

Case study on hierarchical actions: A model of the hierarchical organisation of action in brain, based on reinforcement learning algorithms and architectures, that accomplishes multiple tasks.

Material for the exam related to the lecture:

 

  • Slides integrated with your notes taken at lesson: downloadable file
  • On the hierarchical organisation of basal ganglia and actions, see paper (downloadable file): Yin, H. H. & Knowlton, B. J.. 2006. The role of the basal ganglia in habit formation. Nat Rev Neurosci, Vol. 7 (No. 6). pp. 464-476.

Lecture 6, Motivations and hierarchical actions
(Friday, 22 March 2012, Baldassarre)

Case study of a European project on Cognitive Science and Autonomous Robotics: ''IM-CLeVeR - Intrinsically Motivated Cumulative Learning Versatile Robots''

Introduction to motivations: extrinsic motivations (e.g., pivoting on amygdala and hypothalamus) and intrinsic motivations (superior colliculus, hippocampus).

Case study on an integrative model on intrinsic motivations, extrinsic motivations, and the hierarchical organisation of actions in brain. The model encompasses:
- superior colliculus and learning of actions based on intrinsic motivations;
- the amygdala-based system for extrinsic motivations for goal-directed behaviour (the amygdala-based system n. 6/6, see below);
- the hierarchical organisation of actions in cortex and basal ganglia;

Material for the exam related to the lecture:

  • Slides relate to intrinsic motivations integrated with your notes taken at lesson: downloadable file
  • The model of the case study is under publication, so the slides related to it cannot be furnished. Same for the slides related to the project. These two topics will not be asked at the exam.

Lecture 7, Amygdala and extrinsic motivations
(Friday, 29 March 2013, Baldassarre/Mannella)

(Extrinsic) emotions and motivations: the amygdala, the hub of emotions and values. The two types of associations it performs.

The amygdala as part of 6 brain sub-systems involving the emotional regulation of brain and behaviour.

Hints to the amygdala-based system n. 1/6 underlying body regulation.

The amygdala-based system n. 3/6 underlying the seletion of innate behaviours: the case study of a model of autoshaping.

Hints to the amygdala-based system n. 4/6 underlying memory formation.

The amygdala-based system n. 5/6 underlying the selection of instrumentally-acquired behaviours: the case study of a model of devaluation.

Material for the exam related to the lecture:

  • Slides integrated with your notes taken at lesson: downloadable file
  • On the amygdala functions, see review paper (downloadable file): Mirolli Marco, Mannella Francesco, Baldassarre Gianluca(2010). The roles of the amygdala in the affective regulation of body, brain and behaviour. Connection Science. 22 (3), 215-245.
  • In relation to the case study of devaluation, you can see the paper (downloadable file): Mannella Francesco, Mirolli Marco, Baldassarre Gianluca (2010). The interplay of pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat. In Tosh Colin and Ruxton Graeme (eds), "Modelling Perception With Artificial Neural Networks", pp. 93-113. Cambridge, Cambridge University press.

Lecture 8, Models of stress coping and neuromodulators
(Tesday, 2 April 2012, Baldassarre)

Fiore:

Case study: a model of stress coping and broad neuromodulation of brain (related to amygdala-based system n. 2/6). A few details about the simulation of the effects of the neuromodulators on the synaptic transmission.

Material for the exam related to the lecture:

Slides used for the lecture: downloadable pdf

Baldassarre:

Prof. Baldassarre will help students attending the lesson to choose a topic and a model for the exam thesis.

Material for the exam related to the lecture:

  • Slides related to stress coping, integrated with your notes taken at lesson: downloadable file

Lab 1, Introduction to Matlab 1 and first examples
(Tuesday, 12 February 2013, Mannella)

Introduction to Matlab I.

  • Scalar variables
  • Vectors 
  • Matrices
  • Use of indices
  • operations on vectors and matrices

In the course of this lessons we will start to show some examples.

Lesson notes.

 

 

Lab 2, Introduction to Matlab 2 and first simple models
(Friday, 15 February 2013, Mannella)

Introduction to Matlab II.

  • Comparing operators.
  • Conditional operators.
  •  the istruction 'find'.
  • 'if' statements.
  • 'for' and 'when' statements.
  • Scripts.
  • User-defined unctions.

Lesson notes.

 

Lab 3, Associative learning and models

(Tuesday, 19 February 2013, Mannella)

 

Modelling hopfield networks in matlab. External product and the hebbian rule.

Elements of plotting in matlab.

Lab 4, Associative learning and models
(Tuesday, 26 February 2013, Mannella)

Two examples of matlab code for neural networks (NN).

The first NN is characterised by two layers (3 and 2 units) and a competition between the units of the output layer (Shoner/Bogacs competition).

The second NN shows a Hopfield associator: 6 units working as both input and output, wholly interconnected.

The complete code used in the lecture (please note the image at the beginning of the document exemplifying the first network): downloadable pdf

Lab 5, Unsupervised learning and models
(Tuesday, 5 March 2013, Mannella/Fiore)

A practical implementation of a kohonen map.

A commented version of the code used in the lecture: dovnloadable pdf

Lab 6, Supervised learning and models
(Tuesday, 12 March 2013, Fiore)

Two examples of matlab code for Kohonen-like neural networks.

The first system receives as inputs 8 simple images (10x10 pixels showing black lines on a white background) and learns to create proper discrete categories.

The second system modifies the example explained during Lab5, showing the way it works in the continuous, using leaky integrator.

The complete code used in the lecture and the example images can be dowload as an archive here: Kohonen_Fiore

Lab 7, Reinforcement learning and models
(Tuesday, 19 March 2013, Mannella/Fiore)

Temporal-Difference reinforcement learning: basic elements of its implementation.

Lab 8, Reinforcement learning and models
(Tuesday, 26 March 2013, Mannella/Fiore)

Implementation of the Temporal-Difference reinforcement learning Actor-Ctitic algorithm in matlab. 

The complete code used in the lecture can be dowloaded here: downloadable file

Guide to the preparation of the exam in relation to the short thesis and the oral test based on the lectures' material

This section indicates how to prepare the exam. It distinguishes between the students that attended the lectures and those that did not.

Students that have attended the lectures and labs

The preparation of the students that attended the course has to achieve this goals:

  • Carry out a small research, based on a computational model programmed  by the student, on a topic chosen from those considered during the course. This should lead also to compile a short thesis (''tesina''). The short thesis has to be maximum 12 pages long (use character 12) including figures, references and all the rest. The first page should report: Author(s), title, abstract (max 250 words; indicating topic, problem, type, of model, and results), ''Tesina per il corso Neuroscienze Computazionali, Anno accademico 20xx-20xx'', Prof. Gianluca Baldassarre, tutors (if present). The short thesis can be either in Italian or in English. The short thesis has to be sent to Gianluca at least 1 week before the exam via email.
  • The students have also to sustain an oral exam. This will be on the ''tesina'' and on the issues explained during the theoretical part of the course. The student should prepare this part of the exam on the basis of the slides, suitably integrated with your notes and the info drawn from the papers available below, and on the basis of the sections of the handbook Anastasio indicated below, and on the basis of 1 freely-selected paper among the papers listed below.

The course handbook is:

  • Anastasio, J. Thomas (2010). Neural Systems Modelling. Sunderland, MA: Sinauer.

Other possible reference books are:

  • Rolls T. Edmund, Treves Alessandro (1998). Neural networks and brain function. Oxford: Oxford University Press. (Good initial part on neural networks and their learning algorithms from a strongly biological perspective; focussed chapters on accurate models of hippocampus, amygdala, cortex, cerebellum and basal ganglia)
  • Thomas P. Trappenberg (2010).''Fundamentals of Computational Neuroscience''. Oxford: Oxford University Press. (More difficult but more accurate and more into computational neuroscience).

You can buy these books either from Amazon or from an international library.

Use the following papers to integrate the lectures' slides and your notes on them. Also, you should study 1 of this papers in depth, at your choice (this will be discussed at the oral exam):

  • Paper on the "overall hierarchical organisation of cortical brain": Cisek, P. & Kalaska, J. F.. 2010. Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci, Vol. 33. pp. 269-298.
  • Paper on the ''overall hierarchical organisation of basal ganglia'': Yin, H. H. & Knowlton, B. J.. 2006. The role of the basal ganglia in habit formation. Nat Rev Neurosci, Vol. 7 (No. 6). pp. 464-476.
  • Paper on ''chicks'': Mannella F., Baldassarre G. (2007). A neural-network reinforcement-learning model of domestic chicks that learn to localise the centre of closed arenas. Philosophical Transactions of the Royal Society B – Biological Sciences. Vol. 362 (No. 1479), pp. 383-401.
  • Paper on ''TRoPICALS model'': Caligiore Daniele, Borghi Anna, Parisi Domenico, Baldassarre Gianluca (2010). TRoPICALS: A Computational Embodied Neuroscience Model of Compatibility Effects. Psycological Review vol. 117, Issue 4, pp. 1188-1228.
  • Paper on ''amygdala'': Mirolli Marco, Mannella Francesco, Baldassarre Gianluca(2010). The roles of the amygdala in the affective regulation of body, brain and behaviour. Connection Science. 22 (3), 215-245.
  • Paper on ''instrumental devaluation'': Mannella Francesco, Mirolli Marco, Baldassarre Gianluca (2010). The interplay of pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat. In Tosh Colin and Ruxton Graeme (eds), "Modelling Perception With Artificial Neural Networks", pp. 93-113. Cambridge, Cambridge University press.

This is a complete list of parts you should study from the Anastasio handbook:
Chapters: 1, 2, 3, 4, 5, 6, 11.

You can also find complementary information on these issues in the book of Trappenberg:

  • Chapter 1: Introduction
  • Paragraph 2.1: Biological background
  • Paragraph 2.2: Basic synaptic mechanisms and dendritic processing
  • Paragraph 3.1.1: The leaky integrate-and-fire neuron
  • Paragraph 4.1: Associative memory and Hebbian learning
  • Paragraph 4.3: Mathematical formulation of Hebbian plasticity
  • Paragraph 7.1: Competitive feature representations in cortical tissue
  • Paragraph 7.2: Self-organising maps
  • Paragraph 9.6: Reinforcement learning

Students that did not attend the lectures and labs

The preparation of the students that could not attend the course have to be as for the students who attended. As they could not take notes, they should integrate the slides with the information from the papers listed above, from internet, and from other parts of the book Anastasio not listed above. The students should study Matlab based on the material indicated above. Also the students who did not attend the course should send their thesis to Gianluca at least 1 week before the exam via email.