Computational Embodied Neuroscience

General Information

Note: We try our best to plan things in advance, but check this page frequently for changes.

Informal presentations of the course for students (in Italian)

The course is carried out by Prof. Gianluca Baldassarre with the help of his co-workers Francesco Mannella and Vincenzo Fiore.

Structure of the course:

The course is formed by:

• 8 theory lectures (3 hours each)
  
• 8 hands-on laboratory lessons (3 hours each).

Start of the course:
Tuesday 7 February 2012, 15:00-18:00, Via dei Marsi 78, room 10

Calendar of the lessons:

February: 7-8, 14-15, 21-22, 28-        
March: 1, 6-7, 13-14, 20-21, 27-28

Tuesdays:        15:00-18:00 (3 hours), Room 10
Wednesdays: 15:00-18:00 (3 hours), Room 7

Location:
The course lessons take place at the University of ''La Sapienza'', Psychology, Via dei Marsi 78 (San Lorenzo area), room 10 (on Tuesday) and room 7 (on Wednesday), Rome.

Credits, total hours:

6 formation credits: 8 frontal hours for each credit, 48 total hours for the whole course divided in the "theory" and "laboratory" modules (24+24 hours)

Universita' di Roma "La Sapienza", Facolta' di Medicina e Psicologia, Settore disciplinare M-PSI/01, Corso non curriculare, Laurea Magistrale.

LIST AND CONTENTS OF 8 THEORY LECTURES AND 8 LABS

Lecture 1, Computational research approaches
(Tuesday, 7 February 2012, Baldassarre)

What is: a computational model, a neural network, an embodied model.

Overview of the methodological principles of the main approaches to computational modelling: Artificial Intelligence, Cognitive Science, Machine Learning; Connectionism; Artificial Life, Adaptive Behaviour, Developmental Robotics/Epigenetic Robotics; Computational Neuroscience.

The approach followed in the course: Computational Embodied Neuroscience (CEN).
The goal of CEN (understanding the overall brain organisation underlying behaviour, and the type of computational models used to this purpose).
The 8 fundamental methodological principles of CEN.

Material for the exam related to the lecture:

• Slides integrated with your notes taken at lesson: downloadable file
  
• Web-sites cited in the slides, related to the different computational approaches
• On the CEN method, initial part of paper (downloadable pdf file): Caligiore Daniele, Borghi Anna, Parisi Domenico, Baldassarre Gianluca (2010). TRoPICALS: A Computational Embodied Neuroscience Model of Compatibility Effects. Psycological Review vol. 117, Issue 4, pp. 1188-1228.
•  Anastasio handbook: Chapters 1, 2, 3.
  

Lecture 2, Tools, learning paradigms, assoc. learning
(Wednesday, 8 February 2012, Baldassarre)

Basic mathematical concepts needed for computational modelling: variables,  functions, derivatives, vectors, matrix.

The biological neuron.
Modelling one neuron: sigmoid neuron, leaky neuron, integrate and fire neuron.

Main neural network architectures: feed-forward networks, recurrent networks.

Three (plus 1) learning paradigms and their location in the brain:
(0/3) associative learning; (1/3) unsupervised learning;
(2/3) supervised learning; (3/3) reinforcement learning.

0/3: Associative learning: Hebb rule and its variants.

Models of memory based on Hebb rule: the example of the Hopfield network.

Material for the exam related to the lecture:

• Slides integrated with your notes taken at lesson: downloadable file
  
• Anastasio handbook: Chapters 4.
  

Lecture 3, Unsupervised learning, brain overall archit.
(Tuesday, 14 February 2012, Baldassarre)

1/3: Unsupervised learning: the Kohonen learning algorithm.

The overall brain cortical architecture: sensory, motor, and decision making brain;
two dorsal and the ventral neural routes of what and how; the prefrontal decision making areas

Case study on associative and unsupervised learning: modelling compatibility effects in psychological experiments with a focus on cortex

Material for the exam related to the lecture:

• Slides integrated with your notes taken at lesson: downloadable file
  
• Anastasio handbook: Chapters 5.
• On the overall organisation of cortical brain, see the paper (downlodable file): Cisek, P. & Kalaska, J. F.. 2010. Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci, Vol. 33. pp. 269-298.
• On the case study, see the paper (downloadable pdf file): Caligiore Daniele, Borghi Anna, Parisi Domenico, Baldassarre Gianluca (2010). TRoPICALS: A Computational Embodied Neuroscience Model of Compatibility Effects. Psycological Review vol. 117, Issue 4, pp. 1188-1228.

Lecture 4, Trial-and-error learning, basal ganglia
(Wednesday, 15 February 2012, Baldassarre)

2/3: Supervised learning: the delta rule for two layer networks and for multi-layer networks.

The origin of trial-and-error learning (or reinforcement learning):
the Rescorla-Wagner model of conditioning experiments.

3/3: Most used bio-inspired model of RL: Actor-Critic reinforcement-learning model.

Biological correspodents of the actor-critic model: basal ganglia for action learning and selection and dopamine as a learning signal.

Case study on reinforcement learning: A reinforcement-learning model of navigation in chicks (and rats).

Material for the exam related to the lecture:

• Slides integrated with your notes taken at lesson: downloadable file
  
• Anastasio handbook: Chapters 6, 11.
• On the case study, see paper (downloadable file): Mannella F., Baldassarre G. (2007). A neural-network reinforcement-learning model of domestic chicks that learn to localise the centre of closed arenas. Philosophical Transactions of the Royal Society B – Biological Sciences. Vol. 362 (No. 1479), pp. 383-401.

Lecture 5, Hierarchical actions and basal ganglia
(Tusday, 21 February 2012, Baldassarre)

Introduction to the problem of hierarchical actions: biology and theory of basal ganglia-cortical loops, and the hierarchical organisation of action in brain.

The GPR model of basal ganglia.

Case study on hierarchical actions: A model of the hierarchical organisation of action in brain, based on reinforcement learning algorithms and architectures, that accomplishes multiple tasks.

Material for the exam related to the lecture:

• Slides integrated with your notes taken at lesson: downloadable file
• On the hierarchical organisation of basal ganglia and actions, see paper (downloadable file): Yin, H. H. & Knowlton, B. J.. 2006. The role of the basal ganglia in habit formation. Nat Rev Neurosci, Vol. 7 (No. 6). pp. 464-476.

Lab 1, Introduction to Matlab 1 and first examples
(Wednesday, 22 February 2012, Mannella)

Introduction to Matlab.

Lecture 6, Motivations and hierarchical actions
(Tusday, 28 February 2012, Baldassarre)

Case study of a European project on Cognitive Science and Autonomous Robotics: ''IM-CLeVeR - Intrinsically Motivated Cumulative Learning Versatile Robots''

Introduction to motivations: extrinsic motivations (e.g., pivoting on amygdala and hypothalamus) and intrinsic motivations (superior colliculus, hippocampus).

Case study on an integrative model on intrinsic motivations, extrinsic motivations, and the hierarchical organisation of actions in brain. The model encompasses:
- superior colliculus and learning of actions based on intrinsic motivations;
- the amygdala-based system for extrinsic motivations for goal-directed behaviour (the amygdala-based system n. 6/6, see below);
- the hierarchical organisation of actions in cortex and basal ganglia;

Material for the exam related to the lecture:

• Slides integrated with your notes taken at lesson: downloadable file
• In relation to the case study, the model is under publication. However, you can see the part of the architecture implementing extrinsic motivations and hierarchical actions in this paper (downloadable file): Mannella Francesco, Mirolli Marco, Baldassarre Gianluca (2010). The interplay of pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat. In Tosh Colin and Ruxton Graeme (eds), "Modelling Perception With Artificial Neural Networks", pp. 93-113. Cambridge, Cambridge University press.

Lab 2, Introduction to Matlab 2 and first simple models
(Wednesday, 1 March 2012, Mannella)

Matlab contents:
...to be defined...

In the course of this lessons we will start to show some examples.

Lecture 7, Amygdala and extrinsic motivations 1
(Tusday, 6 Marzo 2012, Mannella)

(Extrinsic) emotions and motivations: the amygdala, the hub of emotions and values. The two types of associations it performs.

The amygdala as part of 6 brain sub-systems involving the emotional regulation of brain and behaviour.

Hints to the amygdala-based system n. 1/6 underlying body regulation.

The amygdala-based system n. 3/6 underlying the seletion of innate behaviours: the case study of a model of autoshaping.

Hints to the amygdala-based system n. 4/6 underlying memory formation.

The amygdala-based system n. 5/6 underlying the selection of instrumentally-acquired behaviours: the case study of a model of devaluation.

Material for the exam related to the lecture:

• Slides integrated with your notes taken at lesson: downloadable file
• On the amygdala functions, see review paper (downloadable file): Mirolli Marco, Mannella Francesco, Baldassarre Gianluca(2010). The roles of the amygdala in the affective regulation of body, brain and behaviour. Connection Science. 22 (3), 215-245.
• In relation to the case study of devaluation, you can see the paper (downloadable file): Mannella Francesco, Mirolli Marco, Baldassarre Gianluca (2010). The interplay of pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat. In Tosh Colin and Ruxton Graeme (eds), "Modelling Perception With Artificial Neural Networks", pp. 93-113. Cambridge, Cambridge University press.

Lab 3, Associative learning and models
(Wednesday, 7 March 2012, Mannella/Fiore)

TO BE DEFINED

Lecture 8, Amygdala and extrinsic motivations 2
(Tusday, 13 Marzo 2012, Fiore/Baldassarre)

Fiore:

Case study: a model of stress coping and broad neuromodulation of brain (amygdala-based system n. 2/6).

Baldassarre:

Plans for the students thesis for the exam.

Material for the exam related to the lecture:

• Slides integrated with your notes taken at lesson: downloadable file

Lab 4, Associative learning and models
(Wednesday, 14 March 2012, Mannella/Fiore)

TO BE DEFINED

Lab 5, Unsupervised learning and models
(Tuesday, 20 March 2012, Mannella/Fiore)

TO BE DEFINED

Lab 6, Supervised learning and models
(Wednesday, 21 March 2012, Mannella/Fiore)

TO BE DEFINED

Lab 7, Reinforcement learning and models
(Tuesday, 27 March 2012, Mannella/Fiore)

TO BE DEFINED

Lab 8, Reinforcement learning and models
(Wednesday, 28 March 2012, Mannella/Fiore)

TO BE DEFINED

Guide to the preparation of the exam in relation to the short thesis and the oral test based on the lectures' material

This section indicates how to prepare the exam. It distinguishes between the students that attended the lectures and those that did not.

Students that have attended the lectures and labs

The preparation of the students that attended the course has to achieve this goals:

• Carry out a small research, based on a computational model programmed  by the student, on a topic chosen from those considered during the course. This should lead also to compile a short thesis (''tesina''). The short thesis has to be maximum 12 pages long (use character 12) including figures, references and all the rest. The first page should report: Author(s), title, abstract (max 250 words; indicating topic, problem, type, of model, and results), ''Tesina per il corso Neuroscienze Computazionali, Anno accademico 20xx-20xx'', Prof. Gianluca Baldassarre, tutors (if present). The short thesis can be either in Italian or in English. The short thesis has to be sent to Gianluca at least 1 week before the exam via email.
  
• The students have also to sustain an oral exam. This will be on the ''tesina'' and on the issues explained during the theoretical part of the course. The student should prepare this part of the exam on the basis of the slides, suitably integrated with your notes and the info drawn from the papers available below, and on the basis of the sections of the handbook Anastasio indicated below, and on the basis of 1 freely-selected paper among the papers listed below.
  

The course handbook is:

• Anastasio, J. Thomas (2010). Neural Systems Modelling. Sunderland, MA: Sinauer.
  

Other possible reference books are:

• Rolls T. Edmund, Treves Alessandro (1998). Neural networks and brain function. Oxford: Oxford University Press. (Good initial part on neural networks and their learning algorithms from a strongly biological perspective; focussed chapters on accurate models of hippocampus, amygdala, cortex, cerebellum and basal ganglia)
  
• Thomas P. Trappenberg (2010).''Fundamentals of Computational Neuroscience''. Oxford: Oxford University Press. (More difficult but more accurate and more into computational neuroscience).

You can buy these books either from Amazon or from an international library.

Use the following papers to integrate the lectures' slides and your notes on them. Also, you should study 1 of this papers in depth, at your choice (this will be discussed at the oral exam):

• Paper on the "overall hierarchical organisation of cortical brain": Cisek, P. & Kalaska, J. F.. 2010. Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci, Vol. 33. pp. 269-298.
• Paper on the ''overall hierarchical organisation of basal ganglia'': Yin, H. H. & Knowlton, B. J.. 2006. The role of the basal ganglia in habit formation. Nat Rev Neurosci, Vol. 7 (No. 6). pp. 464-476.
• Paper on ''chicks'': Mannella F., Baldassarre G. (2007). A neural-network reinforcement-learning model of domestic chicks that learn to localise the centre of closed arenas. Philosophical Transactions of the Royal Society B – Biological Sciences. Vol. 362 (No. 1479), pp. 383-401.
• Paper on ''TRoPICALS model'': Caligiore Daniele, Borghi Anna, Parisi Domenico, Baldassarre Gianluca (2010). TRoPICALS: A Computational Embodied Neuroscience Model of Compatibility Effects. Psycological Review vol. 117, Issue 4, pp. 1188-1228.
• Paper on ''amygdala'': Mirolli Marco, Mannella Francesco, Baldassarre Gianluca(2010). The roles of the amygdala in the affective regulation of body, brain and behaviour. Connection Science. 22 (3), 215-245.
• Paper on ''instrumental devaluation'': Mannella Francesco, Mirolli Marco, Baldassarre Gianluca (2010). The interplay of pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat. In Tosh Colin and Ruxton Graeme (eds), "Modelling Perception With Artificial Neural Networks", pp. 93-113. Cambridge, Cambridge University press.

This is a complete list of parts you should study from the Anastasio handbook:
Chapters: 1, 2, 3, 4, 5, 6, 11.

You can also find complementary information on these issues in the book of Trappenberg:

• Chapter 1: Introduction
• Paragraph 2.1: Biological background
• Paragraph 2.2: Basic synaptic mechanisms and dendritic processing
• Paragraph 3.1.1: The leaky integrate-and-fire neuron
• Paragraph 4.1: Associative memory and Hebbian learning
• Paragraph 4.3: Mathematical formulation of Hebbian plasticity
• Paragraph 7.1: Competitive feature representations in cortical tissue
• Paragraph 7.2: Self-organising maps
• Paragraph 9.6: Reinforcement learning

Students that did not attend the course

The preparation of the students that could not attend the course have to be as for the students who attended. As they could not take notes, they should integrate the slides with the information from the papers listed above, from internet, and from other parts of the book Anastasio not listed above. The students should study Matlab based on the manuals indicated above.

Material for the study of Matlab and the realisation of the models for the short thesis

...to be done...