Two opposing approaches stand out in the field of AI learning: error-based and target-based. Scientists also debate which one is more likely to be implemented in biological networks of neurons. Researchers of the Human Brain Project (HBP) now propose a unified framework for learning in recurrent spiking networks. Their work reconciles the two approaches in the field of supervised machine learning and demonstrates how such networks can solve different tasks: The results were published in the journal PLoS Computational Biology.

A recurrent neural network (RNN) is a type of artificial network that works with sequential data and can be used in tasks such as language translation, speech recognition and image captioning. These deep-learning algorithms are called “recurrent” because they repeatedly perform the same operation, taking information from the output of previous steps. With this process, RNN can retain “memory”.

There are many different proposed learning rules and protocols for recurrent neural networks. In this work, the HBP researchers from the Istituto Nazionale di Fisica Nucleare (INFN, Italy) and the Scuola Internazionale Superiore di Studi Avanzati (SISSA, Italy) aimed to define a generalised framework, to move a step forward towards the unification in this fragmented scenario.

The novel general framework provided by this recent work, which the authors call GOAL (Generalized Optimization of Apprenticeship Learning), reconciles these different approaches.

One method that can be implemented in supervised machine learning is the so-called behavioural cloning, which is a form of learning by imitation. In this framework, an agent observes an expert behaviour, and increasingly improves its mimicking performance by minimising the differences between its own and the expert’s behaviour. 

The HBP team applied their novel framework to behavioural cloning in recurrent spiking network and efficiently solved close-looped tasks – in this case, “Button and Food”, (a navigation task that requires retaining memory for a long time), and “2D Bipedal Walker”, a motor task. 

These are highly specialised tasks, and each one requires a different learning regime. The new framework allows for efficiently solving both of them, by investigating what parameters are best for each specific task.

This indicates that this approach becomes more general, more versatile, and in that sense, more like our brains – which can have different learning regimes in its different areas.

The scientists want to further explore a novel route opened by such a framework: what feedback strategy is actually implemented by biological networks and in the different brain areas. 

“For instance, one can speculate that motor regions (e.g., the motor cortex) implement an error-based strategy, whereas memory regions (e.g., hippocampus) exploit a target-based strategy,” explains Cristiano Capone, a researcher at INFN and one of the authors of the study. “In our model, those strategies are the two faces of the same framework.”

Link to the publication:

Error-based or target-based? A unified framework for learning in recurrent spiking networks
Cristiano Capone, Paolo Muratore, Pier Stanislao Paolucci