EBRAINS neuromorphic systems stand to benefit from new algorithm inspired by the brain

Researchers at TU Graz demonstrate a new design method for particularly energy-saving artificial neural networks that get by with extremely few signals and – similar to Morse code – also assign meaning to the pauses between the signals. The work was funded by the European Human Brain Project (HBP). The novel algorithm will be implemented on brain-inspired computing systems of the HBP’s EBRAINS research infrastructure.

Picture: The two brain-inspired, or “neuromorphic” computing systems of EBRAINS: The spike-based Spinnaker system (left), and the BrainScaleS system (right), which combines analogue and digital elements. https://ebrains.eu/service/neuromorphic-computing

Most new achievements in artificial intelligence (AI) require very large neural networks. They consist of hundreds of millions of neurons arranged in several hundred layers, i.e. they have very "deep" network structures. These large, deep neural networks consume a lot of energy in the computer. Those neural networks that are used in image classification (e.g. face and object recognition) are particularly energy-intensive, since they have to send very many numerical values from one neuron layer to the next with great accuracy in each time cycle. 

Computer scientist Wolfgang Maass, together with his PhD student Christoph Stöckl, has now found a design method for artificial neural networks that paves the way for energy-efficient high-performance AI hardware. The two researchers from the Institute of Theoretical Computer Science at Graz University of Technology (TU Graz) have optimized artificial neuronal networks in computer simulations for image classification in such a way that the neurons – similar to neurons in the brain – only need to send out signals relatively rarely and those that they do are very simple.

The proven classification accuracy of images with this design is nevertheless very close to the current state of the art of current image classification tools.

Information processing in the human brain as a paradigm

Maass and Stöckl were inspired by the way the human brain works. It processes several trillion computing operations per second, but only requires about 20 watts. This low energy consumption is made possible by inter-neuronal communication by means of very simple electrical impulses, so-called spikes. The information is thereby encoded not only by the number of spikes, but also by their time-varying patterns. "You can think of it like Morse code. The pauses between the signals also transmit information," Maass explains.

Conversion method for trained artificial neural networks

That spike-based hardware can reduce the energy consumption of neural network applications is not new. However, so far this could not be realized for the very deep and large neural networks that are needed for really good image classification.

In the design method of Maass and Stöckl, the transmission of information now depends not only on how many spikes a neuron sends out, but also on when the neuron sends out these spikes. The time or the temporal intervals between the spikes practically encode themselves and can therefore transmit a great deal of additional information. "We show that with just a few spikes – an average of two in our simulations – as much information can be conveyed between processors as in more energy-intensive hardware," Maass said.

With their results, the two computer scientists from TU Graz provide a new approach for hardware that combines few spikes and thus low energy consumption with state-of-the-art performances of AI applications. The findings could dramatically accelerate the development of energy-efficient AI applications and are described in the journal Nature Machine Intelligence and elsewhere. 

Contributes to the neuromorphic platform on EBRAINS

Like previous developments of the team, the algorithm will be implemented on the brain-inspired computing systems SpiNNaker and BrainScales, which are part of the Human Brain Project's digital EBRAINS research infrastructure.

Adapted with permission from Technical University of Graz.
 

This research work is anchored in the Fields of Expertise "Human and Biotechnology" and "Information, Communication & Computing", two of the five Fields of Expertise of TU Graz. It was funded by the European Human Brain Project, which combines neuroscience, medicine and the development of brain-inspired technologies.

Publication:
Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes.
C. Stoeckl and W. Maass.
DOI: 10.1038/s42256-021-00311-4

Contact:
TU Graz | Institute of Theoretical Computer Science
Wolfgang MAASS
Em.Univ.-Prof. Dipl.-Ing. Dr.rer.nat.
Tel.: +43 316 873 5822
maass@igi.tugraz.at

Christoph STÖCKL
Dipl.-Ing. BSc
Tel.: +43 316 873 5847
stoeckl@tugraz.at

See also:

Optimizing neural networks on a brain-inspired computer

New research now shows how so-called “critical states” can be used to optimize artificial neural networks running on brain-inspired neuromorphic hardware. Read more

New learning algorithm should significantly expand the possible applications of AI

The e-prop learning method forms the basis for drastically more energy-efficient hardware implementations of Artificial Intelligence. Read more
New approach for a biological programming language

Researchers have succeeded in mathematically modelling the emergence and interaction between so-called "assemblies". Read more


Media Contact:
Peter Zekert
Tel.: +49 2461 61 96860
press@humanbrainproject.eu 

Further information:
www.ebrains.eu
www.humanbrainproject.eu

ABOUT THE HBP

The Human Brain Project (HBP) is the largest brain science project in Europe and stands among the biggest research projects ever funded by the European Union. At the interface of neuroscience and information technology, the HBP investigates the brain and its diseases with the help of highly advanced methods from computing, neuroinformatics and artificial intelligence, and drives innovation in fields like brain-inspired computing and neurorobotics.

ABOUT EBRAINS

EBRAINS is a new digital research infrastructure, created by the EU-funded Human Brain Project, to foster brain-related research and to help translate the latest scientific discoveries into innovation in medicine and industry, for the benefit of patients and society.

It draws on cutting-edge neuroscience and offers an extensive range of brain data sets, a multilevel brain atlas, modelling and simulation tools, easy access to high-performance computing resources and to robotics and neuromorphic platforms. 

All academic researchers have open access to EBRAINS’ state-of-the art services.  Industry researchers are also very welcome to use the platform under specific agreements.  For more information about EBRAINS, please contact us at info@ebrains.eu or visit www.ebrains.eu.