SANeTouch

Shared Autonomy Neuroprosthetics: Bridging the gap between Artificial and Biological Touch

Most commercial upper limb prosthetics do not include sensory feedback, an essential component to regain full functionality of the missing limb. This fellowship proposes to exploit recent developments in neural networks and artificial tactile sensors to answer the following research question: What are the most effective feedback strategies for upper-limb neuroprosthetics, and how should shared autonomy protocols be designed to enhance upper limb amputees’ user experience?

Most commercial upper-limb prosthetics currently lack tactile feedback. This is a notable deficiency since tactile afferents in human hands provide essential information on the geometry, texture and motion of grasped objects. Sensory feedback is thus essential for in-hand manipulation, and would strongly improve user experience and limb functionality for amputees. This fellowship proposes to exploit our world-leading neuromorphic optical tactile sensor along with recent developments in biologically plausible neural networks to develop a neuroprosthetic hand with shared autonomy. Development and validation of the upper-limb prosthetic will be tackled in four phases, with each contributing a step progression towards the overarching project goal. Firstly, a neuromorphic tactile sensor will be developed based on the neuroTac neuromorphic optical tactile sensor and integrated with the Pisa/IIT anthropomorphic softHand. The spike-based output of the sensor will be calibrated to match that of biological mechanoreceptors, based on human microneurography recordings. Next, we will interface the sensor with four distinct feedback methods: pressure, vibration and electrical stimulation (non-invasive) and intraneural microstimulation (invasive). The methods will be compared in a series of psychophysical validation experiments to determine their applicability to upper-limb prostheses. Spiking neural networks will then be developed and optimised to run in real-time on neuromorphic hardware (SpiNNaker) for the principal tactile capabilities of slip detection, hardness and texture recognition. Finally, I will combine tactile feedback to the user with the developed on-board intelligence in shared autonomy protocols that maximise users’ body ownership of the device and their performance on activities of daily living.This project thus proposes an ambitious long-term plan of creating a semi-autonomous neuroprosthetic, with a specific focus on providing essential, high-quality tactile feedback to the user and embedding reflex-like intelligence in the prosthesis. This research could strongly improve the quality of life of upper-limb amputees, as well as having an impact on the fields of tele-operation and brain-machine interfaces.

Time frame: 28/07/2022- 31/03/2023

Funding: Royal Academy of Engineering (UK)