In recent times, we heard how a neurotech startup, Neuralink, plans to improve the human brain’s computation by implanting a minuscule interface onto the brain. The electrodes in the brain-machine interfaces convert neuronal information to commands competent in controlling external systems. The biggest question that arises is how will the signals in your brain be processed.
Best Machine Learning Courses & AI Courses Online
To understand this, we need to know how neurons are structured in the brain and how they transmit information. Everyone who has been following recent machine learning trends is aware of the 2nd generation Artificial Neural Networks. Artificial Neural Networks are usually fully connected, and they deal with continuous values. ANNs have made tremendous progress in many fields.
However, they do not imitate the mechanism of the brain’s neurons. The next generation of Neural Network, the spiking neural network, aims to ease the application of machine learning in neuroscience.
In-demand Machine Learning Skills
Get Machine Learning Certification from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.
Know How Neurons Transmit Information in the Brain
How is information sent and received by a neuron? Neurons need to transmit information for communicating among themselves. Transmission of the information is done both within the neuron or from one neuron to another. In the human brain, the dendrites usually get information from the sensory receptors. The information received is passed to the axon through the cell body.
Popular AI and ML Blogs & Free Courses
As soon as the information reaches the axon, it moves down the axon’s entire length as an electric signal known as the action potential. On reaching the end of the axon, information needs to be transmitted to the next neuron’s dendrites, if required. There is a synaptic gap present between the axon and the dendrites of the next neuron. This gap can be filled on its own or by the help of neurotransmitters.
Spiking Neural Network
A spiking neural network(SNN) is different from traditional neural networks known in the machine learning community. Spiking neural network operates on spikes. Spikes are discrete events taking place at specific points of time. Thus, it is different from Artificial Neural Networks that use continuous values. Differential equations represent various biological processes in the event of a spike.
One of the most critical processes is the membrane capacity of the neuron. A neuron spikes when it reaches a specific potential. After a neuron spike, the potential is again re-established for that neuron. It takes some time for a neuron to return to its stable state after firing an action potential. The time interval after reaching membrane potential is known as the refractory period.
In the refractory period, triggering another action potential is quite difficult even if the excitatory inputs are strong. The sodium ion channels ensure that the action potential remains inactivated and does not reach membrane potential. Thus, a neuron does not continue a firing spree even on getting constant excitatory inputs.Â
The Leaky Integrate-and-Fire(LIF) model is the most common. Spiking Neural Networks are not densely connected.Â
Differential Equation for membrane capacity in the LIF model
In the spiking neural network, neurons are not discharged at every propagation cycle. The firing of neurons is only when the membrane potential reaches a certain value. As soon as a neuron is discharged, it produces a signal. This signal reaches other neurons and changes their membrane potential. Spike train provides us with increased potential to process Spatio-temporal data.
The spatial characteristic points to neurons being only connected to other neurons that are local to them. Thus, the processing of inputs works similarly to a Convolutional Neural Network that uses a filter. The temporal characteristic mentions that spikes occur at a particular time. The information lost in binary encoding is retrieved in the form of temporal information from the spikes.
This permits us to process temporal data naturally, without making cumbersome as in Recurrent Neural Networks. We have proofs showing how spiking neural networks have greater computation power than traditional artificial neural networks.
One question that may arise is why Spiking Neural Networks are not as widely used as traditional neural networks despite being computationally more powerful. The main reason behind not using SNNs frequently is a lack of training algorithms. There are unsupervised biological learning algorithms like Hebbian learning and STDP, but there is a lack of supervised training methods for SNNs.
As spike trains cannot be differentiated, we cannot train Spiking Neural Networks using conventional methods such as gradient descent without losing specific temporal information. Thus, we need to research and develop an efficient supervised learning algorithm for Spiking Neural Network to use it in real-life scenarios. It is a difficult job as we need to know thoroughly how the brain gains an understanding and transmits information between neurons.Â
Spiking Patterns
A spike train denotes a two-dimensional plot of membrane potential and time having multiple spikes. The neuron discharged at a certain time interval can hold much more information.Â
Various spiking patterns
The parameters a,b,c, and d shown above belong to Izhikevich model neurons.Â
Wrapping Up
The future of Spiking Neural Network is quite ambiguous. SNNs are referred to as the successors of the current neural networks, but there is a long way to go. Implementation of Spiking Neural Networks is still difficult in most practical tasks. SNNs have real-time applications in the field of image and audio processing.
However, the number of applications in these fields remains sparse. The research papers on Spiking Neural Networks are mostly theoretical. In some cases, performance analysis of SNNs is shown under a fully connected neural network. There is a huge scope of research in this domain as a major part is still unexplored.
If you are interested to learn about Machine learning in cloud, upGrad in collaboration with IIIT-Bangalore, offers the Master of Science in Machine Learning & AI. The course will equip you with the necessary skills for this role: math, data wrangling, statistics, programming, cloud-related skills, as well as ready you for getting the job of your dreams.
