Diving deep into neural networks

Deep learning is a sub-field of ML that is based on ANNs (see Fig 2.1). Deep learning mimics the human brain and is inspired by the structure and function of the brain. The concept of deep learning is not new and has existed for a number of years. The reason for the popularity and success of deep learning in recent years is due to high powered processing units, such as GPUs, and the presence of enormous amounts of data. One of the reasons for deep neural networks (DNNs) performing better is the complex relationships among features and high-dimensional data:

One of the great things about deep learning is that it eliminates human input. It replaces the costly and inefficient effort of human beings and automates most of the extraction process from features and raw data so that it doesn't require human involvement. Before, we used to extract features ourselves to make ML algorithms work. The automated extraction of features has enabled us to work with larger datasets, eliminating the need for human intervention completely, except at the supervised labeling step at the very end. Thus, when there is huge data, the performance of deep learning algorithms is far better than most ML algorithms, as shown in the following diagram:

In the next section, we will look at the biological neuron—the foundation of neural networks.

Introduction to neurons

In this section, we will discuss neurons, which are the basic building blocks of ANNs. In the following photograph, we can see actual real-life neurons as observed through a microscope:

You can find this photograph at https://commons.wikimedia.org/wiki/Neuron#/media/File:Pyramidal_hippocampal_neuron_40x.jpg.

The question now is how can we recreate neurons in ML? We need to create them since the whole purpose of deep learning is to mimic the human brain, one of the most powerful tools on the planet. So, the first step toward creating an ANN is to recreate a neuron.

Before creating a neuron in ML, we will examine the depiction of neurons created by Spanish neuroscientist Santiago Ramon y Cajal in 1899. 

Nowadays, we have advanced technology that allows us to get a closer look at neurons. We can see a neuron in the following diagram that looks very similar to what Santiago Ramon y Cajal drew.

We can see that it has a body called a neuron, some branches called dendrites, and a long tail called an axon:

One thing to note is that Fig 2.4 shows only one neuron. One neuron alone is not that powerful, but when you have lots of neurons together, they can do wonders. You may be wondering, how do they work together? The answer is with the help of dendrites and axons. Dendrites act as a receiver for other neurons' signals and the axon is the transmitter of the neuron's signal.

Let's move on to see how we are going to create neurons in machines; we are moving away from neuroscience now. In the next section, we will read about neurons and perceptrons in detail and create artificial neurons.