Understanding neurons and perceptrons

As discussed in the previous section, Introduction to neurons, before, ANNs had a basis in biology, and we mimic biological neurons with artificial neurons that are known as perceptrons. The perceptron is a mathematical model of a biological neuron. Later in this section, we will see how we can mimic biological neurons with artificial neurons.

As we know, the biological neuron is a brain cell. The body of the neuron has dendrites. When an electrical signal is passed from the dendrites to the body cell of the neuron, a single output or a single electrical signal comes out through an axon, and then it connects to some other neuron, as shown in the diagram of the generic neurotransmitter system that you can find in the link provided in the Introduction to neurons section. That is the basic idea we have: lots of inputs of electrical signals go through the dendrites, into the body, and then through the axon as a single output signal.

We can see this in the following example, which shows the conversion to an artificial neuron from a biological neuron:

We can see that the artificial neuron also has input and outputs, so the attempt to mimic a biological neuron was done successfully. This simple model is known as a perceptron.

Let's see a simple example of how we are going to convert a biological neuron into an artificial neuron. The step-by-step procedure is as follows. We can see, in the following diagram, that dendrites are converted into input signals, the axon is converted into an output signal, and the body is converted into a neuron:

In this case, we have two inputs and a single output. We will start indexing at 0 and will have the input of 0 and 1. The input will have the value of features. So, when you have your dataset, you are going to have various features, and these features can be anything from how many rooms a house has or how dark an image is, which is represented by some sort of pixel amount or some kind of darkness. In the following example of conversion into an artificial neuron, we will assign input values of 12 and 4:

Our next step is to multiply these input signals (input 0 and input 1) with weight. The actual neuron only fires an output signal when the total input signal intensity reaches a defined threshold. Here, we have weight 0 for input 0 and weight 1 for input 1, as shown in the following diagram. Typically, weights are initialized through some sort of random generation as it chooses a random weight:

In this case, random numbers have been chosen for the value of weights—for example, 0.5 for weight 0 and -1 for weight 1—and the chosen numbers are arbitrary:

So now, these inputs are going to be multiplied by the weights, which means that input 0 is 6 and input 1 is -4. The next step is to multiply the inputs by their weight and pass them to the activation function, as in the following diagram:

There are many activation functions we can choose from, which we will cover in a later section of this chapter, Understanding activation functions. For now, we will choose a simple activation function. So, in the case of this activation function, if the sum of the inputs is positive, then it returns an output of 1, and if the sum of the input is negative, then it returns an output of 0. In our case, the sum of the input is positive, so it will return 1.

There can be a case where the original input is 0, so for every weight, it will be 0. This can be fixed by adding a bias term, which is shown in the following diagram. The bias is the intercept, which is a linear equation of a straight line. The bias is the parameter that helps the neural network adjust the output and the weighted sum of the neurons. In simple terms, we can say that the bias helps the model best fit the given data. In this case, we will choose a bias of 1, as shown in the following diagram:

So, we have designed an artificial neuron from a biological neuron. Now, we are ready to learn about ANNs. In the next section, we will see how a deep learning network is designed using multiple neurons.