Calculating the number of parameters (weights)

Here, we will show how to calculate the number of parameters used by a convolution layer. The formula to calculate the number of parameters in a convolution layer (including biases) is as follows:

We will illustrate with a simple example:

 Input: [32x32x3] input tensor
 Conv layer: Kernel:5x5
               numFilters:10

On the other hand, the number of parameters in a fully connected layer (including biases) will be as follows:

As mentioned before, if you use a traditional artificial neural network directly on the image, all the spatial information will be lost and you will have an explosion of parameters as you will have one parameter per pixel per neuron. Using the same example mentioned previously and with a dense layer of 10 output neurons, we get the following numbers:

This demonstrates the order of magnitude difference in parameters between these two layer types.