Converting convolution layers into fully connected layers

Actually, we can consider fully connected layers as a subset of convolution layers. It’s possible to convert a CNN layer into a fully connected layer if we set the kernel size to match the input size. Setting the number of filters is then the same as setting the number of output neurons in a fully connected layer. Check for yourself that in this case, the operations will be the same.

Example:

Consider an FC layer with 4,096 output neurons and input of size 7x7x512, the conversion would be:

Conv layer: Kernel:7x7, Pad:0, Stride:1, Filters:4,096.

Using the formula to calculate output size, we get an output of size 1 x 1 x 4096.

One of the main reason for doing this is so that your network becomes fully convolutional. When your network is fully convolutional, it doesn't matter if you decide to use a bigger input size image than what you trained on as you don't have any fully connected layers that require a fixed input size.