Logistic regression on the MNIST dataset

This recipe is based on the logistic regressor for MNIST provided at https://www.tensorflow.org/get_started/mnist/beginners, but we will add some TensorBoard summaries to understand it better. Most of you must already be familiar with the MNIST dataset--it is like the ABC of machine learning. It contains images of handwritten digits and a label for each image, saying which digit it is.

For logistic regression, we use one-hot encoding for the output Y. Thus, we have 10 bits representing the output; each bit can have a value either 0 or 1, and being one-hot means that for each image in label Y, only one bit out of the 10 will have value 1, the rest will be zeros. Here, you can see the image of the handwritten numeral 8, along with its hot encoded value [0 0 0 0 0 0 0 0 1 0]: