Validation data

You may be wondering why we train our model blindly for an arbitrary number of iterations and then test it on our holdout data. Wouldn't it be more efficient to gauge our model to some unseen data after each epoch, just to see how well we are doing? This way, we are able to assess exactly when our model starts to overfit, and hence end the training session and save some expensive hours of computing. We could show our model the test set after each epoch, without updating its weights, purely to see how well it does on our test data after that epoch. Since we do not update our model weights at each test run, we don't risk our model overfitting on the test data. This allows us to get a genuine understanding of how generalizable our model is during the training process, and not after. To test your model on a validation split, you can simply pass the validation features and labels as a parameter, as you did with your training data, to the fit parameter.

In our case, we have simply used the test features and labels as our validation data. In a rigorous deep learning scenario with high stakes, you may well choose to have separate test and validation sets, where one is used for validation during training, and the other is reserved for later assessments before you deploy your model to production. This is shown in the following code:

network_metadata=model.fit(x_train, y_train,
                           validation_data=(x_test, y_test),
                           epochs=20,
                           batch_size=100)

Now, when you execute the preceding cell, you will see the training session initiate. Moreover, at the end of each training epoch, you will see that our model takes a brief pause to compute the accuracy and loss on the validation set, which is then displayed. Then, without updating its weights after this validation pass, the model proceeds to the next epoch for another training round. The preceding model will run for 20 epochs, where each epoch will iterate over our 25,000 training examples in batches of 100, updating the model weights after each batch. Note that in our case, the model weights are updated 250 times per epoch, or 5,000 times during the preceding training session of 20 epochs. So, now we can better assess when our model starts to memorize random features of our training set, but how do we actually interrupt the training session at this point? Well, you may have noticed that instead of just executing model.fit(), we defined it as network_metadata. As it happens, the fit() parameter actually returns a history object containing the relevant training statistics of our model, which we are interested in recovering. This history object is recorded by something called a callback in Keras.