TensorBoard

TensorBoard is a web-based utility provided with TensorFlow that allows you to visualize your constructed TensorFlow graph. On top of this, it allows you to keep track of a wealth of statistics or variables that may be important for training your model. The examples of such variables that you might like to keep track of include training loss, test set accuracy, or learning rate. We saw earlier that we can visualize the value of our loss function using tensorboard.

To run TensorBoard, open a new terminal and type the following:

$ tensorboard --logdir=/somepath

Here, somepath points to the place where your training code saves tensorboard logging data.

Inside your code, you need to define which tensors to visualize by creating a tf.summary for each one. So for example if we wanted to to inspect all trainable variables and the loss we would need to use the following code:

with tf.Session() as sess:

    """Create your model"""

    # Add all trainable variables to tensorboard
    for var in tf.trainable_variables(): 
        tf.summary.histogram(var.name, var) 
    # Add loss to tensorboard 
    tf.summary.scalar("softmax_cross_entropy", loss) 
    # Merge all summaries 
    merged_summary = tf.summary.merge_all() 
    # Initialize a summary writer  
    train_writer = tf.summary.FileWriter( /tmp/summarys/ , sess.graph) 
 
    train_writer.add_summary(merged_summary, global_step) 

    """Training loop"""

We need to create a tf.summar.FileWriter which is responsible for creating a directory where logs of our summaries will be stored. If we pass in a graph when we create the FileWriter then this will also be displayed in TensorBoard for us. By passing in sess.graph we are supplying the default graph that the session is using. The result of displaying a graph in TensorBoard might look something like this: