Building Docker images that monitor applications

When I add these new features to the NerdDinner Dockerfile and run a container from the image, I'll be able to see the web request and response logs with the docker container logs command, which relays all the IIS log entries captured by Docker, and I can use an environment variable to specify the database user credentials. This makes running and administering the legacy ASP.NET application consistent with how I use any other containerized application running on Docker. But I can also configure Docker to monitor the container for me, so I can manage any unexpected failures.

Docker provides the ability to monitor the application health rather than just checking whether the application process is still running, with the HEALTHCHECK instruction in the Dockerfile. With HEALTHCHECK, you tell Docker how to test whether the application is still healthy. The syntax is similar to the RUN and CMD instructions you pass in a shell command to execute, which should have a return code of 0 if the application is healthy and 1 if it is not. Docker runs the health check periodically when the container is running and emits status events if the health of a container changes.

The simple definition of healthy for a web application is the ability to respond normally to HTTP requests. Which request you make depends on how thorough you want the check to be ideally, the request should execute key parts of your application, so you're confident it is all working correctly. But equally, the request should complete quickly and have a minimal compute impact, so processing lots of health checks doesn't affect consumer requests.

A simple health check for any web application just uses the Invoke-WebRequest PowerShell cmdlet to fetch the home page and check whether the HTTP response code is 200, which means the response was successfully received:

try { 
    $response = iwr http://localhost/ -UseBasicParsing
    if ($response.StatusCode -eq 200) { 
        return 0
    } else {
        return 1
    } 
catch { return 1 }

For a more complex web application, it can be useful to add a new endpoint specifically for healthchecks. You can add a diagnostic endpoint to APIs and websites that exercise some of the core logic for your app and returns a Boolean result to indicate whether the app is healthy. You can call that endpoint in the Docker health check and check the response content as well as the status code in order to give you more confidence that the app is working correctly.

The HEALTHCHECK instruction in the Dockerfile is very simple. You can configure the interval between checks and the number of checks that can fail before the container is considered unhealthy, but to use the default values, just specify the test script in HEALTHCHECK CMD . This example from the Dockerfile for the dockeronwindows/ch03-iis-healthcheck image uses PowerShell to make a GET request to the diagnostics URL and check the response status code:

HEALTHCHECK --interval=5s `
 CMD powershell -command `
    try { `
     $response = iwr http://localhost/diagnostics -UseBasicParsing; `
     if ($response.StatusCode -eq 200) { return 0} `
     else {return 1}; `
    } catch { return 1 }

I've specified an interval for the health check, so Docker will execute this command inside the container every five seconds (the default interval is 30 seconds if you don't specify one). The health check is very cheap to run, as it's local to the container, so you can have a short interval like this and catch any problems quickly.

The application in this Docker image is an ASP.NET Web API app, which has a diagnostics endpoint, and a controller you can use to toggle the health of the application. The Dockerfile contains a health check, and you can see how Docker uses it when we run a container from that image:

docker container run -d -P --name healthcheck dockeronwindows/ch03-iis-healthcheck

If you run docker container ls after starting that container, you'll see a slightly different output in the status field, similar to Up 3 seconds (health: starting). Docker runs the health check every five seconds for this container, so at this point, the check hasn't been run. Wait a little longer and then the status will be something like Up 46 seconds (healthy).

This container will stay healthy until I make a call to the controller to toggle the health. I can do that with a POST request that sets the API to return HTTP status 500 for all subsequent requests:

$ip = docker inspect -f '{{ .NetworkSettings.Networks.nat.IPAddress }}' healthcheck
iwr "http://$ip/toggle/unhealthy" -Method Post

Now the application will respond with a 500 response to all the GET requests the Docker platform makes, which will fail the health check. Docker keeps trying the health check, and if there are three failures in a row, then it considers the container to be unhealthy. At this point, the status field in the container list shows Up 3 minutes (unhealthy). Docker doesn't take automatic action on single containers that are unhealthy, so this one is left running and you can still access the API.

Health checks are important when you start running containers in a clustered Docker environment (which I cover in Chapter 7, Orchestrating Distributed Solutions with Docker Swarm), and it's a good practice to include them in all Dockerfiles. Being able to package an application that the platform can test for health is a very useful feature; this means that wherever you run the app, we can keep a check on it.

Now you have all the tools to containerize an ASP.NET application and make it a good Docker citizen, integrating with the platform so it can be monitored and administered in the same way as other containers. A full .NET Framework application running on Windows Server Core can't meet the expectation of running a single process because of the all the necessary background Windows services. But we should still build container images so they run only one logical function and separate any dependencies.