Indexed field case 4 - slow requests

Consider a web access log with a trailing request time in microseconds:

[31/Jan/2012:18:18:07 +0000] "GET / HTTP/1.1" 200 7918 "" 
"Mozilla/5.0..." 11/11033255 

Let's say we want to find all requests that took longer than 10 seconds. We can easily extract the value into a field, perhaps request_ms. We could then run the search request_ms>10000000. This query will work, but it requires scanning every event in the given time frame. Whether the field is extracted or indexed, we would face the same problem, as Splunk has to convert the field value to a number before it can test the value.

What if we could define a field and instead search for slow_request=1? To do this, we can take advantage of the fact that, when defining an indexed field, the value can be a static value. Having Splunk search for a static value—rather than examining the value of every event and then trying to match it—would improve the efficiency of the search. This can be accomplished with a transform, like so:

REGEX = .*/(\d{7,})$ 
FORMAT = slow_request::1 

We will cover transforms, and the configurations involved, in Chapter 11, Configuring Splunk.

Once again, this is only worth the trouble if you need to efficiently search for these events and not simply report on the value of request_ms.