Transducers

Before we finish up with our core.async portion of this book, we should mention what came up in Clojure 1.7, as well as how this affects core.async.

One of the big changes in this release was the introduction of transducers. We will not cover the nuts and bolts of it here, but rather focus on what it means at a high-level with examples using both Clojure sequences and core.async channels.

If you would like to know more, I recommend Carin Meier's Green Eggs and Transducers blog post[4]. It's a great place to start.

Additionally, the official Clojure documentation site on the subject is another useful resource[5].

Let's get started by creating a new Leiningen project:

$ lein new core-async-transducers  

Now, open your project.clj file and make sure you have the right dependencies:

... 
  :dependencies [[org.clojure/clojure "1.9.0"] 
                 [org.clojure/core.async "0.4.474"]] 
... 

Next, fire up a REPL session in the project root and require core.async, which we will be using shortly:

    $ lein repl
    user> (require '[clojure.core.async :refer [go chan map< filter< into >! <! go-loop close! pipe] :as async])  

We will start with a familiar example:

    (->> (range 10)
         (map inc)           ;; creates a new sequence
         (filter even?)      ;; creates a new sequence
         (prn "result is "))
    ;; "result is " (2 4 6 8 10)  

The preceding snippet is straightforward and highlights an interesting property of what happens when we apply combinators to Clojure sequences: each combinator creates an intermediate sequence.

In the previous example, we ended up with three in total: the one created by range, the one created by map, and finally, the one created by filter. Most of the time, this won't really be an issue, but for large sequences, this means a lot of unnecessary allocation.

Starting in Clojure 1.7, the previous example can be written like so:

    (def xform
      (comp (map inc)
            (filter even?)))  ;; no intermediate sequence created
    
    (->> (range 10)
         (sequence xform)
         (prn "result is "))
    ;; "result is " (2 4 6 8 10)  

The Clojure documentation describes transducers as composable algorithmic transformations. Let's see why that is.

In the new version, a whole range of the core sequence combinators, such as map and filter, have gained extra arity: if you don't pass it a collection, it instead returns a transducer.

In the previous example, (map inc) returns a transducer that knows how to apply the inc function to elements of a sequence. Similarly, (filter even?) returns a transducer that will eventually filter elements of a sequence. Neither of them do anything yet—they simply return functions.

This is interesting because transducers are composable. We can build larger and more complex transducers by using simple function composition:

    (def xform
      (comp (map inc)
            (filter even?)))  

Once we have our transducer ready, we can apply it to a collection in a few different ways. For this example, we chose sequence, as it will return a lazy sequence of the applications of the given transducer to the input sequence:

    (->> (range 10)
         (sequence xform)
         (prn "result is "))
    ;; "result is " (2 4 6 8 10)  

As we highlighted previously, this code does not create intermediate sequences; transducers extract the very core of the algorithmic transformation at hand and abstract it away from having to deal with sequences directly.