Mastering Asynchronous JavaScript: Part 3 – Generators

Last time I showed you how promises can make asynchronous programming in JavaScript easier than using callbacks. Now I’ll show you some cutting edge JavaScript features that will make asynchronous programming a breeze.

Introduction To Generators

One of the more recent updates to the JavaScript language, ES6, introduced the concept of generators. The best way to describe generators is that they are functions which return iterable objects. These objects allow the function to return multiple values. There’s some new syntax and a new keyword to learn here, so let’s just jump straight in with a simple example.

function * myGenerator () {
    yield 1;
    yield 2;
    yield 3;
    return 4;
}

// If myGenerator was a normal function, myIterator would have the value of 4
// Since it's a generator, myIterator actually contains an iterator object
var myIterator = myGenerator();

// Following logs:
// 1
// 2
// 3
// 4
// to console
var done = false;
while (!done) {
    var current = myIterator.next();
    done = current.done;
    console.log(current);
}

We create a generator function the produces multiple numbers. It is marked as an generator by the ‘*’ before the function name. We call the generator function, which returns an iterator. We iterate it to log out all the values.

The most important thing to note here is that calling an iterator function does not run the function, but instead returns an iterator. Notice we have introduced a new keyword, ‘yield’. The yield keyword instructs the generator to return the value given after the yield, then pause the function’s execution until the next iteration.

Calling next() on the iterator performs the next iteration, and returns and an object containing the yielded value, and a boolean (done) which is true if all the yield statements (and the return statement) have been executed, otherwise false.

Now I’m going to make things a little more confusing. A generator, as well as being able to return multiple values, can also take multiple inputs, one at each iteration. Look at this example:

function * myGenerator (normalArg) {
    var a = yield 1;
    return a + normalArg;
}

var myIterator = myGenerator(10);

// Following logs:
// 1
// 15
// to console
var done = false;
console.log(myIterator.next().value);
console.log(myIterator.next(5).value);
// Iterator is now done

We call the generator function, passing in an argument (normalArg) in the normal way with a value of 10. On the first iteration, 1 is yielded and so this is the value within the object returned by next(). The function is then paused until next() is called again. This time we pass it a value, 5, and this resumes the function’s execution. The yield statement evaluates to the passed in value (5), which gives variable ‘a’ the value 5. We then return normalArg + a which evaluates to 10 + 5, so next(5) gives us a value of 15.

The point that I found tricky when first learning generators is how yield evaluates to the value passed to the next call to next, not the current call to next that returns the current yielded value. This is why in our code, 5 is passed to next the second time we call it, not the first. This makes sense when we realise that the function pauses at the point the value is yielded. It is only on the subsequent call to next that the value of yield is evaluated and passed to the variable a.

I’m pretty sure at this point you’re thinking, “Well that’s nice, functions that can take and return multiple values… What has this got to do with asynchronous coding though?”. Good question. Now I’ll show you how we can use generators to make asynchronous coding a breeze.

Generators For Asynchronous Programming

To use generators for asynchronous tasks, we need to make use of a module called co, which is available on npm: https://www.npmjs.com/package/co.

Let’s return to our database example from previous posts so we can see generators making easy work of some asynchronous operations.

var co = require('co');

// Make db functions return promises
var promisify = require("promisify-node");
var MongoClient = promisify("mongodb").MongoClient;

var url = 'mongodb://localhost:27017/myproject';
function output (user, orders) {
    // output data in pretty format
}

function logError (error) {
    // Log the error (could be console, a service, a file etc)
}

co(function * () {
    const userId = process.argv[2];
    try {
        const db = yield MongoClient.connect(url);
        yield db.collection('access_log').insert({ userId, date: new Date() });
        const users = yield db.collection('users').find({ userId }).toArray();
        const orders = yield db.collection('orders').find({ userId }).toArray();
        output(user, orders);
    } catch (e) {
        logError(e);
    }
});

This is clearly the best version of this code we’ve seen. It’s the most concise and readable. Following and modifying it would be easy. In fact, it’s almost as if the code was sequential, rather than asynchronous. Also notice that, unlike with callbacks or promises, we can have proper error handling by means of a try / catch block (this isn’t actually required in this example since co returns a promise so we could have done .catch(logError) at the end of the co call, but wanted to show that generators work fine with try / catch blocks). What isn’t obvious from this code, is how it works. Let me explain.

Co gets passed a generator function. Co will call this function, which returns an iterator object. It will then call next() on the iterator. The function will then run until it hits the yield when we set up the database connection. At that point the function’s execution pauses (as discussed above, yield pauses it), and the value returned by that call to next will be the promise returned by the database connection call. Co is clever and knows that the value yielded is a promise. When co encounters a yielded promise, it pretty much does something like the following:

promise
    .then(function (value) {
        iterator.next(value);
    })
    .catch(function (err) {
        iterator.throw(err);
    }

Please note this is just for illustrative purposes, what co actually does will be quite different, as for example, here we do not see how it performs the next call to next(). Anyway, in the success handler for the promise it calls the next iteration of the iterator, passing in the success value from the promise. In this case, that will be the database connection object. The generator function will then resume execution. Since the yield evaluates to the value passed into next, this is what will be assigned to the ‘db’ const. The function will then continue to run until it is paused again on the next line when it again yields a promise, this time for the database insert operation. Things will continue in this manner until execution reaches the end of the function.

If all this sounds a bit complicated, just think of it like this. The generator function gets run. Each time we ‘yield’ an asynchronous function call the function pauses and waits for the result. This result is then assigned to a variable (if we wish). The function will then continue to run until it hits the next yield. It waits for that result then continues running… and on and on till the function returns.

It is also worth noting that although our asynchronous operations here are all functions that return promises, any of these promise returning functions could be swapped for functions which are themselves generators. As with promises, Co detects that the yielded value is an iterator, and just like it resolves the promise before calling next again with the result, it will exhaust all the calls to the yielded iterator’s next function, and then in it’s next call pass the final value of the iterator back to the parent iterator so the yield expression evaluates to this value. Again this may sound very complicated, but all you really need to know is that if you yield a call to a generator, the result will be whatever the generator finally returns.

Next Time

So now we have a very simple way of handling asynchronous code in JavaScript. The only thing about generators is they aren’t exactly designed for asynchronous code. It seems like this is almost a side effect of the iterators returned, and we need something like co for them to actually be useable in terms of asynchronous flow control. Next time I will talk about async / await, a feature designed to specifically to handle asynchronous programming.

The full series:

  1. Callbacks
  2. Promises
  3. Generators
  4. Async / await
  5. RxJS

3 comments

Leave a Reply

Your email address will not be published. Required fields are marked *