Friday, June 28, 2019

Generators and Iterators in JavaScript

This post concludes the series Symbols, Iterables, Iterators and Generators in JavaScript. It would be a whirlwind tour of generator functions in JavaScript and how they can be put to use when defining Iterators.

It is a direct follow up of Typing Iterables and Iterators with TypeScript and it would take the Iterator defined in the post, and improve it by making use of generator functions.

Basically, it will show how we can go from a verbose implementation that does not make use of generator functions:

class MakeRange {
    private _first: number;
    private _last: number
    constructor(first, last) {
     this._first = first;
     this._last = last;
   }
 
    [Symbol.iterator]() {
        return {
            next: () => {
                  if (this._first < this._last) {
                      return {value: this._first++, done: false}
                  } else {
                      return {value: undefined, done: true}
                 }
           }
       }
   }
 }

Into a more terse and clearer implementation that makes use of generator functions:

class MakeRange {
    private _first: number
    private _last: number

    constructor(first, last) {
     this._first = first;
     this._last = last;
    }

    *[Symbol.iterator]() {
        while (this._first < this._last) {
            yield this._first++
       }
   }
}

This post contains the following sections:

Meaning, we are going to first look at what generator functions are, then we see the various syntaxes for creating them, and finally we bring them to use in simplifying how we define iterables and iterators.

Let's get started.



What are generator functions

Normal functions, when called run from beginning of the function, to the end, and then they exit. Generator functions also run from top to the bottom, but they have points where they give back some computed result, then pause in their execution, until when triggered again, in which case they continue to run from the last pause point, till the next pause point is reached, where they yield another computed result. This goes on, till the function reaches the end and exist, just like normal functions.

An example will illustrate the point.

// normal function
function countToThree() {
   let result = 0;

   result = result + 1
   result = result + 1
   result = result + 1

   return result  
}

When this function is called, 3 is returned, but:

// generator function
function* countToThree2() {
   let result = 0;
  
   result = result + 1
   yield result

   result = result + 1
   yield result

   result = result + 1
   yield result   
}

When we call it, we do not get a 3 as the return value, but we get a value that has a next method, which when we call, allows us to move one step forward pass the paused point in the function definition.

The paused points are indicated with the use of the yield keyword. It's syntax is yield value. When the line containing yield is reached, the function returns the value after the yield keyword and then pauses its execution.

We see this below:

let generator = countToThree2()
console.log(generator.next().value) // prints 1
console.log(generator.next().value) // prints 2
console.log(generator.next().value) // prints 3
console.log(generator.next().value) // prints undefined
console.log(generator.next().done) // prints true

The call to next() returns an object that has a value property (and also done property). With the value property, we can access the computed value at each paused point. The done property indicates if the function has actually ran to its end.

If you have read Iterables and Iterators in JavaScript and you see similarities with the next method, value and done property? It is not a coincidence. It is the presence of these that makes it possible to use generator function when working with Iterables and iterator. But we will get to this later on.

If you are familiar with breakpoints in debugging tools, you can think of generator functions as functions with breakpoints embedded within them, which you can then step over, one at a time by calling the next method.

Now that we know what generator functions are, let us take a look at how to define them.


Creating generator functions

The asterix (*) is the defining aspect to creating generator functions. The absence or presence of it, is what determines whether we have a normal function or a generator function. Its exact presence also varies depending on how the generator function is to be created: we take a look at the various ways below.

Stand alone generator function

Generator functions get created via the same, various ways normal function gets created. The only difference is that * needs to be appended at the end of the function key word. Hence the following ways are valid when creating stand alone generator functions:

Generator function via function expressions

let genFn = function* () {
   yield 1;
   yield 2;
}

Generator function via named function expressions

let genFn = function* genFn () {
   yield 1;
   yield 2;
}

Generator function via function statement (function declaration)

function* genFn() {
  yield 1;
  yield 2;
}

Any of the above syntaxes would lead to the same result of having a generator function, named genFn created; which can be used:

let generator = genFn()
console.log(generator.next()) // prints {value:1, done:false}
console.log(generator.next().value) // prints {value:2, done:false}
console.log(generator.next().undefined) // prints {value:undefined, done:true}

Note: it is not possible to create generator functions using the arrow syntax

Generator function as a method on object literal

There are two ways of defining methods within an object literals in JavaScript. The first way, pre ES6, requires a full syntax of specifying the property name and the function. The second way, post ES6, introduces a shorthand that does not require having to specify property name explicitly.

Both ways can then be used to define an object that has generator function as a property.

Using the explicit, pre ES6 will look like this:

let objGenerator = {
    max : 3,
    range: function* () {
          let returnValue = 0;
          while (returnValue < this.max) {
                yield returnValue;
                returnValue++
          }
      }
}

// usage
for (let num of objGenerator.range()) {
     console.log(num) // will print 1 to 3
}

Using the shorthand, post ES6, the definition would look like this:

let objGenerator = {
      max : 3,
      * range() {
          let returnValue = 0;
          while (returnValue < this.max) {
                yield returnValue;
                returnValue++
         }
      }
}

// usage
for (let num of objGenerator.range()) {
      console.log(num) // will print 1 to 3
}

The main thing to take note when using the shorthand syntax, is that there is no function keyword, and the asterix comes before the name of the property.

As a method on a class

The syntax for defining a generator function as a method on a class is similar to the syntax when defining generator function using the shorthand syntax for defining object literals that was introduced in ES6: Basically the generator function is defined with the asterix placed in front of the method name.

With ES6 class definition, this looks like:

class MakeRange {
    constructor(first, last) {
        this.first = first;
        this.last = last;
    }
    *range() {
        while (this.first < this.last) {
            yield this.first;
            this.first++
        }
  
}

// and on usage
let genObj = new MakeRange(0, 3).range()

console.log(genObj.next()) //  prints {value: 0, done: false }
console.log(genObj.next()) //  prints {value: 1, done: false }
console.log(genObj.next()) //  prints {value: 2, done: false }
console.log(genObj.next()) //  prints {value: undefined, done: true }

With TypeScript syntax for defining classes, it looks like this:

class MakeRange {
    private first:number
    private last:number
    constructor(first, last) {
        this.first = first;
        this.last = last;
   }
    *range() {
        while (this.first < this.last) {
            yield this.first;
            this.first++
       }
   }
}

let genObj = new MakeRange(0, 3).range()
console.log(genObj.next()) //  prints {value: 0, done: false }
console.log(genObj.next()) //  prints {value: 1, done: false }
console.log(genObj.next()) //  prints {value: 2, done: false }
console.log(genObj.next()) //  prints {value: undefined, done: true }

The definition is almost the same as with the ES6, and as can be seen, the generator function is also defined with the asterix before the method name.

Now that we have seen how to define generator functions, now let us see how to use them with iterables and iterators.


Defining Iterator with generator functions

Iterating through a data structure is basically picking a value from the structure, one at a time. If you think about it, this maps conceptually with how generator function works: which also computes values one at a time. Hence it is not totally out of place that generator functions can be used to implement iterators.

In Iteration Protocol And Configuring The TypeScript Compiler we defined a MakeRange class like the following:

class MakeRange {
    private _first: number;
    private _last: number
    constructor(first, last) {
     this._first = first;
     this._last = last;
   }
 
    [Symbol.iterator]() {
        return {
            next: () => {
                  if (this._first < this._last) {
                      return {value: this._first++, done: false}
                  } else {
                      return {value: undefined, done: true}
                 }
           }
       }
   }
 }

And as shown already in the beginning of this post, We can update this definition and make use of generator function. It will then be:

class MakeRange {
    private _first: number
    private _last: number

    constructor(first, last) {
     this._first = first;
     this._last = last;
    }

    *[Symbol.iterator]() {
        while (this._first < this._last) {
            yield this._first++
       }
   }
}

As can be seen, using generator function greatly simplified things. We went from an implementation that has a lot of elaborate steps, to a succinct one that basically described the core of what we want to achieve.

I will end this post by pointing out the fact that, not only can generator function be used to create iterators as seen above, they can be also be used to create iterables. Infact, the value returned from calling a generator functions, which technically is the GeneratorFunction object, can implement both the iterable and iterator interface (that is, the IterableIterator interface) in TypeScript.

We can prove this with the code below:

function* genFn() {
    yield 1;
    yield 2;
  }

let asIterator: Iterator<number> = genFn()
let asIterable: Iterable<number> = genFn()
let asIterableIterator: IterableIterator<number> = genFn()

If this code is put into a TypeScript file, and compiled, it will compile successfully, showing that values returned from generator functions are both Iterator and Iterables since they can be assigned to variables typed as an Iterator, Iterable and an IterableIterator

Also, it is worth pointing out that generators in JavaScript are an independent language feature that finds application when defining iterables or Iterators; but it is not the only thing you could use generator functions for in JavaScript. They could also be used to implement finite and infinite streams/lazy lists, used in asynchronous programming etc. For more information about generators, you can start from the MDN web docs section on function*

I am writing a book: TypeScript Beyond The Basics. Sign up here to be notified when it is ready.

No comments: