Tuesday, June 18, 2019

Iterables and Iterators in JavaScript

This post is the second in a series of posts on Symbols, Iterables, Iterators and Generators. The previous post in the series was about Symbols in JavaScript. This post on the other hand, will be focusing on explaining what Iterables and Iterators are in JavaScript.

It contains the following sections:

Introduction to Iterable and Iterator via an example

In JavaScript, an Iterable is a structure that can be looped over using the for of syntax.

Iterators, one the other hand, is a structure that is used to retrieve values from an iterable: effectively, the iterator makes iterating the iterable possible.

Let us quickly get a feel for these two concepts via an example, before attempting a more formal definition.

Let us use Array as an example.

Arrays in JavaScript are inbuilt iterables, this means that we can iterate over them using for of, for example:

for (let value of [1,2,3,4]) {
// prints 1, and then 2, and then 3, etc...

Since the array is an iterable, it means it must possess an iterator; and this we can confirm by manually retrieving the Symbol.iterator symbol and using it to loop through the values of the array.

<sidenote>If you have little or no idea what Symbol.iterator means? Now is the time to take a pause and first read Symbols In JavaScript</sidenote>

Once we have access to the Array's iterator, we can use it to manually iterate through the content of the array.

This will look like this:

let an_array = [1,2,3,4] // define an array
var it = an_array[Symbol.iterator]() // retrieve its iterator

// manually use the iterator to go through the array
while (true) {
   var next = it.next()
   if (!next.done) {
   } else {

// prints 1, and then 2, and then 3, etc...

It should be noted that in real life, a manually retrieval of an iterator to use for manual iteration won’t be something to do. This is done only for illustrative purposes.

With the above, we see that Array is an iterable: it can be looped over using for of. We also confirm that it has an iterator, which we retrieved and used to manually loop through the array.

The next question is, what makes an iterator or iterable one? What is the structure of these two things? And how can we create custom data structure that can also be used with the for of syntax?

Structure of an Iterator

We start by looking at the structure of an Iterator.

In the previous section, an Iterator was described as a structure that is used to retrieve values from an iterable. It was also stated that the iterator makes iterating the iterable possible.

A more formal definition would be to say that an Iterator is any JavaScript value that adheres to the Iterator protocol

Basically, adhering to the Iteration protocol means having a next method, that when called, returns objects with at least two properties: done and value, where done property is used to indicate if the iteration has reached its end, and the value property is the value that is being returned on each iteration. ie:

  done: false | true // Iteration done or not
  value: T // value of iteration. May not be present when done is true

That is an Iterator. Let us look at an Iterable next.

Structure of an Iterable

An Iterable is an object that has the symbol Symbol.iterator as a method, that when called returns an Iterator.

String, Array, Map and Set are some examples of in built iterators and hence, the following...

typeof “Hello world”[Symbol.iterator] // function
typeof [0,1][Symbol.iterator] // function
typeof new Map([])[Symbol.iterator] // function
typeof new Set([])[Symbol.iterator] // function

...will all return a function that when called, returns the associated iterators.

Basically for a thing to be an iterable, there must be a corresponding iterator that makes the iteration possible.

It should be noted that Objects in JavaScript are not iterable, hence, why it is not possible to use for of with an object. If you attempt to do so:

for (i of obj) {

You get a descriptive error message:

Uncaught TypeError: obj is not iterable
    at <anonymous>:1:11

By now, we know that a value needs to be an iterable in other for it to be used with the inbuilt for of syntax: But the for of is not the only inbuilt syntax that requires an iterable. Other such inbuilt syntax in JavaScript that expects an iterable include: Spread Syntax, Deconstructing Assignment (works also on objects and not just iterables), the yield* expression etc

Defining Custom Iterables

Defining a custom iterable is nothing complex. It means defining a custom data structure that has a Symbol.iterator method that returns an iterator. That is it.

Two of the ways this can be achieved include:

via a function:

In this approach, we define a function that returns an object that satisfy the structure of an iterable.

var makeRange = function(first, last) {
var _first = first;
var _last = last
return {

  [Symbol.iterator]: function() {
     return {
         next: function() {
               if (_first < _last) {
                   return {value: _first++, done: false}
               } else {
                   return {done: true}

Using it:

for (item of makeRange(0,10)) {
    console.log(item) // prints 0 to 9

via a class:

In this approach, we create a class whose instance would be a value that satisfy the structure of an iterable.

class MakeRange {
 constructor(first, last) {
  this._first = first;
  this._last = last;

 [Symbol.iterator]() {
     return {
         next: () => {
               if (this._first < this._last) {
                   return {value: this._first++, done: false}
               } else {
                   return {done: true}

Using it:

for (item of new MakeRange(0,10)) {
     console.log(item); // prints 0 to 9

And because the values created by either the makeRange function or by instantiating MakeRange class are iterables, they can also be used in other places that expects iterables.

For example with deconstructing assignment:

var [first, second] = makeRange(0,10)
console.log(first) // prints 0
console.log(second) // prints 1

Or with the spread syntax:

var an_array = [...new MakeRange(0,10)]  // expands the range into an array

Just as we can define custom iterable, we can also define a custom iterator, but that is not as interesting as having a custom iterable since there is no inbuilt language feature that an iterator can hook into like an iterable does. If you have an iterator, what you can do is to manually call the next method...and that is boring :)

Clarification on @@iterator

@@Iterator was one thing I came across while digging into iterables and iterators that was a bit confusing. The section about Iterables (mentioning @@iterator)on MDN web docs has the following:
In order to be iterable, an object must implement the @@iterator method, meaning that the object (or one of the objects up its prototype chain) must have a property with a Symbol.iterator key.  

Reading this, naturally leads to asking questions like: what exactly is the @@iterator method? Why does it start with a double @? How is it supposed to be implemented? And how does it relates with Symbol.iterator?

I was not the only one to find the mention of @@iterator confusing as I found the following questions on StackOverflow,  ES 6: Difference between Symbol.iterator and @@iterator and What does @@ (“at at”) mean in ES6 JavaScript? asking more or less the same questions I had.

So what is the deal with @@Iterator?

For all intents and purposes, we can totally ignore it. It is a low level terminology used in the specification to refer to inbuilt symbols (aka well known symbols). It is not a valid syntax, hence it cannot be used anywhere in a JavaScript code and as far as I am concerned, the term should not be mentioned in any user space documentation, or if mentioned, it should be followed up immediately with a clarification.

Next up

So far, so good, in this series, we have looked into Symbols in JavaScript, and now, with this post, we explored Iterables and Iterators. Next up would be how TypeScript comes into the picture and what facilities TypeScript bring to the table in ensuring we can correctly model these concepts safely. To do that we start by looking at how to configure the TypeScript compiler for working with Iterables and Iterators

I am writing a book: TypeScript Beyond The Basics. Sign up here to be notified when it is ready.

No comments: