The document discusses JavaScript iteration protocols including generators, iterators, iterables, async iterators, and async iterables. It provides examples of using generator functions to create custom iterables. It explains that generator objects are both iterators and iterables. The document also explains the iterator and iterable protocols, defining what makes an object an iterator and what makes an object iterable. It shows how to implement these protocols using generator functions, anonymous objects, and classes. Finally, it covers async iteration protocols, showing how to create async iterators and async iterables, typically using async generator functions. It provides examples of built-in Node.js async iterables like readable streams.

1. 1

2. META_SLIDE!
loige
loige.link/lets-iter
2

3. const array = ['foo', 'bar', 'baz']
for (const item of array) {
console.log(item)
}
loige
Output:
foo
bar
baz
3

4. const str = 'foo'
for (const item of str) {
console.log(item)
}
loige
Output:
f
o
o
4

5. const set = new Set(['foo', 'bar', 'baz'])
for (const item of set) {
console.log(item)
}
loige
Output:
foo
bar
baz
5

6. const map = new Map([
['foo', 'bar'], ['baz', 'qux']
])
for (const item of map) {
console.log(item)
}
loige
Output:
[ 'foo', 'bar' ]
[ 'baz', 'qux' ]
6

7. const obj = {
foo: 'bar',
baz: 'qux'
}
for (const item of obj) {
console.log(item)
}
loige
Output:
⛔ Uncaught TypeError: obj is not iterable
OMG `for ... of`
does not work with plain objects! 😱
7

8. const obj = {
foo: 'bar',
baz: 'qux'
}
for (const item of Object.entries(obj)) {
console.log(item)
}
loige
Output:
[ 'foo', 'bar' ]
[ 'baz', 'qux' ]
8

9. loige 9

10. const array = ['foo', 'bar', 'baz']
console.log(...array)
loige
Output:
foo bar baz
spread syntax!
10

11. loige 11

12. loige 12

13. loige
Iterators
Iterables
Async Iterators
Async Iterables
Generator functions
Async Generators
📸THE BIGGER PICTURE
13

14. loige
Knowing iteration protocols allows us:
Understand JavaScript better
Write more modern, interoperable and idiomatic code
Be able to write our own custom iterables (even async)
😥WHY SHOULD WE CARE?
14

15. WHO IS THIS GUY !?
👋I'm Luciano ( 🍕🍝 )
Senior Architect @ fourTheorem (Dublin )
nodejsdp.link
📔Co-Author of Node.js Design Patterns 👉
Let's connect!
(blog)
(twitter)
(twitch)
(github)
loige.co
@loige
loige
lmammino 15

16. ALWAYS RE-IMAGINING
WE ARE A PIONEERING TECHNOLOGY CONSULTANCY FOCUSED ON AWS AND SERVERLESS
| |
Accelerated Serverless AI as a Service Platform Modernisation
loige
✉Reach out to us at
😇We are always looking for talent:
hello@fourTheorem.com
fth.link/careers
16

17. We host a weekly podcast about AWS
loige
awsbites.com
loige 17

18. 📒AGENDA
Generators
Iterator protocol
Iterable protocol
Async Iterator protocol
Async Iterable protcol
Real-life™ examples
loige 18

19. LET'S CLONE 🐑🐏
loige
loige.link/lets-iter-repo
git clone https://github.com/lmammino/iteration-protocols-workshop.git
cd iteration-protocols-workshop
npm i
19

20. ➡GENERATORS
loige 20

21. GENERATOR FN & OBJ
loige
function * myGenerator () {
// generator body
yield 'someValue'
// ... do more stuff
}
const genObj = myGenerator()
genObj.next() // -> { done: false, value: 'someValue' }
21

22. function * fruitGen () {
yield '🍑'
yield '🍉'
yield '🍋'
yield '🥭'
}
const fruitGenObj = fruitGen()
console.log(fruitGenObj.next()) // { value: '🍑', done: false }
console.log(fruitGenObj.next()) // { value: '🍉', done: false }
console.log(fruitGenObj.next()) // { value: '🍋', done: false }
console.log(fruitGenObj.next()) // { value: '🥭', done: false }
console.log(fruitGenObj.next()) // { value: undefined, done: true }
loige 22

23. function * fruitGen () {
yield '🍑'
yield '🍉'
yield '🍋'
yield '🥭'
}
const fruitGenObj = fruitGen()
// generator objects are iterable!
for (const fruit of fruitGenObj) {
console.log(fruit)
}
// 🍑
// 🍉
// 🍋
// 🥭
loige 23

24. function * range (start, end) {
for (let i = start; i < end; i++) {
yield i
}
}
// generators are lazy!
for (const i of range(0, Number.MAX_VALUE)) {
console.log(i)
}
const zeroToTen = [...range(0, 11)]
loige 24

25. // generators can be "endless"
function * cycle (values) {
let current = 0
while (true) {
yield values[current]
current = (current + 1) % values.length
}
}
for (const value of cycle(['even', 'odd'])) {
console.log(value)
}
// even
// odd
// even
// ...
loige 25

26. 📝MINI-SUMMARY
loige
A generator function returns a generator object which is both an iterator and an iterable.
A generator function uses `yield` to yield a value and pause its execution. The generator
object is used to make progress on an instance of the generator (by calling `next()`).
Generator functions are a great way to create custom iterable objects.
Generator objects are lazy and they can be endless.
26

27. 📝EXERCISE(S)
loige
02-generators/exercises/zip.js
function * take (n, iterable) {
// take at most n items from iterable and
// yield them one by one (lazily)
}
02-generators/exercises/zip.js
function * zip (iterable1, iterable2) {
// consume the two iterables until any of the 2 completes
// yield a pair taking 1 item from the first and one from
// the second at every step
}
27

28. ➡ITERATORS & ITERABLES
loige 28

29. ITERATOR OBJ
loige
An object that acts like a cursor to iterate over blocks of data sequentially
29

30. ITERABLE OBJ
loige
An object that contains data that can be iterated over sequentially
30

31. THE ITERATOR PROTOCOL
An object is an iterator if it has a next() method.
Every time you call it, it returns an object with the
keys done (boolean) and value.
loige 31

32. function createCountdown (from) {
let nextVal = from
return {
next () {
if (nextVal < 0) {
return { done: true }
}
return {
done: false,
value: nextVal--
}
}
}
} loige 32

33. const countdown = createCountdown(3)
console.log(countdown.next())
// { done: false, value: 3 }
console.log(countdown.next())
// { done: false, value: 2 }
console.log(countdown.next())
// { done: false, value: 1 }
console.log(countdown.next())
// { done: false, value: 0 }
console.log(countdown.next())
// { done: true } loige 33

34. 🔥
Generator Objects are iterators (and iterables)!
loige 34

35. function * createCountdown (from) {
for (let i = from; i >= 0; i--) {
yield i
}
}
loige 35

36. const countdown = createCountdown(3)
console.log(countdown.next())
// { done: false, value: 3 }
console.log(countdown.next())
// { done: false, value: 2 }
console.log(countdown.next())
// { done: false, value: 1 }
console.log(countdown.next())
// { done: false, value: 0 }
console.log(countdown.next())
// { done: true, value: undefined } loige 36

37. THE ITERABLE PROTOCOL
An object is iterable if it implements the
Symbol.iterator method, a zero-argument function
that returns an iterator.
loige 37

38. function createCountdown (from) {
let nextVal = from
return {
[Symbol.iterator]: () => ({
next () {
if (nextVal < 0) {
return { done: true }
}
return { done: false, value: nextVal-- }
}
})
}
}
loige 38

39. function createCountdown (from) {
return {
[Symbol.iterator]: function * () {
for (let i = from; i >= 0; i--) {
yield i
}
}
}
}
loige 39

40. function * createCountdown () {
for (let i = from; i >= 0; i--) {
yield i
}
}
loige
🔥or just use generators!
40

41. const countdown = createCountdown(3)
for (const value of countdown) {
console.log(value)
}
// 3
// 2
// 1
// 0
loige 41

42. const iterableIterator = {
next () {
return { done: false, value: 'hello' }
},
[Symbol.iterator] () {
return this
}
}
An object can be an iterable and an iterator at the same time!
loige 42

43. 📝MINI-SUMMARY 1/2
loige
An iterator is an object that allows us to traverse a collection
The iterator protocol specifies that an object is an iterator if it has a `next()` method that
returns an object with the shape `{done, value}`.
`done` (a boolean) tells us if the iteration is completed
`value` represents the value from the current iteration.
You can write an iterator as an anonymous object (e.g. returned by a factory function),
using classes or using generators.
43

44. 📝MINI-SUMMARY 2/2
loige
The iterable protocol defines what's expected for a JavaScript object to be considered
iterable. That is an object that holds a collection of data on which you can iterate on
sequentially.
An object is iterable if it implements a special method called `Symbol.iterator` which
returns an iterator. (An object is iterable if you can get an iterator from it!)
Generator functions produce objects that are iterable.
We saw that generators produce objects that are also iterators.
It is possible to have objects that are both iterator and iterable. The trick is to create the
object as an iterator and to implement a `Symbol.iterator` that returns the object itself
(`this`).
44

45. 📝EXERCISE(S)
loige
04-iterable-protocol/exercises/binarytree.js
class BinaryTree {
// <implementation provided>
// ...
// make this iterable!
}
04-iterable-protocol/exercises/entries.js
function entriesIterable (obj) {
// Return an iterable that produce key/value pairs
// from obj
}
45

46. OK, very cool!
But, so far this is all synchronous iteration.
What about async? 🙄
loige 46

47. ➡ASYNC ITERATORS & ITERABLES
loige 47

48. THE ASYNC ITERATOR PROTOCOL
An object is an async iterator if it has a next() method.
Every time you call it, it returns a promise that resolves
to an object with the keys done (boolean) and value.
loige 48

49. import { setTimeout } from 'node:timers/promises'
function createAsyncCountdown (from, delay = 1000) {
let nextVal = from
return {
async next () {
await setTimeout(delay)
if (nextVal < 0) {
return { done: true }
}
return { done: false, value: nextVal-- }
}
}
} loige 49

50. const countdown = createAsyncCountdown(3)
console.log(await countdown.next())
// { done: false, value: 3 }
console.log(await countdown.next())
// { done: false, value: 2 }
console.log(await countdown.next())
// { done: false, value: 1 }
console.log(await countdown.next())
// { done: false, value: 0 }
console.log(await countdown.next())
// { done: true } loige 50

51. loige 51

52. import { setTimeout } from 'node:timers/promises'
// async generators "produce" async iterators!
async function * createAsyncCountdown (from, delay = 1000) {
for (let i = from; i >= 0; i--) {
await setTimeout(delay)
yield i
}
}
loige 52

53. THE ASYNC ITERABLE PROTOCOL
An object is an async iterable if it implements the
`Symbol.asyncIterator` method, a zero-argument
function that returns an async iterator.
loige 53

54. import { setTimeout } from 'node:timers/promises'
function createAsyncCountdown (from, delay = 1000) {
return {
[Symbol.asyncIterator]: async function * () {
for (let i = from; i >= 0; i--) {
await setTimeout(delay)
yield i
}
}
}
}
loige 54

55. HOT TIP 🔥
With async generators we can create objects that are
both async iterators and async iterables!
(We don't need to specify
Symbol.asyncIterator explicitly!)
loige 55

56. import { setTimeout } from 'node:timers/promises'
// async generators "produce" async iterators
// (and iterables!)
async function * createAsyncCountdown (from, delay = 1000) {
for (let i = from; i >= 0; i--) {
await setTimeout(delay)
yield i
}
}
loige 56

57. const countdown = createAsyncCountdown(3)
for await (const value of countdown) {
console.log(value)
}
loige 57

58. 📝MINI-SUMMARY 1/2
loige
Async iterators are the asynchronous counterpart of iterators.
They are useful to iterate over data that becomes available asynchronously (e.g. coming
from a database or a REST API).
A good example is a paginated API, we could build an async iterator that gives a new page
for every iteration.
An object is an async iterator if it has a `next()` method which returns a `Promise` that
resolves to an object with the shape: `{done, value}`.
The main difference with the iterator protocol is that this time `next()` returns a promise.
When we call next we need to make sure we `await` the returned promise.
58

59. 📝MINI-SUMMARY 2/2
loige
The async iterable protocol defines what it means for an object to be an async iterable.
Once you have an async iterable you can use the `for await ... of` syntax on it.
An object is an async iterable if it has a special method called `Symbol.asyncIterator` that
returns an async iterator.
Async iterables are a great way to abstract paginated data that is available
asynchronously or similar operations like pulling jobs from a remote queue.
A small spoiler, async iterables can also be used with Node.js streams...
59

60. 📝EXERCISE
loige
06-async-iterable-protocol/exercises/rickmorty.js
function createCharactersPaginator () {
// return an iterator that produces pages
// with name of Rick and Morty characters
// taken from the API
// https://rickandmortyapi.com/api/character
}
// This is what we want to support 👇
const paginator = createCharactersPaginator()
for await (const page of paginator) {
console.log(page)
}
60

61. ➡TIPS &
ASYNC ITERATORS IN NODE.JS
loige 61

62. function isIterable (obj) {
return typeof obj[Symbol.iterator] === 'function'
}
const array = [1, 2, 3]
console.log(array, isIterable(array)) // true
const genericObj = { foo: 'bar' }
console.log(isIterable(genericObj)) // false
console.log(isIterable(Object.entries(genericObj))) // true
const fakeIterable = {
[Symbol.iterator] () { return 'notAnIterator' }
}
console.log(fakeIterable, isIterable(fakeIterable)) // true 😡
IS THIS OBJECT AN ITERABLE?
loige 62

63. function isAsyncIterable (obj) {
return typeof obj[Symbol.asyncIterator] === 'function'
}
IS THIS OBJECT AN ASYNC ITERABLE?
loige
Are there async iterable objects in Node.js core?
63

64. console.log(
typeof process.stdin[Symbol.asyncIterator] === 'function'
) // true!
let bytes = 0
for await (const chunk of process.stdin) {
bytes += chunk.length
}
console.log(`${bytes} bytes read from stdin`)
READABLE STREAMS!
loige 64

65. import { createWriteStream } from 'node:fs'
const dest = createWriteStream('data.bin')
let bytes = 0
for await (const chunk of process.stdin) {
// what if we are writing too much too fast?!! 😱
dest.write(chunk)
bytes += chunk.length
}
dest.end()
console.log(`${bytes} written into data.bin`)
⚠BACKPRESSURE WARNING
loige 65

66. import { createWriteStream } from 'node:fs'
import { once } from 'node:events'
const dest = createWriteStream('data.bin')
let bytes = 0
for await (const chunk of process.stdin) {
const canContinue = dest.write(chunk)
bytes += chunk.length
if (!canContinue) {
// backpressure, now we stop and we need to wait for drain
await once(dest, 'drain')
// ok now it's safe to resume writing
}
}
dest.end()
console.log(`${bytes} written into data.bin`)
loige
✅handling backpressure like a pro!
... or you can use pipeline()
66

67. import { pipeline } from 'node:stream/promises'
import {createReadStream, createWriteStream} from 'node:fs'
await pipeline(
createReadStream('lowercase.txt'),
async function* (source) {
for await (const chunk of source) {
yield await processChunk(chunk)
}
},
createWriteStream('uppercase.txt')
)
console.log('Pipeline succeeded.')
loige
pipeline() supports async iterables! 😱
67

68. import { on } from 'node:events'
import glob from 'glob' // from npm
const matcher = glob('**/*.js')
for await (const [filePath] of on(matcher, 'match')) {
console.log(filePath)
}
loige
creates an async iterable that will yield
every time the `match` event happens
68

69. import { on } from 'node:events'
import glob from 'glob' // from npm
const matcher = glob('**/*.js')
for await (const [filePath] of on(matcher, 'match')) {
console.log(filePath)
}
// ⚠ WE WILL NEVER GET HERE 👇
console.log('Done')
loige
This loop doesn't know when to stop!
You could pass an
AbortController here
for more fine-grained
control
69

70. 📝FINAL SUMMARY
loige
You can check if an object is an iterable by checking `typeof obj[Symbol.iterator] === 'function'`
You can check if an object is an async iterable with `typeof obj[Symbol.asyncIterator] === 'function'`
In both cases, there's no guarantee that the iterable protocol is implemented correctly (the function
might not return an iterator 😥)
Node.js Readable streams are also async iterable objects, so you could use `for await ... of` to
consume the data in chunks
If you do that and you end up writing data somewhere else, you'll need to handle backpressure
yourself. It might be better to use `pipeline()` instead.
You can convert Node.js event emitters to async iterable objects by using the `on` function from the
module `events`.
70

71. Cover picture by on
Back cover picture by on
Camille Minouflet Unsplash
Antonino Cicero Unsplash
fourtheorem.com
THANKS! 🙌
loige.link/lets-iter
loige
nodejsdp.link
71