← Home

The JavaScript Hard Parts Series by Will Sentance

Notes on Will Sentance’s JavaScript Hard Parts series:

Thread of execution

There is a global execution context with its own global memory and, for each function, there is a local execution context with its own local memory.

Since JavaScript has a single thread of execution, it runs each line one by one at a time, always at the global execution context but entering and exiting local execution contexts whenever functions are called.

Call stack

The call stack (think a stack of dishes) is used by JavaScript to track which execution context it is at—whatever function is at the top of the call stack is the function currently runnning. If we are in the global execution context and call a function, the function executes: its mini-execution context is created and the function is pushed onto the stack. Once the function fully executes and its mini-execution context is destroyed, that function is popped off the stack and JavaScript knows that it must now return to the global execution context.

The stack receives a function call, runs the function and is emptied out. Since JavaScript is single-threaded, the call stack can only have a single function at a time, unless that function calls a second function, which is placed on top of the stack, executed and taken out of the stack, so that the first function can finish executing.

Asynchronicity

Synchronous execution: Execution of code line by line, only moving over to the next line after finishing the previous line. Blocking mode of execution.

Asynchronous execution: Execution of more than one line at a time. Non-blocking mode of execution. In async JavaScript we want to:

  1. initiate a task that takes a long time (HTTP request),
  2. move on to more synchronous regular code in the meantime, and
  3. run some functionality once the requested data has come back.

JavaScript is single-threaded (one command at a time) and has a synchronous execution model (each line is executed in the order the code appears). But what if we need to wait some time before we can execute certain bits of code? Waiting could mean e.g. waiting for an API request to return a response, or for a timer to go off.

We therefore have a tension between wanting to delay some code execution until something happens, but not wanting to block the thread of execution from running other code while we wait.

Promises allow you to track the eventual result and add code to be triggered once the eventual result becomes a concrete result, all the while the thread of execution is not blocked and continues to run the rest of the code.

Browser features

Many JavaScript functions like setTimeout() and fetch() are actually facade functions (fronts) that call functionality implemented in the browser. When you call a facade function from JavaScript, the thread of execution moves from the facade function straight on to the next function, because the functionality of the facade function has been delegated to the browser.

const sayHello = () => console.log("Hello");

setTimeout(sayHello, 1000); // delegated to browser, goes off 1 second later

console.log("Me first!"); // printed first despite being located after

Once the functionality in the browser takes as long as it needed to, before going to the call stack in JavaScript, it goes to the callback queue, a.k.a. the macro task queue. This way, once the call stack in JavaScript becomes empty again, the event loop checks the callback queue for any functions there waiting to be executed. The event loop constantly asks “Is the call stack empty?” and if so, “Is there something in the callback queue?”

Remember: A function is allowed to go from the callback queue back into the call stack only when the call stack is empty, i.e. only when all regular execution or synchronous code in the global context is done.

Promises

While there is background (browser) functionality going on, a promise is placeholder for the eventual result of that functionality. Facade functions have two prongs: one initiates background (browser) functionality; the other returns a promise object (placeholder) that sits in memory and is filled in with the result of the functionality if successful. This way we can keep track and tack consequences onto the promise.

A promise has:

  • a value property to be filled in with the eventual result, and
  • an onFulfilled array containing the function that is triggered when the result comes back, i.e. the function that goes in .then().

Promise.then(fn) stores fn in the Promise’s onFulfilled hidden property. That stored function will be auto-triggered when the Promise’s property value receives an actual value and will auto-receive that value as an argument.

Important difference: Any browser functionality that returns a promise, when it has the result, sends it to the microtask queue (for promises only), which is different from and has priority over the callback queue (for other deferred functionality).

The event loop checks first the microtask queue and, if the call stack is empty, moves the function in the microtask queue over to the call stack and auto-inserts the result of the call as an argument. Once both the call stack and the microtask queue are empty, only then does the event loop check the callback queue.

Any function that is attached to a promise object goes to the microtask queue. Any function that is passed to a facade function that triggers a browser feature, when completed, goes back to the callback queue.

If the facade function takes in a function worked on by the browser, then callback queue. If the facade function returns a promise, then microtask queue.

Functions and queues
Functions and queues

For example, if you defer a function using setTimeout (non-promise-returning facade function) and then you defer a function using fetch (promise-returning facade function), setTimeout goes to the micro task queue and fetch goes to the macro task queue, and so the order of execution becomes:

  1. synchronous code,
  2. promise-returning facade functions in microtask queue, and
  3. non-promise-returning functions in macrotask queue.

Closure

A closure is a function that remembers its outer variables and can access them:

let b = 1;
function closureSum(a) {
	return a + b; // `b` is closed over!
}

A function may access outer variables, but this works only from the inside out. The code outside of the function does not see any inner variables.

A function is a closure when it “closes over” (i.e. encompasses, grabs onto) all the external variables used in its body, so that the function has all it needs to execute. This “closing over” happens when the function is defined.

By “external variable” is meant a variable that is neither in the parameters of the function nor defined in the body of the function. Thus, a closure is the combination of a function and the scope object in which it was created. These closed over variables can be read and updated.

JavaScript function values contain more information than just the code required to execute when they are called. They also internally store any variables they may refer to that are defined in their enclosing scopes. Functions that keep track of variables from their containing scopes are known as closures.

  • Closed-over variable environment: Scope where function was defined.
  • Persistent lexically scoped data: The data in that scope.
  • Backpack: Subset of that data, namely the external data needed and stored by a function, accessible via the hidden property [[scope]].

The backpack is the result of JavaScript being a lexically scoped language.

Closure gives functions persistent memories, as used in once(), memoize(), iterators, generators, module pattern and promises.

Scope is the data environment available at a given line in a section of code. In JavaScript, we have lexical a.k.a. static scoping, so where I save/define my function determines what data that function will have access to whenever it gets run. (Dynamic scoping, not in JavaScript, means that where I run the function determines that scope.) Static scoping means that, even if I return a function out from another function, that returned function will still have access to the data environment it needs (in its backpack, or store of external necessary data) in order for the function to be run.

Prototype

Problem: We want to link shared functions to multiple objects without creating multiple copies of the shared functions.

Solution: We can define the shared functions in a single object and link every object to that single object storing the shared functions.

Explanation: This link we create is the hidden property __proto__. This __proto__ of an object refers to (links back to) another object as its prototype, i.e. the base object. When JavaScript does not find a member on an object, it walks up the prototype chain—it looks it up in that object’s prototype.


Alternate explanation

Objects have references to other objects. To keep code DRY, objects that share functionality have a reference to a store of that shared functionality.

function userCreator(name, score) {
	const newUser = Object.create(userFunctionStore);
	newUser.name = name;
	newUser.score = score;
	return newUser;
}

const userFunctionStore = {
	increment: function() {
		this.score++;
	}
};

Object.create() lets us gain control of the __proto__ property.

Object.create(userFunctionStore) sets userFunctionStore as prototype of the created newUser, so newUser has access to the function in userFunctionStore. newUser has a hidden property __proto__ which refers to userFunctionStore, i.e. userFunctionStore is the prototype of newUser.

Mind the difference: __proto__ is a hidden property linking back to an object’s prototype or base object. The prototype is that base object.

This link we created means that, if you call a method on newUser, JavaScript will look for that method on newUser and, if not found, it will walk up the prototype chain and find in on its prototype: userFunctionStore.

If you do not use Object.create(), all objects in JavaScript have a default prototype: Object.protoype. In this case, newUser refers to its protoype userFunctionStore, which then refers to its prototype Object.prototype, which is the God object with commonly used methods and properties—its __proto__ is simply null.

Functions are a composite of functions and objects

function multiplyBy2(num) {
	return num * 2;
}

multiplyBy2.stored = 5; // a function can have a property! just like an object

Every function is a function-object combo and the “object side” automatically has a property on it called prototype with another object as its value. This prototype property is not a hidden property.

Mind the difference: Do not confuse the non-hidden prototype property in a function with the __proto__ hidden property in normal objects.

A function-object combo
A function-object combo
function userCreator() {
	this.name = name;
	this.score = score;
}

userCreator.prototype.increment = function() {
	this.score++;
};

increment is added to the object that is the value of the property prototype in userCreator.

The new keyword

The new keyword:

  1. creates an empty object and sets this to the new object that is being created,
  2. sets the new object’s __proto__ to the constructor’s prototype (constructor’s object containing shared members), and
  3. returns the new object.

Meanwhile, we add fields and methods to this (the new object) via arguments.

function User(name) {
	this.name = name;
	this.isAdmin = false;
	this.sayHi = () => console.log("Hi!");
}

let jack = new User("Jack"); // `jack.__proto__` will refer to `User.prototype`, so if we store functions in `User.prototype`, then `jack` will have access to those functions

alert(jack.name); // Jack
alert(jack.isAdmin); // false
jack.sayHi(); // "Hi!`

The constructor function is itself also an object that has a prototype property that has an object value. That prototype object value may contain functions. Any instance created out of the constructor function will have the constructor function at the instance’s __proto__ and therefore will have access to the functions at the object at the constructor’s prototype.


Alternate explanation

Whenever you declare a function, you get a function coupled with an object containing a property prototype that has an object as its value.

function userCreator(name, score) {
	this.name = name;
	this.score = score;
}

// `userCreator.prototype` is an object and a shared function is added inside that object
userCreator.prototype.increment = function() {
	this.score++;
};

When you new up an object using that constructor function, the newly created object gets a __proto__ hidden property pointing to the constructor function’s prototype property, which has an object storing the function that was added to the prototype.

Linking to the prototype
Linking to the prototype

__proto__ is the new object’s arrow pointing to prototype, a bundle of functions attached to the constructor.

In this way, the function stored at the prototype of the constructor function becomes a method on the newly created object.

const john = new userCreator("John", 4);
john.increment();

Classes as syntactic sugar

A class is a joint declaration of fields and methods that would be declared separately if using a function’s prototype property. A class is syntactic sugar over the function-object combo. The class bundles up the constructor (which sets up the instance variables or fields) together with the methods (instance functions).

Separate:

function userCreator(name, score) {
	this.name = name;
	this.score = score;
}

userCreator.prototype.increment = function() {
	this.score++;
};

Joint:

class User {
	constructor(name, score) {
		this.name = name;
		this.score = score;
	}
	increment() {
		this.score++;
	}
}

Subclassing prototypes

Objects in JavaScript do not really inherit from other objects so much as they refer to other objects via __proto__. Always think “lookup chain”. The parent’s properties are not in the child—it is that the child has access to them, but the parent’s properties are still in the parent.

Goal:

user with __proto__userFunctionStore

const newUser = Object.create(userFunctionStore);

paidUser with __proto__paidUserFunctionStore with __proto__userFunctionStore

const newPaidUser = userCreator(paidName, paidScore);
Object.setPrototypeOf(newPaidUser, paidUserFunctionStore);

// ...

Object.setPrototypeOf(paidUserFunctionStore, userFunctionStore);

In full:

function userCreator(name, score) {
	const newUser = Object.create(userFunctionStore);
	newUser.name = name;
	newUser.score = score;
	return newUser;
}

userFunctionStore = {
	sayName: function() {
		console.log("I'm " + this.name);
	}
};

function paidUserCreator(paidName, paidScore, accountBalance) {
	const newPaidUser = userCreator(paidName, paidScore);
	Object.setPrototypeOf(newPaidUser, paidUserFunctionStore);
	newPaidUser.accountBalance = accountBalance;
	return newPaidUser;
}

const paidUserFunctionStore = {
	increaseBalance: function() {
		this.accountBalance++;
	}
};

Object.setPrototypeOf(paidUserFunctionStore, userFunctionStore);

const john = paidUserCreator("John", 8, 25);

john.sayName();

Execution context

An execution context is a wrapper for your code, created by the JavaScript engine. The execution context can be global (accessible to everything in your code) or local (accessible only to a function). The execution context:

  • stores data (variables and functions) → create phase
  • acts on data (runs functions on data) → execute phase
  • has a reference to its outer environment
  • has a reference to arguments, any parameters passed in to the function
  • has a reference to [[scope]], any closed over variables in its outer environment

Aside: arguments is array-like. It acts and looks like an array, but it is not exactly an array.

The outer environment of a function is the environment where the function is saved, not where it is eventually called from.

function b() {
	console.log(myVar); // → "1"
	// not found in local, found in global does NOT search in function `a`
}

function a() {
	var myVar = 2;
	b();
}

var myVar = 1;
a();

But if you change the lexical environment…

function a() {
	function b() {
		console.log(myVar); // → "2"
		// searched in local, found in environment of function `a`
	}
	var myVar = 2;
	b();
}

var myVar = 1;
a();

Another change…

function a() {
	function b() {
		console.log(myVar); // → "1"
		// searched in local, searched in outer, found in global
	}
	b();
}

var myVar = 1;
a();

At this last change, b is only created once a is called. Any execution context, in this case that of b, is first created (creation phase) and then run (execution phase).

Global execution context

“Global” means “not inside a function”.

The global execution context creates two things for you:

  • the global object
  • the special variable this

Inside the browser, this is equal to window. Inside Node, this is equal to global.

The global execution context creates an object and a variable pointing to that object. Any variables and functions created in the global scope are added to the global object.

// app.js

var a = "hello";

// browser console

Window.a; // → "hello"

Execution context stacking

<body>
	<script src="lib1.js"></script>
	<script src="lib2.js"></script>
	<script src="app.js"></script>
</body>

When these three JavaScript files are run, a single execution context is created and the three files are treated as if they were a single JavaScript file, so they are stacked in a single execution context. Variables global to each of the files may collide and cause overwriting of variables!

Hoisting

Functions are hoisted (moved upwards) to the top of the file—or so it is usually said. What actually happens is that, when the execution context is created, the JavaScript engine prepares space for variables and functions that are declared in your code. Variables are prepared by being set to undefined, i.e. uninitialized.

function b() {
	console.log("Hello World!");
}

b(); // → "Hello World!"

console.log(a); // → undefined

var a = "Hello World!";

console.log(a); // → "Hello World!"

Local execution context

When you call a function, you create a local execution context and push it onto the call stack.

Scope

Scope is where code is available in your code.

Scope chain

When reference is made to a variable that does not exist in the local execution context, the JavaScript engine looks for it in its outer execution context, which can be a surrounding function and simply the global execution context.

This only works from the inside out, not from the outside in!

The scope chain is the chain of scopes where the JavaScript engine looks for data.

let and const restrict the scope of the variable to inside the scope (block) the variable is created in. The scope chain will not search for let and const variables outside the scope they are created in.

Passing by value and by reference

Primitive values are “passed by value”, i.e. passed by having the value be actually copied into two separate locations in memory, so the second variable name is assigned to a new independent primitive value. The primitive value is thus duplicated, created again. Two names point to two different locations.

var a = 3;
var b;
var b = a; // two different locations in memory holding `3`

a = 2; // `b` is unaffected!

Objects are “passed by reference”, i.e. passed by having the variable point to the same location in memory as the original location in memory, so the second variable name is assigned to that same reference. No new object is created. Two names point to the same location.

var c = { greeting: "hi" };
var d;
d = c; // both point to same location in memory holding single object

c.greeting = "hello"; // `d` is also affected!

Passing by reference also occurs in parameters/arguments:

var c = { greeting: "hi" };
var d;

d = c;

function changeGreeting(obj) {
	obj.greeting = "Hola";
}

changeGreeting(c); // `d` is also affected!

The = operator sets up a new memory space:

c = { greeting: "howdy" };
d = { greeting: "howdy" };
// `c` and `d` are two different objects in two different locations in memory

Iterators

Iterating is going through each item in a collection (object, array, set, etc.) and doing something to each item in the collection. This implies:

  • the intermediate work of accessing (getting, grabbing) each item in the collection, and
  • the actual work on doing something to each item.
// back to basics

const numbers = [4, 5, 6];

for (let i = 0; i < numbers.length; i++) {
	console.log(numbers[i]);
}

// condition before all iterations      let i = 0;
// condition before each iteration      i < numbers.length;
// action at each iteration             console.log(numbers[i]);
// action after each iteration          i++;

Functional iteration

An iterator automates the intermediate step of accessing each item in the collection and makes it available, so we can focus only on what to do to each item.

An iterator is a function that creates a collection and, when the iterator is called, it returns the next item in the collection. Naturally, to be useful, the iterator needs to remember (keep track of) which item is next. It is closures that allow iterators to have memory!

Iterators allow you to think of a collection as a flow of data (think water tap) to be turned on and off (opened and closed) whenever you need to act on that data.

function createFunction(array) {
	let i = 0;
	function inner() {
		const item = array[i]; // `array` and `i` are closed over!
		i++;
		return item;
	}
	return inner;
}

const returnNextElement = createFunction([4, 5, 6]);

Now returnNextElement is the new name for what was called inner:

// body of function formerly called `inner`s
{
	const element = array[i]; // two closed over variables!
	i++;
	return element;
}

So whenever you call returnNextElement, you get the next element in the array passed in as an argument when returnNextElement was created, until you hit undefined. In other words, now instead of actively accessing the items in the collection, you call your iterator and are given the next element.

The inner function, when called, searches for the variables it its local memory, then in its closed over variables (backpack!) and finds them there. The i variable is updated and persists, enabling the iterator to have memory and thus to track where it is at after being called.

This function is an iterator. Any function that, when called, returns the next element in a collection, is an iterator. We have now effectively decoupled the process of accessing each item from the action done on each item.

Generators

    function createFlow(array) {
        let i = 0;
        const inner = { next:
            function() {
                const element = array[i];
                i++;
                return element;
            }
        return inner;
    }

    const returnNextElement = createFlow([4,5,6]);

    const element1 = returnNextElement.next(); // → 4

Now returnNextElement is an object with a next method.

function* createFlow() {
	yield 4;
	yield 5;
	yield 6;
}

const returnNextElement = createFlow();
const element1 = returnNextElement.next();

Implementing map()

function mapImplementation(array, cb) {
	let output = [];
	for (let i = 0; i < array.length; i++) {
		output.push(cb(array[i]));
	}
	return output;
}

mapImplementation([1, 2, 3], input => input * 2);

map() takes the first element from an initial array, applies a function to the element, and pushes the resulting element to an empty array. Then map() takes the second element from the initial array, applies a function to the element and pushes the resulting element to an accumulator array (i.e. an array with a resulting element obtained from applying the function to the first element). This pushing of a resulting element into an accumulator array, thereby converting the resulting element and the accumulator array into a single entity, is a reduction.

The original name of map is mapreduce!

Map and reduce
Map and reduce

Implementing reduce()

Just as an array is the accumulator in map(), a number is the accumulator in reduce().

Reduction is combining two things into one thing repeatedly through each element. The reducer is the function or rule by which we combine two things to have them become one. The accumulator is the running output, i.e. the output that absorbs the elements resulting from applying the function, that serves as the input to the next application of the function.

const reduce = (array, howToCombine, buildingUp) => {
	for (let i = 0; i < array.length; i++) {
		buildingUp = howToCombine(buildingUp, array[i]);
	}
	return buildingUp;
};

const add = (a, b) => a + b;

const summed = reduce([1, 2, 3], add, 0);
Reducer
Reducer

Higher-order functions

Higher-order functions (HOFs) are functions that take other functions:

  • map
  • forEach
  • filter
  • reduce

All iterate over each element and apply a function on it.

Chaining

Chaining is passing the output one HOF as the input of the next.

const array = [1, 2, 3, 4, 5, 6];
const greaterThan2 = num => num > 2;
const add = (a, b) => a + b;
const sumOfGreaterThan2 = array.filter(greaterThan2).reduce(add, 0);

Chaining with dots relies on JavaScript’s prototype feature: functions return arrays, which have access to all the HOF (map, filter, reduce). The output array is passed as an input into the next function.

But what if I want to chain functions that just return a regular output?

We could keep track with global variables, but it is risky because people may overwrite a step:

const multiplyBy2 = x => x * 2;
const add3 = x => x + 3;
const divideBy5 = x => x / 5;

const initialResult = multiplyBy2(11);
const nextStep = add3(initialResult);
const finalStep = divideBy5(nextStep);

Here is where function composition comes in.

Function composition

The principle behind reduce() enables function composition.

One attempt: Nesting functions

const result = divideBy5(add3(multiplyBy2(11))); // unreadable and inverse order!

We are combining a function with a value to get a result, and then combining that result with another function to get another result, and so on. This is reminiscent of reduce().

So, using reduce(), we get:

const multiplyBy2 = x => x * 2;
const add3 = x => x + 3;
const divideBy5 = x => x / 5;

const reduce = (array, howToCombine, buildingUp) => {
	for (let i = 0; i < array.length; i++) {
		buildingUp = howToCombine(buildingUp, array[i]);
	}
	return buildingUp;
};

const runFunctionOnInput = (input, fn) => fn(input);

const output = reduce([multiplyBy2, add3, divideBy5], runFunctionOnInput, 11);

Pass in an array of functions to reduce(), a function that represents the rule to combine the result of running each of those functions with the accumulator, and the initial value for the accumulator.

Step by step:

  1. Pass in args.
  2. Loop over funcs.
  3. First func multiplyBy2 is called with acc and element.(*)
  4. Result of call goes into acc.
  5. Second func add3 is called with acc (now result) and element.
  6. Result of call goes into acc.

(*) Why? Because runFunctionOnInput takes in the acc and the element. input is the acc and fn is each func in the array. runFunctionOnInput simply calls that func with the acc.

Composing functions is using a reducer to convert their operations to a single operation. Composition is taking a function (one of many operations) and using a function or combinatorial rule (reducer) to combine that taken function with a value (accumulator) into a resulting value.

  • multiplyByTwo → element in array
  • runFunctionOnInput → reducer / combinatorial rule
  • number → accumlator / running output passed in as next argument
Composing functions
Composing functions

If you are listing out functions to compose, you need to have them take in one input and return one output. Ensure this consistency.

const output = reduce([multiplyBy2, add3, divideBy5], runFunctionOnInput, 11);

// tasks listed out sequentially in human-readable name → readability!

For this, instead of reduce(), we specifically use compose() or pipe().

Composing functions means combining functions to be run one by one on our data.

Purity & Immutability

No line should alter another line. No side effects!

Pure functions are those whose only consequence is the evaluated result.

let num = 10;

const add3 = x => {
	num++; // side effect! big NO!
	return x + 3;
};

add3(7);

Immutable data is data that cannot be changed. You change the data but save it under a different label, effectively preserving the original data. This is why map() returns a new array.

Currying

In functional programming, “currying” means transforming a function that takes two or more arguments into a function that takes one argument. This is achieved by having the currying function take one argument and return another function that takes another argument and does something with both arguments.

Closures enable currying to make sure that the arity of our functions matches up: avoid arity mismatch! When you return a function out from another function, give it a global label and run it, that returned function has access to the external variables it needs to run.

Granted, closures are not exactly pure, as they keep state and so every time you call the function it behaves differently.

Function decoration

Editing a function so as to reuse code, without rewriting the function from scratch.

const oncify = convertMe => {
	// `convertMe` is closed over
	let counter = 0;
	const inner = input => {
		if (counter === 0) {
			// `counter` is closed over
			const output = convertMe(input);
			counter++;
			return output;
		}
		return "Sorry!";
	};
	return inner;
};

const multiplyBy2 = num => num * 2;
const oncifiedMultiplyBy2 = oncify(multiplyBy2);

oncifiedMultiplyBy2(10); // → 20
oncifiedMultiplyBy2(2); // → "Sorry!"

Partial application

To prevent arity mismatch, prefill an argument in the closure/backpack.

const multiply = (a, b) => a * b; // arity of 2

function prefillFunction(fn, prefilledValue) {
	// `prefilledValue` is closed over by `inner`!
	const inner = liveInput => {
		return fn(liveInput, prefilledValue);
	};
}

const multiplyBy2 = prefillFunction(multiply, 2);

const result = multiplyBy2(3); // arity of 1