Have you ever received an unexpected `null` reference? Have you ever written a function to validate some input only to have it turn into spaghetti over time? How do you anticipate exceptions and protect against them at runtime?

@ -30,31 +32,35 @@ _[Defensive programming][]_ as a practice tries to protect against errors and un

In this post I will provide **effects** as a model for characterizing nondeterminism and unknowns in programs. I will introduce a **design pattern** to abstract complexity using this model.

> The code that accompanies this post may be found [here]({{code_repo}}).

## Conventions

**This post assumes familiarity with Scala code**. I will provide Scala code for concrete examples and note where they are different. Abstract examples will employ notation that looks like **"math"**.

Terminology will leverage common functional programming vocabulary.

Terminology will leverage common functional programming vocabulary. Terms when they are first introduced will be **bold** and important references are _italicized_. Some terminology is provided up-front, other terms will be defined inline either explicitly or by context.

Concepts and phrases that are important are _italicized_.

Where there is conceptual overlap with object oriented programming, I will leverage those terms to drive the intent behind abstractions.

### How to read "math"

**Uppercase letters** and words starting with uppercase letters are names of _types_. For example **A** reads as _"A"_ and means _"type of A"_.

**Uppercase letters** and words starting with uppercase letters are names of _types_. For example **`A`** reads as _"A"_ and means _"type of A"_.

**Lowercase letters** and words starting with a lowercase letter are functions or variables. **f** reads as _"f"_ and means _"function f"_ or _"variable f"_.

**Lowercase letters** and words starting with a lowercase letter are functions or variables. For example **`f`** reads as _"f"_ and means _"function f"_ or _"variable f"_.

**A => B** reads as _"A to B"_ and means _"function type of input type A mapped to output type B"_. Functions types can map any kind of input type to any other kind of output type, including function types.

**`A => B`** reads as _"A to B"_ and means _"function type of input type A mapped to output type B"_. Function types can map any kind of input type to any kind of output type, including function types as in `(A => B) => C` or `A => B => C`. Note that the `=>` operator is right-associative.

**F[_]** reads as _"context F of underscore"_ or _"context F"_. Contexts are a type constructor[^hkt] that take types as an argument and produce another type. They become proper types when their _underscore_ is replaced by another type, as in **F[A]** or _"context F of A"_.

**`F[_]`** reads as _"context F of underscore"_ or _"context F"_. Contexts are a type constructor[^hkt] that take types as an argument and produce another type. They become proper types when their _underscore_ is replaced by another type, as in **`F[A]`** or _"context F of A"_.

**()** a pair of parentheses, reads as _"unit"_ and means _"nothing"_ or _"void"_. It is both a type and a solitary value.

**`()`** a pair of parentheses, reads as _"unit"_ and may be treated as _"nothing"_ or _"void"_. It is both a type and a solitary value.

**f: A => B** reads as _"f is A to B"_ or _"function f has type of A mapped to B"_.

**`f: A => B`** reads as _"f is A to B"_ or _"function f has type of A mapped to B"_.

**fa: F[A]** reads as _"fa is F of A"_ or _"variable fa has type of context F of A"_.

**`fa: F[A]`** reads as _"fa is F of A"_ or _"variable fa has type of context F of A"_.

**h = g ∘ f** reads as _"h is g after f"_ or _"h is defined as function g composed after function f"_. Composition is described in [Terminology](#terminology).

**`h = g ∘ f`** reads as _"h is g after f"_ or _"h is defined as function g composed after function f"_. Composition is described in [Terminology](#terminology).

### Terminology

@ -62,7 +68,7 @@ Where there is conceptual overlap with object oriented programming, I will lever

**Expressions** are values that are described by some type `A`.

**Functions** are a _special case_ of expressions that map some type `A` to some type `B`. They are described by `A => B`.

**Functions** are a _special case_ of expressions that map some type `A` to some type `B`. They are described by some function type `A => B`.

**Terms** are identifiers naming unitary or indivisible variables and types.

@ -76,13 +82,13 @@ Where there is conceptual overlap with object oriented programming, I will lever

**Contexts** describe circumstances within which their contents are found. They are noted using `F[_]` when their contents are unspecified, and `F[A]` when their contents are known to be of type `A`. They are more concretely defined in later sections.

**Lifting** describes injecting a term `A` into a context `F[_]` such that `lift: A => F[A]`.

**Lifting** describes injecting a term `A` into a context `F[_]` such that `lift: A => F[A]`.

A **Lifted** term or expression already has the form `F[A]`.

**Lowering** describes extracting a term `A` from a context `F[A]` such that `lower: F[A] => A`.

**Lowering** describes extracting a term `A` from a context `F[A]` such that `lower: F[A] => A`.

**Composition** describes chaining the output of a function `f: A => B` to the input of a function `g: B => C` such that a new function `h: A => C` may defined as `h = g ∘ f`.

**Composition** describes chaining the output of a function `f: A => B` to the input of a function `g: B => C` such that a new function `h: A => C` may defined as `h = g ∘ f`.

* This algebraic notation demonstrates how function `g` is applied _after_ function `f` is applied to the argument `x`:

@ -104,20 +110,20 @@ A **Lifted** term or expression already has the form `F[A]`.

Programming broadly consists of two categories of functions:

1. **Pure functions** which produce the same result for the same argument, for all arguments. They are _deterministic_.

2. **Impure functions** which produce different results for the same argument, for any argument. They are _nondeterministic_.

1. **Pure functions** which produce the same result for the same argument, for all arguments. They are **deterministic**.

2. **Impure functions** which produce different results for the same argument, for any argument. They are **nondeterministic**.

**Nondeterminism** arises from outputs dependent on factors other than input to a function. These factors are referred to as **side effects** as **implicit input**. In addition, functions may produce **side effects** as **implicit output**.

Nondeterminism arises from _outputs dependent on factors other than input_ to a function. These factors are referred to as **side effects** as **implicit input**. In addition, functions may produce side effects as **implicit output**.

Both categories of functions may produce their results in **unknown quantities** along any measurable dimension, such as presence, length, or validity of their result. These quantities require specific knowledge of a given input in order to be known with certainty in the result. Unknown quantities are nondeterministic in impure functions as they are particularly influenced by side effects.

Both categories of functions may produce their results in **unknown quantities** along any measurable dimension, such as presence, length, or validity of their result. _These quantities require specific knowledge of a given input in order to be known with certainty in the result._ Unknown quantities are nondeterministic in impure functions as they are particularly influenced by side effects.

### Manifesting complexity

#### Nondeterminism

Nondeterminism as a dependence on factors other than initial state and input arises when a function `f: A => B` maps to a different member of `B` for any number of times `f` has been applied to the same member of `A`. This means that `f` is influenced by **side effects** that occur independent of its signature.

Nondeterminism as a dependence on factors other than initial state and input arises when a function `f: A => B` maps to a different member of `B` for any number of times `f` has been applied to the same member of `A`. _This means that `f` is influenced by implicit input that occurs independent of its signature._

An extreme example of a nondeterministic function is the random number generator `rng: () => Int` as it maps the solitary unit value `()` to all members of type `Int`. This mapping is influenced by some side effect or _implicit input_ which is external to the function's signature of `() => Int`:

An extreme example of a nondeterministic function is the random number generator `rng: () => Int` as it maps the solitary unit value `()` to all members of type `Int`. This mapping is influenced by some side effect or _implicit input_ which is external to the function's signature of `() => Int`:

:::{.numberLines}

```scala

@ -140,44 +146,44 @@ Nondeterminism is significant in that operations may be unpredictable, and that

Unknown quantities along measurable dimensions arise in functions returning types such as lists, potentially `null` references, or validation results. These outputs have unknown length, presence, and validity respectively, and require specific handling for undesirable cases. _Even pure functions produce results having unknown quantities._

* A simple example is a function `toBits: Int => List[Boolean]` where the known quantity of `Boolean` bits returned requires specific knowledge of the input argument.

* A simple example is a function `toBits: Int => List[Boolean]` where the known quantity of `Boolean` bits returned requires specific knowledge of the input argument.

* Hashmap lookups may or may not return a value associated to a given key. Unless you have specific knowledge of the key used to lookup a value, you don't have any guarantee whether the value actually exists.

Both of these operations are pure functions and are deterministic, but their results are contextualized by length and presence. Any unknown quantity along some measurable dimension requires specific handling in code. This means that in addition to writing code that handles the desired case of an operation, code must be specifically written for each dimension that exhibits unknown quantities.

Both of these operations are pure functions and are deterministic, but their results are _contextualized_ by length and presence. Any unknown quantity along some measurable dimension requires specific handling in code. This means that in addition to writing code that handles a _desired output case_ of an operation, code must be specifically written for each dimension that exhibits unknown quantities.

Side effects as **implicit output** include **faults** such as the _divide by zero_ error and thrown exceptions. They impose an additional layer of protection to prevent or recover from them. Exceptions are fully nondeterministic as there is no single input that guarantees that an exception will never be thrown, as some side effect as an **implicit input** may influence the outcome.

Side effects as _implicit output_ include **faults** such as the _divide by zero_ error and thrown exceptions. They impose an additional layer of protection to prevent or recover from them. Exceptions are fully nondeterministic as there is no single input that guarantees that an exception will never be thrown, as some side effect as an implicit input may influence the outcome.

> In contrast with most faults, a _divide by zero_ error only occurs if the input divisor is `0`. The additional check for `0` that division sometimes requires is not considered complexity in practice.

>

> Running out of memory will throw an exception even in pure functions. Exceptions are truly nondeterministic and you must choose when and how to handle their cases. Hopefully you know ahead of time where you will need to do so.

In addition to being side effects, exceptions may be reasoned about as a _dimension of success or failure_ in an operation. This quantity is unknowable ahead of time and highly dependent upon implicit input.

In addition to being side effects as implicit output, exceptions may be reasoned about as a _dimension of success or failure_ in an operation. This quantity is unknowable ahead of time and highly dependent upon implicit input.

Concurrency and asynchronous operations are driven entirely by side effects. Asynchronous operations have an unknown temporal quantity that imposes costly specific handling, as execution must wait for operations to complete. Support for asynchronous operations requires runtimes to manage limited computing resources and scheduling of tasks, forming an entire system within a program.

Concurrency and asynchronous operations are driven entirely by side effects. Asynchronous operations have an _unknown temporal quantity_ that imposes costly specific handling, as execution must wait for operations to complete. Support for asynchronous operations requires runtimes to manage limited computing resources and scheduling of tasks, forming an entire system within a program.

#### Relating nondeterminism and unknown quantities

Side effects enable nondeterminism which influences unknown quantities in the results of operations. Undesired cases along dimensions such as length, presence, validity, success, and time require specific handling in addition to the code to handle the desired output case of operations. This specific handling creates complexity and draws engineering focus away from business logic. Yet side effects drive the business value of programs in the real world, which requires that we embrace nondeterminism and unknown quantities.

Side effects enable nondeterminism which influences unknown quantities in the results of operations. _Undesired output cases_ along dimensions such as length, presence, validity, success, and time require specific handling in addition to the code to handle the _desired output cases_ of operations. _This specific handling creates complexity and draws engineering focus away from business logic._ Yet side effects drive the business value of programs in the real world, which requires that we embrace nondeterminism and unknown quantities.

_How might complexity in programs be reduced if they must also be driven by side effects?_

### Implied complexity

Given a function `f: A => B` and another `g: B => C`: a third function `h: A => C` may be composed of `h = g ∘ f` or _h is g after f_. Programs may be modeled as a function `program: Input => Output`, where `program` is composed of innumerable smaller functions, together in concert building the necessary mappings to generate the desired program output. However composition is not so simple in programs as the results of some operations may not produce the desired input for subsequent operations.

Given a function `f: A => B` and another `g: B => C`: a third function `h: A => C` may be composed of `h = g ∘ f` or _h is g after f_. Programs may be modeled as a function `program: Input => Output`, where `program` is composed of innumerable smaller functions, together in concert building the necessary mappings to generate the desired program output. However composition is not so simple in programs as the results of some operations may not produce the desired input for subsequent operations.

Functions in real world programs must internally interact with implicit inputs and outputs _not present_ in the program's signature of `program: Input => Output`. An employee payroll system for example must make database queries and integrate with banks. These implicit inputs and outputs have **effects** which determine how their associated functions produce their desired outputs. For example, database queries return nondeterministic responses of unknown length and an error might occur when performing a direct deposit. _These effects determine how and whether payday is successfully produced._

Functions in real world programs must internally interact with implicit inputs and outputs _not present_ in the program's signature of `program: Input => Output`. An employee payroll system for example must make database queries and integrate with banks. These implicit inputs and outputs have **effects** which determine how their associated functions produce their desired outputs. For example, database queries return nondeterministic responses of _unknown length_ and _an error might occur_ when performing a direct deposit. _These effects determine how and whether payday is successfully produced._

Errors and unknown quantities as **effects** of these operations are opaque in functions modeled as simple input to output, as in `getEmployee: Int => Employee`. The signature of this function requires _[tacit knowledge][]_ of what effects may determine how an `Employee` is produced from it. For example:

Errors and unknown quantities as _effects_ of these operations are opaque in functions modeled as simple input to output, as in `getEmployee: Int => Employee`. The signature of this function requires _[tacit knowledge][]_ of what effects may determine how an `Employee` is produced from it. For example:

1. An associated `Employee` may not be found.

2. The returned `Employee` may change between applications of the same `Int` employee ID.

3. The database or network may fault and the function generates an exception that must be handled.

You might be thinking that these cases are a given when working with database code, but that knowledge only comes with experience. These cases are **effects** which describe the circumstances under which an `Employee` may be produced and can be modeled accordingly as part of the typed API of `getEmployee`. I will soon explain how this modeling works; first we will consider how to characterize complexity.

You might be thinking that these cases are a given when working with database code, but that knowledge only comes with experience. These cases are _effects_ which describe the circumstances under which an `Employee` may be produced and can be modeled accordingly as part of the typed API of `getEmployee`. Capturing these effects might look like `getEmployee: Int => Probably[Employee]`. I will soon explain how this modeling works; first we will consider how to characterize the complexity which defines effects.

### Operations producing undesired cases

Can you think of some program operations that produce undesired cases in addition to their desired output? How might these cases cause code to become complex?

Can you think of some program operations that produce undesired output cases in addition to their desired output cases? How might these cases cause code to become complex?

:::{.wide-list-items}

* When a program starts, it may **read configuration** from the environment, a database, or files. Reading configuration values may be blocking or asynchronous, and some configuration keys may not have associated values.

@ -194,7 +200,7 @@ Can you think of some program operations that produce undesired cases in additio

### A model for characterizing complexity

Complexities can be characterized in terms of **effects**. The operations listed above impose complexity because they feature the effects of:

Complexities can be characterized in terms of _effects_. The operations listed above impose complexity because they feature the effects of:

Presence

: Some configuration keys may not have an associated value.

@ -212,8 +218,8 @@ Success

: Some operations may be aborted.

IO

: Operations are dependent on external systems' state as **implicit input**

: Operations can affect external systems' state as **implicit output**.

: Operations are dependent on external systems' state as _implicit input_.

: Operations can affect external systems' state as _implicit output_.

: Operations may be concurrent, paused, or interrupted.

: Operations may block execution of the calling operation.

: Interacting with concurrency primitives may block execution and produce nondeterministic outputs.

@ -272,7 +278,7 @@ The code above demonstrates effects in the dimensions of:

* **Validity** where the `"SUCCESS"` response from the `achClient` is the only check performed. Other responses are simply not handled.

* **Success** where logging occurs but swallows errors, resulting in opaque `false` return cases. Exceptions also bubble up from database and network operations.

_These effects are not obvious from the code above_. I have rewritten it to handle all undesirable cases to highlight where complexity exists:

_These effects are not obvious from the code above_. Below I have rewritten the operation to handle all undesirable output cases to highlight where complexity exists:

:::{.numberLines}

```java

@ -326,13 +332,15 @@ class PayrollRunner {

There's a large amount of complexity in this code due to various effects. In order to make clear all effects and where they occur, I leveraged checked exceptions and all exceptions that are known to be thrown by other functions are translated into exceptions representing the domain of this operation. As all false cases effectively communicated no information I replaced them with exceptions typed according to the reason for failure. The operation now returns void as it is side-effecting and a specific return for success would be superfluous.

There are a number of checks along the dimension of presence. There's several along the dimension of success. Each operation is dependent upon the success of the operation preceding it, following a _procedurally validated_ imperative flow.

There are a number of checks along the _dimension of presence_. There's several along the _dimension of success_. Each operation is dependent upon the success of the operation preceding it, following a _procedurally validated_ imperative flow.

> I highlight _procedural validation_ as a concept to illustrate that the above operation advances by manually verifying success or failure of each operation in sequence. This kind of logic is error-prone and fragile, as dependencies between operations are non-obvious and changes to error handling independent of domain logic can easily introduce bugs.

There's a lot of effects just in this code. Fortunately there are ways to model sets of these effects and contain the scope of their impact so that code is less complex. Rewriting the above code using an effect model may look like this:

There's a lot of inner operations subject to effects within the main operation. Fortunately there are ways to model sets of these effects and contain the scope of their impact so that code is less complex. Rewriting the above code using an effect model might look like this:

This code looks similar, doesn't it? What hides between the lines here is a custom effect model `PayrollEffect` that abstracts away the effects of presence, async IO, success, and implicit input and output. This code is thus unburdened of most complexity, and makes the rest of it easier to work with.

This code looks similar, doesn't it? What hides between the lines here is a custom effect model `PayrollRunner` that abstracts away the effects of presence, async IO, success, and implicit input and output. This code is thus unburdened of most complexity, and makes the rest of it easier to work with.

You may notice that there are no return statements for error cases: the flow of execution through this code is not performed procedurally. Instead flow is controlled by declaring where errors occur and the main operation short-circuits itself should any inner operation fail.

You may notice that there are no return statements for error cases: the flow of execution through this code is not performed procedurally. Instead flow is controlled by declaring where errors occur and the main operation short-circuits itself should any inner operation fail. This decouples domain logic from validating imperative operations, reducing a significant source of bugs.

_This abstraction of effects allows for safer code that better focuses on the business logic at-hand._

@ -361,7 +369,9 @@ What is a **context**? _A context is a setting where stuff exists under some cir

> Contexts in Scala may be neatly represented by a letter and brackets such as `F[_]` read as _context F_ with an underscore when the type of the term is unspecified, and `F[A]` read as _F of A_ when the term is known to be of type `A`. Other letters work nicely of course, as do concrete names, as in `Option[Int]` or _Option of Int_.

Each kind of context models a set of **effects**. Contexts thus represent a concrete, typed API describing how their terms may be produced. Names of contexts can hint at the effects they model, and with some intuition you may be able to figure out what each context’s effects may be.

The letter `F` is a shorthand for _<span style="font-weight: bold; text-decoration: underline;">F</span>unctional Effect_, which is another term for context. Contexts each model a set of **effects** which represent concrete, typed APIs that describe how their terms may be produced. This mean that for any impure function `f: A => F[B]` you receive an output of type `B` whose production is contextualized by the effects of `F[_]`. A pure function `g: A => B` in comparison returns a fully-evaluated `B` absent of effects due to the application of function `g`.

Names of contexts can hint at the effects they model, and with some intuition you may be able to figure out what each context’s effects may be.

### Common contexts and some of their effects

@ -404,13 +414,31 @@ Each kind of context models a set of **effects**. Contexts thus represent a conc

Each of these contexts have two shared characteristics in that they _produce_ some term `A` and that their effects determine _how_ term `A` is produced. But with such a wide array of effects, and with so little overlap between each context, how can instances of term `A` be consumed in a manner unburdened of complexity?

In order to generalize contexts, the key differentiator between them must be abstracted: **effects**. By shedding effects as an _implementation detail_, the production of term `A` remains a shared characteristic. This opens an opportunity to create a **seam** between an impure function _producing_ the context itself and a pure function _consuming_ instances of the term `A` contained within.

In order to generalize contexts, the key differentiator between them must be abstracted: _effects_. By shedding effects as an _implementation detail_, the production of term `A` remains a shared characteristic. This opens an opportunity to create a **seam** between an impure function _producing_ the context itself and a pure function _consuming_ instances of the term `A` within the context.

The term _seam_ comes from Michael Feathers' [Working Effectively With Legacy Code][]: _"A place where you can alter behavior in your program without editing in that place."_ A seam is where we introduce an abstraction to separate the impure output of an operation from pure consumption of the desired output case. How functions handle an unknown length of items, for example, does not change and is independent of how functions consume each item. How functions consume each item varies on a case-by-case basis, and changes frequently according to requirements within business logic. This illustrates a need for a separation of concerns that can be enabled by abstraction.

Take for example the following JavaScript code:

```javascript

// traversing a length of items does not change

for (var i; i <businessCases.length;i++){

// businessLogic() changes frequently

businessLogic(businessCases[i]);

}

// asynchronous dispatch of API calls does not change

callApiAsynchronously("/get/stuff", function (stuff) {

// doThingsWithStuff() changes frequently

doThingsWithStuff(stuff);

});

```

_How do you create this seam?_

## A design pattern for contexts

**For any context `F[_]`, it produces some term `A`.** If you have a function `f : A => B`, how would you apply it to the term produced by the context `F[A]`? That would require _lowering_ the term. Specifically, you can’t apply the following function directly to the context:

**For any context `F[_]`, it produces some term `A`.** If you have a function `f: A => B`, how would you apply it to the term produced by the context `F[A]`? That would require _lowering_ the term. Specifically, you can’t apply the following function directly to the context:

:::{.numberLines}

```scala

@ -424,7 +452,7 @@ f(fa)

```

:::

Recall from the previous section, contexts share two characteristics: that they produce a term, and that they have effects which determine how the term is produced. After abstracting effects, contexts do not expose an obvious shared interface to extract the term. Consider the following definitions for `Option[A]`, `Either[X, A]`, and `List[A]`:

Recall from the previous section, contexts share two characteristics: that they _produce_ a term, and that they have _effects_ which determine how the term is produced. After abstracting effects, contexts do not expose an obvious shared interface to extract the term. Consider the following definitions for `Option[A]`, `Either[X, A]`, and `List[A]`:

**`Option[A]`** is the effect of presence, absence, or _optionality_ of an instance of term `A`:

@ -466,7 +494,7 @@ case object Nil extends List[Nothing]

### An object-oriented approach to contexts

Both `Option[A]` and `Either[X, A]` have roughly the same shape in that there either is or isn’t an instance of the desired term `A`. Because of this, a _lowering_ operation `extract: F[A] => A` is possible as it means the same thing between both of them: `extract` either gets the existing instance of the term `A` or it faults. In object oriented programming, `Option[A]` and `Either[X, A]` might expose such an interface:

Both `Option[A]` and `Either[X, A]` have roughly the same _structure_ in that there either is or isn’t an instance of the desired term `A`. Because of this, a _lowering_ operation `extract: F[A] => A` is possible as it means the same thing between both of them: `extract` either gets the existing instance of the term `A` or it faults. In object oriented programming, `Option[A]` and `Either[X, A]` might expose such an interface:

> [See here]({{code_repo}}/src/main/scala/green/thisfieldwas/embracingnondeterminism/extractables/one) for the definitions of `Option`, `Either`, and `List` using the single-value `Extractable` interface.

#### Generalizing for an unknown length of term `A`

How do you `extract` the term `A` from a `List[A]` such that it means the same thing as in `Option[A]` and `Either[X, A]`?

@ -495,39 +527,45 @@ As in `Option[A]` and `Either[X, A]` there is a notion of the presence or absenc

_This interface however is not coherent._ Faulting on absence is preserved as a behavior in `Option[A]` and `Either[X, A]`, but `Seq[A]` is allowed to be empty per its definition. Allowing `List[A]` to be empty implies that it should be allowed to return an empty `Seq[A]` from `extract`. In order to preserve faulting on absence, `extract` must return a `NonEmptyList[A]` instead, as this has the effect of always having _at least one_ instance of term `A`. You're still stuck with an unknown length of instances, though.

> [See here]({{code_repo}}/src/main/scala/green/thisfieldwas/embracingnondeterminism/extractables/seq) for the definitions of `Option`, `Either`, and `List` using the multi-value `Extractable` interface with `Seq`.

_This interface however is not coherent._ Faulting on absence is preserved as a behavior in `Option[A]` and `Either[X, A]`, but Scala's `Seq[A]` is allowed to be empty per its definition. Allowing `List[A]` to be empty implies that it should be allowed to return an empty `Seq[A]` from `extract`. In order to preserve faulting on absence, `extract` must return a [`NonEmptyList[A]` instead]({{code_repo}}/src/main/scala/green/thisfieldwas/embracingnondeterminism/extractables/nel), as this has the effect of always having _at least one_ instance of term `A`. You're still stuck with an unknown length of instances, though.

This interface essentially transforms these contexts into `NonEmptyList[A]` and imposes its specific complexity on all of your code. You would probably be very unhappy using it.

What about implementing `extract` for `Future[A]`? When applied to `Future[A]`, the `extract` function is by its own signature a blocking call. You want your dependency on `A` to be properly asynchronous.

What about implementing `extract` for `Future[A]`? When applied to `Future[A]`, the `extract` function by its own signature is a blocking call. You want your dependency on `A` to be properly asynchronous.

This interface does not generalize for more than the contexts of `Option` and `Either`. A better interface exists.

This interface does not generalize for more than the contexts of `Option` and `Either`, and it requires that they both subclass a shared trait. A better interface exists. It also doesn't force these types into the same hierarchy.

### Motivating functors as a design pattern

`Option[A]`, `Either[A]`, `List[A]`, `Future[A]`, and `IO[A]` each have different effects that determine how term `A` is produced. You must follow an axiom from object oriented programming: _abstract what changes_. Therefore you have to shed effects as an implementation detail. How might that impact lowering the term `A`?

`Option[A]`, `Either[X, A]`, `List[A]`, `Future[A]`, and `IO[A]` each have different effects that determine how term `A` is produced. You must follow an axiom from object oriented programming: _abstract what changes_. Therefore you have to shed effects as an implementation detail. How might that impact lowering the term `A`?

You may be unsatisfied by the answer: _extraction cannot be generalized_. All you know is that there is term `A`. You don't know whether an instance is present, how many of it there are, whether it's here already, or if it's arriving later. How do you consume term `A` when you know nothing about its instances' nature of existence? Functors solve this problem.

**Functors** are abstractions that allow you to consume term `A` within the context of `F[A]`. Functors are a class of types for which they have defined a single function `map: F[A] => (A => B) => F[B]`. Functors in Scala may be formally declared using the `Functor` typeclass:

**Functors** are abstractions that allow you to consume term `A` within the context of `F[A]`. Functors are a class of types for which they have defined a single function `map: F[A] => (A => B) => F[B]`. Functors in Scala may be formally declared using the `Functor` typeclass:

:::{.numberLines}

```scala

@ -540,13 +578,15 @@ object Functor {

```

:::

What `map` does is _lift_ the function `f : A => B` into the context so that it behaves as `F[A] => F[B]`, giving back `F[B]`.

> [See here]({{code_repo}}/src/main/scala/green/thisfieldwas/embracingnondeterminism/typeclasses/Functor.scala) for the accompanying code.

What `map` does is _lift_ the function `f: A => B` into the context so that it behaves as `F[A] => F[B]`, giving back `F[B]`.

This _lifting_ of functions that `map` performs is _coherent across contexts_. With `map` you can apply `f : A => B` to any `List[A]` just as you can any `IO[A]`. The results of both operations are predictable: your `List[A]` maps to `List[B]` and your `IO[A]` maps to `IO[B]`.

This _lifting_ of functions that `map` performs is _coherent across contexts_. With `map` you can apply `f: A => B` to any `List[A]` just as you can any `IO[A]`. The results of both operations are predictable: your `List[A]` maps to `List[B]` and your `IO[A]` maps to `IO[B]`.

How would you consume the term produced by `Future[A]` or `Option[A]`? You would also use a functor.

What this enables is your function `f: A => B` to be used with any functor regardless of its specific effects. Your function `f: A => B` is immediately reusable across all contexts and can be unit tested in isolation of effects.

What this enables is your function `f: A => B` to be used with any functor regardless of its specific effects. Your function `f: A => B` is immediately reusable across all contexts and can be unit tested in isolation of effects.

#### Why does the `map` function return `F[B]`?

@ -554,9 +594,9 @@ Recall that contexts generally do not permit extracting terms. Think for a momen

Extracting _the_ term from `List[A]` flatly doesn't make sense as it has the effect of an unknown number of instances.

Because there is no way to generalize extracting a term from a context, functors don’t allow you to operate on contexts in such a way that an instance of the term can "escape" them.

Because there is no way to generalize extracting a term from a context, functors don’t allow you to operate on contexts in such a way that an instance of the term can _escape_ them.

Most importantly, by keeping all operations against terms within their context, the context’s specific effects remain abstracted. Asynchronous operations with `Future[A]` remain asynchronous, the length of `List[A]` remains unknown, and `Option[A]` may or may not be present.

Most importantly, by keeping all operations against terms within their context, contexts' specific effects remain abstracted. Asynchronous operations with `Future[A]` remain asynchronous, the length of `List[A]` remains unknown, and `Option[A]` may or may not be present.

Functors thus _preserve structure_ by keeping operations within the context. For example, applying `map` on a `List[A]` or `BinaryTree[A]`.

@ -585,7 +625,7 @@ for (const x in list) {

```

:::

Iteration as a form of lowering _destroys structure_. In order to get a `List[B]` back you have to rebuild it yourself and any structural guarantees must be manually implemented following _procedural_ steps.

Iteration as a form of lowering _destroys structure_. In order to get a `List[B]` back you have to rebuild it yourself and any structural guarantees must be manually implemented following _procedural steps_.

This isn't to say that functional programming is only about iteration and loops versus `map`. Can you think of other operations that might destroy structure? For example, if you use an `await` operation on a `Future[A]` you will destroy its _asynchronous structure_ and potentially harm the performance of your program.

@ -594,24 +634,28 @@ This isn't to say that functional programming is only about iteration and loops

> :::{.numberLines}

> ```scala

> val myLookup = map.find("myKey").getOrElse("myDefault")

> val myMatchingLookup = map.find("myKey") match {

> case Some(myValue) => myValue

> case None => "myDefault"

> }

> ```

> :::

#### Context `F[A]` must produce some term `A`

Recall my statement from above: _"For any context `F[_]`, it produces some term `A`."_ If a context were guaranteed to have an instance of a term `A` then you should be able to consume it with your function `f : A => B`, right?

Recall from above: For any context `F[_]`, it produces some term `A`_. If a context were guaranteed to have an instance of a term `A` then you should be able to consume it with your function `f: A => B`, right?

But what if there’s nothing there, as in there are _zero_ instances of term `A`? Can you do anything? When a context has this kind of effect, a sort of "nothing here" or _void_, then the `map` function above doesn’t do anything because there isn’t anything to do. If you try to `map` a _void_`F[A]` with `f: A => B` then it returns a _void_`F[B]` as there’s "nothing here". _It does this without having used `f: A => B` to get there._

But what if there’s nothing there, as in there are _zero_ instances of term `A`? Can you do anything? When a context has this kind of effect, a sort of "nothing here" or _void_ effect, then the `map` function above doesn’t do anything because there isn’t anything to do. If you try to `map` a void `F[A]` with `f : A => B` then it returns a void `F[B]` as there’s "nothing here". _It does this without having used `f : A => B` to get there._

> The _void effect_ is a very abstract concept. Void in this context simply means that a particular context has a case where no instances of its term `A` exist, even though the context itself may contain other data.

This behavior is referred to as _short-circuiting_ and it is a key feature of contexts that encode some notion of void. It is exploited in particular to enable two key features of _imperative programming_, control flow and error handling, which I will expand on in later parts.

> `Option[A]` and `Either[X, A]` are two prime examples of short-circuiting in functors. An `Option[A]` will only `map` an instance of its term `A` if it is present, and an `Either[X, A]` will only `map` if an instance of the desired term `A` is present.

>

> In contrast, the `Id[A]` context has the effect of the _identity_ of term `A`. To put it plainly, `Id[A]`_is_ the instance of term `A`. As the instance is always present, this context never short-circuits.

The two contexts `Option[A]` and `Either[X, A]` demonstrate simple short-circuiting. An `Option[A]` will only `map` an instance of its term `A` if it is present as the `Some[A]` case, and an `Either[X, A]` will only `map` if an instance of the desired term `A` is present as the `Right[X, A]` case. In contrast, the `Id[A]` context has the effect of the _identity_ of term `A`. To put it plainly, `Id[A]`_is_ the instance of term `A`: as the instance is always present, the `Id[A]` context never short-circuits.

### Implementing functors in Scala

Each context of course must provide its own implementation of `map` in order for it to be used as a functor. Functor implementations in Scala are provided via typeclasses, and any type that has the shape`F[_]` may become a functor by implementing the `Functor` typeclass from above:

Each context must provide its own implementation of `map` in order for it to be used as a functor. Functor implementations in Scala are provided via typeclasses, and any type that has the _structure_ of`F[_]` may become a functor by implementing the `Functor` typeclass from above:

:::{.numberLines}

```scala

@ -624,7 +668,7 @@ object Functor {

```

:::

Instances of the `Functor` typeclass simply implement this trait and make themselves available implicitly:

Instances of the `Functor` typeclass implement this trait and make themselves available implicitly:

:::{.numberLines}

```scala

@ -654,7 +698,7 @@ object FunctorInstances {

```

:::

These three instances for `Option[_]`, `Either[X, _]`, and `List[_]` show remarkable similarities, and this isn’t uncommon across functors for most data structures. Note in particular how `List[_]` is recursive, with the base case `Nil` representing void. Implementations of `Functor` are more complex in contexts such as `IO[_]` and `Future[_]` because they are managing side effects. _What is key is that the complexities imposed by each of these contexts are completely abstracted_, allowing function `f: A => B` to operate unburdened by effects with a focus on specific program logic.

These three instances for `Option[_]`, `Either[X, _]`, and `List[_]` show remarkable similarities, and this isn’t uncommon across functors for most data structures. Note in particular how `List[_]` is recursive, with the base case `Nil` representing void. Implementations of `Functor` are more complex in contexts such as `IO[_]` and `Future[_]` because they are managing side effects. _What is key is that the complexities imposed by each of these contexts are completely abstracted_, allowing function `f: A => B` to operate unburdened by effects with a focus on specific program logic.

Can you see how functors enable control flow and short-circuiting? The void cases are the specific branches of logic that enable this. If there’s "nothing here", then they don’t do anything. In the specific case of `Either[X, _]`, `Left` may be used to carry some error state in its term `X`. This satisfies the effect of _either_`A` for success _or_`X` for failure.

@ -675,14 +719,27 @@ object FunctorInstances {

```

:::

To support an object-oriented API, the following `map: (A => B) => F[B]` extension method may be defined:

:::{.numberLines}

```scala

object FunctorSyntax {

implicit class FunctorOps[F[_], A](val fa: F[A]) extends AnyVal {

def [B](f: A => B)(implicit F: Functor[F]): F[B] = Functor[F].map(fa)(f)

}

}

```

:::

#### Using Scala functors as a general abstraction

Defining a `fizzBuzz : F[Int] => F[String]` function that uses a functor looks like this:

Defining a `fizzBuzz: F[Int] => F[String]` function that uses a functor looks like this:

@ -724,32 +781,32 @@ By using functors, the `fizzBuzz` function is free to focus on its specific prog

* Produce "fizz" when `x` is divisible by `3`

* Produce "buzz" when `x` is divisible by `5`

* Produce "fizzbuzz" when `x` is divisible by both `3` and `5`

* Produce `x` as a `String` otherwise

* Produce `x`printed as a `String` otherwise

At no point is `fizzBuzz` burdened by the effects of the context it executes against. Given a `Functor` instance for them, it's usable against `IO[Int]` and `Future[Int]` as well!

### Functors are universal

### Why do so many objects define `map`?

You might be thinking that lists and arrays in the wild already have a `map` operation available. `Promise`s in JavaScript also have their own `map` operation named [`then`][]. You've been using functors for a while and never realized!

You might be thinking that this functor pattern is superfluous, or even that the concept itself has dubious value. Lists and arrays in the wild already have a `map` operation available after all, and they don't necessarily require special knowledge in order to use them. Yet the pattern of a `map` operation exists in many places: `Promise`s in JavaScript for example have their own `map` operation named [`then`][].

Functors as a formal abstraction API, such as in the Scala `Functor` typeclass, find their strongest use in cases where the concrete type of the context is unimportant. However, you might observe that the _shape_ of functors appears in many places without being _called_ a functor. Using `map` on a list conceptually performs the same operation as on both arrays and `Promise`s. Other structures defining `map` operations may be functors because _functors arise from settings where stuff exists under some circumstances_.

But think for a moment: This means you've been using functors for a while and never realized!

#### Composition of functional effects

That so many functors appear in the wild is no coincidence. Functors appear anywhere stuff exists under some circumstances, and most frequently that is concretely realized in collections such as lists. The existence of `Promise` with its `next` operation in JavaScript demonstrates that the pattern appears in a very general manner. Functors as a formal abstraction API, such as in the `Functor` typeclass defined above, find their strongest use in cases where the concrete type of the context is unimportant. However, the _structure_ of functors appears in many places without being _called_ a functor, which implies a certain universal quality of functors as a pattern.

That so many functors appear in the wild is no coincidence. Functors even have a [formal definition][] within the higher math of [category theory][]. This definition can be applied to any structures that have the shape of a functor to assert that they behave as functors.

Curiously, this pattern manifests as a [formal definition][] within the higher math of [category theory][], which gives functors a particular property: they can be mathematically proven.

In order to be a functor, a context defining a `map`function must satisfy two laws:

In order to be a functor, a context defining a `map`operation must satisfy the two **functor laws**:

1. Preservation of identity functions:

```scala

context.map(id) == id(context)

context.map(identity) == identity(context)

```

2. Preservation of function composition:

```scala

context.map(g ∘ f) == context.map(f).map(g)

context.map(g compose f) == context.map(f).map(g)

```

Here are the two laws applied against Scala's builtin `List` type, which defines its own `map` operation:

These laws assert that functors preserve the behavior of functions `f` and `g` as if they were applied in sequence and also if they were composed independent of `map`. Functors thus _compose functional effects_ because this property of composition is retained within the context of their effects.

> See the [functor laws definition]({{code_repo}}/src/test/scala/green/thisfieldwas/embracingnondeterminism/typeclasses/FunctorLaws.scala) and its implementations for the [Option]({{code_repo}}/src/test/scala/green/thisfieldwas/embracingnondeterminism/effects/OptionSpec.scala), [Either]({{code_repo}}/src/test/scala/green/thisfieldwas/embracingnondeterminism/effects/EitherSpec.scala), and [List]({{code_repo}}/src/test/scala/green/thisfieldwas/embracingnondeterminism/effects/ListSpec.scala) functors.

Because of this rigorous definition, functors as a design pattern represent a concept that _transcends_ codebases and languages. In contrast, design patterns as they are realized in object-oriented programming form idioms that must be relearned between codebases written even in the same language.

These laws assert that functors preserve the behavior of functions `f` and `g` as if they were composed and also applied in sequence independent of `map`. Functors thus _compose functional effects_ because this property of composition is retained within the context of their effects. The seam introduced by `map` creates a hard delineation between any context's complexity of effects and the business logic of pure functions `f` and `g`.

Functors may be universally regarded as a _context of effects_. What this means is that ideally, and _provably_, `map` is the same regardless of specific context.

> Functors may be universally regarded as a _context of effects_. Because `map` is rigorously defined, the operation is ideally and _provably_ the same regardless of specific context. This means that functors as a design pattern represent a concept that _transcends_ codebases and languages. In contrast, design patterns as they are realized in object-oriented programming form idioms that must be relearned between codebases written even in the same language.

## Building upon functors

@ -795,7 +852,7 @@ This abstraction is enabling on its own, as it frees the logic in function `f` f

Consider for a moment: with a functor you are able to work against the _desired_ case of a result. You can chain any number of `map` operations against a functor, or `map` any number of composed functions against it. At the end, you still have a functor that is in the _desired_ case. In order to write _imperative_ code, you have to be able to force an _undesirable_ case so that subsequent operations are skipped.

Consider for a moment: with a functor you are able to work against a function's _desired_ output case. You can chain any number of `map` operations against a context, or `map` any number of composed functions against it. Given a context in the desired output case, after applying `map` you will still have a context that is in the desired output case. In order to write _imperative_ code, you have to be able to force an _undesirable_ output case so that subsequent operations are skipped.

Working against _two or more contexts at once_ opens opportunities to introduce _undesirable_ cases. Take for example these two instances of the context `F[_]` and the function signature for `combine`:

Working against _two or more contexts at once_ opens opportunities to introduce _undesirable_output cases. Take for example these two instances of the context `F[_]` and the function signature for `combine`:

:::{.numberLines}

```scala

@ -841,6 +898,7 @@ In my next post, we will explore how **applicatives** enable working within two