
Zod 4.1 introduced a new `z.codec()` API for defining bi-directional transformations in Zod.

## The problem with transforms

Zod's `.transform()` method is great for one-way data conversion:

```ts
const stringToNumber = z.string().transform(val => parseFloat(val));
stringToNumber.parse("42"); // 42
```

But what if you need to go both ways? Say, you're storing dates as ISO strings in a database but want to work with `Date` objects in your app.

```ts
const stringToDate = z.string().transform(str => new Date(str));
const dateToString = z.date().transform(date => date.toISOString());

// Two separate schemas, manually kept in sync
stringToDate.parse("2024-01-15T10:30:00.000Z"); // Date
dateToString.parse(new Date()); // "2024-01-15T10:30:00.000Z"
```

This works, but it's brittle. You need to keep track of two schemas and remember that they are intended as inverses. You need to manually verify that the output type of one matches the input type of the other. If you change one, you have to remember to update the other. 

## Introducing codecs

Codecs are a new Zod API for defining *bidirectional transformations* between two types. You specify an input schema, output schema, and transformation functions in both directions:

```ts
const stringToDate = z.codec(
  z.iso.datetime(),  // input schema: ISO string
  z.date(),          // output schema: Date object
  {
    decode: isoString => new Date(isoString), // string → Date
    encode: date => date.toISOString(),       // Date → string
  }
);
```

You can process data in both directions using the new top-level `.decode()` and `.encode()` methods:

```ts
stringToDate.decode("2024-01-15T10:30:00.000Z"); // Date
stringToDate.encode(new Date("2024-01-15"));     // "2024-01-15T00:00:00.000Z"
```

> **Note** — For bundle size reasons, these new methods have not added to Zod Mini schemas. Instead, this functionality is available via equivalent top-level functions. 
>
> ```ts
> // equivalent at runtime
> z.decode(stringToDate, "2024-01-15T10:30:00.000Z");
> z.encode(stringToDate, new Date());
> ```


This is particularly important when you are using Zod to *map data* back and forth between two different domains. One common use case is to convert data to/from a serializable format like JSON into a richer JavaScript representation (with `Date`, `bigint`, etc).

![Zod schemas shared](/codecs/codecs-network-dark.svg)

### Async 

The transformation functions can be `async`.

```ts
const asyncCodec = z.codec(z.string(), z.number(), {
  decode: async str => Number(str),
  encode: async num => num.toString(),
});
```

The usual "safe" and "async" variants exist:

```ts
syncCodec.encode("42");
syncCodec.safeEncode("42");
await asyncCodec.encodeAsync("42");
await asyncCodec.safeEncodeAsync("42");
```


### Composability

Codecs can be composed inside other schemas, just like any other schema. There are no special rules.

```ts
const queryParams = z.object({
  before: stringToDate,
  after: stringToDate
})

queryParams.encode({
  before: new Date(),
  after: new Date()
});
// => { before: string, after: string }
```



### `.parse()` vs `.decode()`

Let's compare the existing `.parse()` APIs to `.decode()`. _`.parse()` is equivalent to `.decode()` at runtime._

```ts
// equivalent at runtime
stringToDate.parse("2024-01-15T10:30:00.000Z"); 
stringToDate.decode("2024-01-15T10:30:00.000Z");
```

Though they're identical at runtime, their type signatures differ in an important way. While `.parse()` accepts `unknown`, `decode` expects a _strongly-typed inputs_.

```ts
stringToDate.parse(12345); 
// No TypeScript error but fails at runtime

stringToDate.decode(12345);
// ❌ TypeScript error: Argument of type 'number' is not assignable to parameter of type 'string'
```

Here's a diagram demonstrating the differences:

![Codec directionality diagram](/codecs/codecs-dark.png)

> This is a highly requested feature unto itself.
> 
> - [#3860](https://github.com/colinhacks/zod/issues/3860) Add strongly typed parse function
> - [#1748](https://github.com/colinhacks/zod/issues/1748) Typed input for parse methods
> - [#3978](https://github.com/colinhacks/zod/issues/3978) Type-safe parsing with known input types
> - [#1892](https://github.com/colinhacks/zod/issues/1892) Strongly typed decode function

## How encoding works

Most Zod schemas in the universe don't perform any kind of transformation. Their inferred input and output types are identical. For these schemas, there is no difference between parsing/decoding and encoding.

```ts
const mySchema = z.object({
  name: z.string()
});

// no difference
mySchema.parse({ name: "colinhacks" });
mySchema.decode({ name: "colinhacks" })
mySchema.encode({ name: "colinhacks" })
```

A small number of APIs cause the input and output types to diverge. In these scenarios, the runtime behavior of `.decode()`/`.encode()` also differ.

### Codecs

This is an obvious one. During `.decode()`, the `decode` function runs. During `.encode()`, the `encode` function runs. Simple.

### Transforms ⚠️

This is the #1 rule of `.encode()`: you can't use `.transform()`. That API is inherently unidirectional. If your schema contains any transforms, attempting an "encode" operation with it will throw a runtime error. You'll need to refactor to use `z.codec()`.

```ts
const schema = z.string().transform(val => val.length);

schema.encode(5); 
// ❌ ZodEncodeError: Encountered unidirectional transform during encode
```

### Pipes

> **Note** — Codecs are actually implemented as a subclass of `ZodPipe` augmented with "interstitial" transform logic.

Pipes reverse their order during encoding, from `A → B` to `B → A`. That said, pipes are typically used in conjunction with transforms, so "vanilla" pipes are rarely useful in the context of encoding. Prefer `z.codec()` everywhere.

### Refinements

All checks (`.refine()`, `.min()`, `.max()`, etc.) are still executed in both directions. 

```ts
const schema = stringToDate.refine((date) => date.getFullYear() > 2000, "Must be this millenium");

schema.encode(new Date("2000-01-01"));
// => Date

schema.encode(new Date("1999-01-01"));
// => ❌ ZodError: [
//   {
//     "code": "custom",
//     "path": [],
//     "message": "Must be this millenium"
//   }
// ]
```

To avoid unexpected errors in your custom `.refine()` logic, Zod performs two "passes" during `.encode()`. The first pass ensures the input type conforms to the expected type (no `invalid_type` errors). If that passes, Zod performs the second pass which executes the refinement logic.

This approach means all parsing & refinement logic runs in exactly the reverse order during encoding. Even "mutating refinements" like `z.string().trim()` or `z.string().toLowerCase()` work as expected. 

```ts
const schema = z.string().trim();

schema.decode("  hello  ");
// => "hello"

schema.encode("  hello  ");
// => "hello"
```

### Default/prefault

Default and prefault values are only applied in the forward direction. 

```ts
const withDefault = z.string().default("hello");

withDefault.decode(undefined); // "hello"
withDefault.encode(undefined); // ❌ ZodError
```

This is by design. When you add a default, the input becomes `string | undefined` but the output stays `string`. As such, `undefined` isn't considered a valid input to `.encode()`.

### Catch

Similarly, `.catch()` values are only applied in the forward direction.

### Stringbool

> **Note** — [Stringbool](https://zod.dev/api?id=stringbool) pre-dates the introduction of codecs in Zod. It has since been internally re-implemented as a codec. 

The `z.stringbool()` API converts string values (`"true"`, `"false"`, `"yes"`, `"no"`, etc.) into `boolean`. By default, it will convert `true` to `"true"` and `false` to `"false"` during `.encode()`..

```ts
const stringbool = z.stringbool();

stringbool.decode("true");  // => true
stringbool.decode("false"); // => false

stringbool.encode(true);    // => "true"
stringbool.encode(false);   // => "false"
```

If you specify a custom set of `truthy` and `falsy` values, the *first element in the array* will be used instead.

```ts
const stringbool = z.stringbool({ truthy: ["yes", "y"], falsy: ["no", "n"] });

stringbool.encode(true);    // => "yes"
stringbool.encode(false);   // => "no"
```


## Official codecs

Zod doesn't provide any predefined codecs out of the box. Instead, the docs provide some "canonical" codec implementations you can copy/paste into your projects as needed. These have all been tested internally. 

- [`stringToNumber`](https://zod.dev/codecs?id=stringtonumber)
- [`stringToInt`](https://zod.dev/codecs?id=stringtoint)
- [`stringToBigInt`](https://zod.dev/codecs?id=stringtobigint)
- [`numberToBigInt`](https://zod.dev/codecs?id=numbertobigint)
- [`isoDatetimeToDate`](https://zod.dev/codecs?id=isodatetimetodate)
- [`epochSecondsToDate`](https://zod.dev/codecs?id=epochsecondstodate)
- [`epochMillisToDate`](https://zod.dev/codecs?id=epochmillistodate)
- [`jsonCodec`](https://zod.dev/codecs?id=jsoncodec)
- [`utf8ToBytes`](https://zod.dev/codecs?id=utf8tobytes)
- [`bytesToUtf8`](https://zod.dev/codecs?id=bytestoutf8)
- [`base64ToBytes`](https://zod.dev/codecs?id=base64tobytes)
- [`base64urlToBytes`](https://zod.dev/codecs?id=base64urltobytes)
- [`hexToBytes`](https://zod.dev/codecs?id=hextobytes)
- [`stringToURL`](https://zod.dev/codecs?id=stringtourl)
- [`stringToHttpURL`](https://zod.dev/codecs?id=stringtohttpurl)
- [`uriComponent`](https://zod.dev/codecs?id=uricomponent)
- [`stringToBoolean`](https://zod.dev/codecs?id=stringtoboolean)

Some selected examples are below.

### `stringToBigInt`

```ts
const stringToBigInt = z.codec(z.string(), z.bigint(), {
  decode: str => BigInt(str),
  encode: bigint => bigint.toString(),
});

stringToBigInt.decode("12345");  // 12345n
stringToBigInt.encode(12345n);   // "12345"
```

### `jsonCodec`

```ts
const jsonCodec = z.codec(z.string(), z.json(), {
  decode: (jsonString, ctx) => {
    try {
      return JSON.parse(jsonString);
    } catch (err: any) {
      ctx.issues.push({
        code: "invalid_format",
        format: "json_string",
        input: jsonString,
        message: err.message,
      });
      return z.NEVER;
    }
  },
  encode: value => JSON.stringify(value),
});
```

You can pipe `jsonCodec` into other schemas for additional validation:

```ts
const UserFromJson = jsonCodec.pipe(z.object({ 
  name: z.string(), 
  age: z.number() 
}));

UserFromJson.decode('{"name":"Alice","age":30}');  // { name: "Alice", age: 30 }
UserFromJson.encode({ name: "Bob", age: 25 });     // '{"name":"Bob","age":25}'
```

### `base64ToBytes`

```ts
const base64ToBytes = z.codec(z.base64(), z.instanceof(Uint8Array), {
  decode: base64String => z.core.util.base64ToUint8Array(base64String),
  encode: bytes => z.core.util.uint8ArrayToBase64(bytes),
});

base64ToBytes.decode("SGVsbG8=");  // Uint8Array([72, 101, 108, 108, 111])
base64ToBytes.encode(bytes);       // "SGVsbG8="
```

---

For further reading, see the [Zod 4.1 release notes](https://github.com/colinhacks/zod/releases/tag/v4.1.0) and the [Codecs documentation page](https://zod.dev/codecs).

Introducing Zod Codecs

I propose the following convention for JavaScript libraries to make their AI resources auto-discoverable via package.json.

```json
{
  // package.json
  "llms": "https://docs.cool-library.com/llms.txt",
  "llmsFull": "https://docs.cool-library.com/llms-full.txt",
  "mcpServer": "https://mcp.cool-library.com",
}
```

I've implemented this in [Zod](https://github.com/colinhacks/zod/blob/main/packages/zod/package.json) and encourage my fellow library authors to do the same. Moreover, I encourage agentic IDEs/CLIs to implement first-party support for this convention. 

If you add support to your library or agent, please fill out [this form](https://docs.google.com/forms/d/e/1FAIpQLSew8sx6KxjwHkBGlgbUWHNEzSAt9GM8ryJ-X39d0gEltGT_-g/viewform?usp=header).

## Details

1. The `llms` field is a URL pointing to an [llms.txt](https://llmstxt.org/) file. If you aren't familiar, this is a proposed convention for documentation sites to provide an AI-friendly "sitemap" to AI agents. It's a simple spec: basically just a bunch of categorized Markdown links.
2. The `llmsFull` field is a URL pointing to an `llms-full.txt` file containing the full documentation for the library. Typically this is a concatenation of all the `.md`/`.mdx` documentation pages in the docs.
3. The `mcpServer` field is a URL pointing to an MCP server. For a typical library, this server would offer a simple Q&A tool that is capable of answering questions about the library's API/usage. This can use RAG over the library's docs, GitHub issues, Discord server, etc.

## Do agents already look at these fields?

No. This is a proposed convention. As more libraries implement this, there is more incentive for agents (IDEs, CLIs, etc) to implement first-party support. On the flip side of the coin, if any agentic CLI/IDE implements support, there is more incentive for libraries to implement this convention. There's a chicken-and-egg problem.

Fortunately, you can probably get your agent of choice to support this with some custom instructions in your `AGENTS.md` file.

```md

For each package in my package.json, read its package.json and look for the `llms`, `llmsFull`, and `mcpServer` fields. If found, use these resources to work with that library:

- `llms`: Use this URL to fetch the llms.txt file, which contains categorized Markdown links to the library's documentation. Use this as a sitemap to understand what documentation is available and how it's organized.

- `llmsFull`: Use this URL to fetch the llms-full.txt file, which contains the complete documentation for the library. Use this for comprehensive understanding of the library's API, features, and usage patterns.

- `mcpServer`: Connect to this MCP server URL to access specialized tools for answering questions about the library. Use this server's Q&A capabilities when I need specific help with the library's API, troubleshooting, or best practices. If possible, prompt the user to add this server to their MCP server registry.
```

## Why not `AGENTS.md`?

This doesn't solve the same problem.

`AGENTS.md` is for telling agents how to work *inside* your codebase. It's like a contributor's guide: how to run tests, run builds, lint, update configs, etc. It gives the agent best practices for modifying a codebase.

This is a qualitatively different problem that a library trying to advertise it's own functionality to an agent. The codebase is already installed in `node_modules`; agents shouldn't be modifying it. 

An agent doesn't care about the package's internals, it only cares about the public API. Like any developer, it doesn't want to read your code to learn how to do stuff—that's hard, time-consuming, and expensive. Having some well-known `package.json` fields that have an established semantic meaning is a cleaner solution with higher signal-to-noise ratio.

---

If you maintain a library that implements this convention, fill out [this form](https://docs.google.com/forms/d/e/1FAIpQLSew8sx6KxjwHkBGlgbUWHNEzSAt9GM8ryJ-X39d0gEltGT_-g/viewform?usp=header).

Making AI resources auto-discoverable via package.json


> EDIT: A previous version of this post recommended `publishConfig`, operating under the mistaken belief that it could be used to override `"exports"` during `npm publish`. As it turns out, `npm` only uses `"publishConfig"` to override certain `.npmrc` fields like `registry` and `tag`, whereas `pnpm` has expanded its use to override package metadata like `"main"`, `"types"`, and `"exports"`. There are a number of reasons you may not wish to strongly couple your deployment logic to `pnpm` (detailed in the `publishConfig` section below). My updated recommendation is to use a custom export condition plus `customConditions` in `tsconfig.json`.

> Jump into the code: https://github.com/colinhacks/live-typescript-monorepo

In development, your TypeScript code should feel "alive". When you update your code in one file, the effects of that change should propagate to all files that import it instantaneously, with no build step. This is true even for monorepos, where you may not be importing things from a file, but from a local package.

```diff
- import { Fish } from "../pkg-a/index";
+ import { Fish } from "pkg-a"
```

This is vital to TypeScript's value proposition.

This post explains a few strategies you can use to make your TypeScript monorepo feel more alive. Refer to the corresponding repo where you can play around with the code for each strategy: https://github.com/colinhacks/live-typescript-monorepo

The repo contains three subdirectories, each containing a monorepo with the following file structure:

```
.
├── package.json
├── packages
│   ├── pkg-a
│   │   ├── README.md
│   │   ├── index.ts
│   │   ├── package.json
│   │   └── tsconfig.json
│   └── pkg-b
│       ├── README.md
│       ├── index.ts
│       ├── package.json
│       └── tsconfig.json
├── pnpm-lock.yaml
├── pnpm-workspace.yaml
├── tsconfig.base.json
└── tsconfig.json
```

This is a pnpm monorepo (`pnpm-workspace.yaml`) with two packages, `pkg-a` and `pkg-b`. **`pkg-b` has a dependency on `pkg-a`**. Each package has a `tsconfig.json` that extends a `tsconfig.base.json` in the root of the monorepo.

Here is a quick rundown of the solutions. Don't worry if there are terms you're not familiar with, everything is explained in the breakdown.

1. Use project references (`"references"` in `tsconfig.json`)
2. Use `publishConfig` in `package.json` to specify `.ts` file in development and `.js` file in production. _Requires `pnpm`._
3. Configure `compilerOptions.paths` in `tsconfig.json` to override resolution for local package names.
4. **Recommended** Define a custom conditional export condition in `package.json#exports`.

Note that I'm explaining all the solutions I found for the sake of education, but the recommended solution is #5 (custom export conditions) so feel free to jump ahead if you're just looking for a solution!

## A primer: runtime vs. static

The fundamental annoyance here is this: Node.js has an algorithm for _module resolution_. When it sees an import from a _bare specifier_ like `"pkg-a"`, it scans up the directory tree checking for a directory called `"pkg-a"` in each `node_modules` folder it encounters. Once it finds `"pkg-a"`, it reads the `package.json` inside and uses the `main` and `exports` to figure out how to resolve the bare specifier to a file on disk.

The TypeScript server does _almost_ the same thing, though it uses the `"types"` field (or the `"types"` export condition in `"exports"`) to find the _declaration file_ corresponding to a particular package. There are also _lots_ of ways to hack TypeScript's module resolution algorithm. (We'll get to that in a bit.)

But in development, we want things to behave differently. We want the TypeScript server to look at our "raw" `.ts` files when resolving imports to other local packages in our monorepo, _not_ the compiled declaration files. Similarly, when we execute our local code (say, when running tests) we want it to "run" our TypeScript source code. You shouldn't need to rebuild your project before running tests.

So we need a way to hijack module resolution both statically (for TypeScript) and at runtime (for Node.js or tools like Vitest). We also need to make sure _both_ of these things are hijacked in a way that they agree with each other. If TypeScript is looking at our `src/index.ts` files but Node.js is still importing `lib/index.js`, the types may not reflect the runtime behavior of the code. You may have seen this referred to as _static-runtime disagreement_.

Okay, enough background. Let's get into the details.

## 1. Project references

> The code for this is under the `project-references` subdirectory.
> Project references are a TypeScript feature that make it easier to split a large TypeScript codebase into chunks that are typechecked separately by the TypeScript server. This can be a huge performance win for large codebases.

In our monorepo, `pkg-b` has a dependency on `pkg-a`. So in our `packages/pkg-b/tsconfig.json`, we can add a reference to `pkg-a` like so:

```json
{
  "references": [{"path": "../pkg-a"}]
}
```

Unfortunately the `"references"` field is [not inherited](https://twitter.com/andhaveaniceday/status/1798613232208527826) when you use `"extends"`, so you have to declare it in every package's `tsconfig.json`. If your monorepo packages have a lot of interconnection, this can get unwieldy fast.

You may also notice that `"references"` will end up [mirroring the `"dependencies"`](https://twitter.com/andhaveaniceday/status/1798717319990112609) list in `package.json`. These will need to be kept in sync to work as expected. There are tools ([`nx`](https://nx.dev/)) that try to automate this, but the need to run a command to re-generate configs starts to feel like a "build step" in itself...and that's what we're trying to avoid.

> There are [long-gestating efforts](https://github.com/microsoft/TypeScript/issues/25376) to make this more ergonomic, but there seems to be little movement on this front.

The final nail in the coffin is the difficulty of incorporating `"references"` into runtime module resolution. It's purely a TypeScript feature, and no tools incorporate this into their module resolution. So in all likelihood, you'd need to use one of the approaches below _in conjunction_ with project references to achieve true "live types" with runtime `.ts` resolution.

> There are absolutely scenarios where project references are indispensable. If you have a large enough monorepo, project references may become necessary to avoid re-typechecking the entire codebase when you make a change! But for non-giant projects, they aren't necessary and introduce too much complexity and potential footguns.

## 2. `"publishConfig"` in `package.json`

> This approach requires `pnpm` to work! The `publishConfig` field behaves very differently between `npm publish` and `pnpm publish`.

One obvious solution is just to have each package's `package.json` point to our `.ts` files in `package.json`.

```json
{
  "name": "pkg-a",
  "main": "./src/index.ts",
  "types": "./src/index.ts",
  "exports": {
    ".": {
      "import": "./src/index.ts",
      "require": "./src/index.ts",
      "types": "./src/index.ts"
    },
    "./package.json": "./package.json"
  }
}
```

This introduces an obvious and immediate problem. We can't publish our raw `.ts` files to npm without breaking most tools. They need to be properly transpiled to JavaScript first. When we run `npm publish`, we need these fields to point to the appropriate `lib/index.js` and `lib/index.d.ts` files.

Many people have written custom build scripts that will duplicate `package.json` and rewrite these fields before publishing. Fortunately the good folks at `pnpm` have given us a better option: `publishConfig`.

```json
{
  "name": "pkg-a",

  // development config
  "exports": "./src/index.ts",

  // production config
  "publishConfig": {
    "main": "./lib/index.js",
    "types": "./lib/index.d.ts",
    "exports": {
      ".": {
        "import": "./lib/index.js",
        "require": "./lib/index.js",
        "types": "./lib/index.d.ts"
      },
      "./package.json": "./package.json"
    }
  }
}
```

When you run `pnpm publish`, pnpm will read the `publishConfig` field in `package.json` and use those values to override the top-level values for `"main"`, `"exports"`, and `"types"`.

> While `npm` supports a field called `publishConfig`, it only lets you set `npm config` settings like `registry` and `tag`. Sad.

In essence, our top-level `exports`, `main`, `types`, etc. now _only apply in development_, so we can point these to raw `.ts` files! TypeScript will happily parse these files, as will any modern bundler, framework, or a tool like `tsx`.

For simplicity, I've only set one field: `exports`, and I'm setting it to a simple string value. This has some benefits:

1. It's clean! You don't need to redundantly declare a big `"exports"` object in both the top-level `package.json` and `"publishConfig"`. (Though you still can if you rely on subpath imports.)
2. It's safe! By specifying `"exports"` as a single string, no subpath imports are allowed. That level of strictness is good. It means you can't use subpath imports internally that would be illegal using the configuration specified in `publishConfig`.
3. It just works at runtime. Tools like `tsx`, `esbuild`, Vite, etc. can all happily resolve this import using just the single `exports` key. You don't need `"main"` unless you're using Node.js 10 or earlier in your development environment.
4. Similarly, TypeScript is happy. I lied a bit earlier...it doesn't need a special `"types"` field. It's more than happy to fall back to `"exports"` and pull type signatures out of a regular `.ts` source file.

The big downside is the reliance on `pnpm`. That means if you ever run `npm publish` by accident, you'll accidentally publish a package that isn't runnable by Node.js 💀 You also can't rely on popular GitHub Actions like `JS-DevTools/npm-publish` since those use `npm`. This obstacle is surmountable with some tweaks to CI and diligence around publishing, but it is an important gotcha.

## 3. `"paths"` in `tsconfig.json`

TypeScript provides another way to "hijack" module resolution: the `compilerOptions.paths`.

```json
{
  "compilerOptions": {
    "paths": {
      "pkg-a": ["./packages/pkg-a/src/index.ts"],
      "pkg-b": ["./packages/pkg-b/src/index.ts"]
    }
  }
}
```

The `compilerOptions.paths` option overrides TypeScript's normal module resolution. Any import that matches a key in paths will be resolved to the corresponding file. (If you specify multiple paths, TypeScript will use the first one that exists on your file system.)

This should be added to the `tsconfig.json` for every package in your monorepo. You can avoid redundancy by having all of your package `tsconfig`s extend a shared `tsconfig.base.json`.

Remember, we also need to incorporate this into our runtime module resolution.

The popular `tsx` tool by [`@privatenumbr`](https://twitter.com/privatenumbr) [does this automatically](https://tsx.is/faq#how-does-tsx-compare-to-ts-node). This is the best way to run a TypeScript file with Node.js.

```sh
$ npm install -g tsx
$ tsx src/index.ts
```

You can also use `tsx` in conjunction with Node.js's `--import` flag.

```sh
$ npm install tsx
$ node --import tsx src/index.ts
```

In the Vite/Vitest ecosystem, there is a popular plugin to do the same called [`vite-tsconfig-paths`](https://www.npmjs.com/package/vite-tsconfig-paths).

This solution can be a bit fiddly, and requires some diligence in how you configure per-package `tsconfigs`. It should also be noted that the TypeScript team [kinda hates](https://twitter.com/andhaveaniceday/status/1770466153615577306) `tsconfig.paths`, and there are a lot of ways to shoot yourself in the foot with it.

## 4. `liveDev` mode in `tshy`

The [`tshy`](https://github.com/isaacs/tshy) (**T**ype**S**cript **Hy**bridizer) tool is an opinionated tool by the creator of `npm` that makes it simple to build ESM and CommonJS packages from your TypeScript source code.

It recently added support for a `liveDev` mode that will hardlink your TypeScript source code into `./dist/esm` and `./dist/commonjs` directories. To set this up, install `tshy` into `devDependencies`.

```sh
$ pnpm add tshy --dev
```

Add the following `"tshy"` config to your `package.json`:

```json
{
  "tshy": {
    "liveDev": true
  }
}
```

Then run `tshy` in your package directory.

```sh
$ npx tshy
```

> For simplicity, add `"tshy"` as the `"build"` script in each of your workspaces. Then you can run this script in each workspace with one command from the root of your workspace. (The exact command depends on your package manager.)

This lets VS Code discover your live TypeScript source code without any additional `package.json` configuration! And tools like TypeScript and Vitest will be able to resolve your workspace imports to the `.ts` files with no additional configuration at all!

The downside is that this requires running `tshy` in each of your workspaces packages, which starts to feel like a build step.

The upside is that you only need to do this once! Once your `src` files are hard-linked into `dist`, you can edit them like normal and those changes are automatically reflected in `dist`. (This is how hard links work.)

> The downside is that you'll need to re-run `tshy` each time you add a new TypeScript file (due to how hard links work...). Running `tshy --watch` can mitigate the "new file" problem, but for the purposes of this repo, I'm avoiding any solutions that require a file system watcher.

## 5. Custom conditions in `"exports"`

This lets you specify your TypeScript files in `package.json#exports` under a custom _[export condition](https://nodejs.org/api/packages.html#conditional-exports)_ of your choosing.

The mostly widely-utilized export conditions are `import` (for specifying ESM code), `require` (for specifying CJS code), and `types` (for specifying type definitions).

```jsonc
// package.json
{
  "name": "pkg-a",
  "exports": {
    "*": {
      "import": "./lib/index.js",
      "require": "./lib/index.cjs"
    }
  }
}
```

But you're allowed to use any string you like as a custom export condition! Export conditions are intended as an open-ended mechanism for users to specify import entrypoints for specific runtimes, bundlers, or other tooling. For instance, `"deno"`, `"bun"`, and `"workerd"` (Cloudflare Workers) all support custom conditions so libraries can ship build that are specific to those runtimes.

Here's how a custom condition might look in your `package.json`. There's nothing special about the string `"@colinhacks/zod"` here! It could be anything.

```jsonc
// package.json
{
  "name": "pkg-a",
  "exports": {
    "*": {
      "import": {
        "@colinhacks/source": "./src/index.ts", // must be first, order matters!
        "default": "./lib/index.js",
        "types": "./lib/index.d.ts"
      },
      "require": {
        "@colinhacks/source": "./src/index.ts",
        "default": "./lib/index.cjs",
        "types": "./lib/index.d.ts"
      }
    }
  }
}
```

> You should put your custom condition _first_. Order matters! It's important that your custom condition is the first one in the list (even before `"types"`).

By default, neither TypeScript nor Node.js pays attention to this new `"@colinhacks/source"` export condition. You need to tell them to incorporate it into their respective module resolution algorithms.

To tell TypeScript, add `"@colinhacks/source"` to the `customConditions` field in `tsconfig.json` for all packages in your monorepo. (You can avoid redundancy by having all of your package `tsconfig`s extend a shared `tsconfig.base.json`.)

```json
{
  "compilerOptions": {
    // include this in the tsconfig for all packages
    "customConditions": ["@colinhacks/source"]
  }
}
```

Node.js can accept custom conditions via the `--conditions` flag.

```sh
node --import=tsx --conditions=@colinhacks/source ./src/index.ts
```

In the Vite/Vitest ecosystem, this can be configured with `resolve.conditions` setting.

```js
// vite.config.json
export default {
  resolve: {
    conditions: ['@colinhacks/source'],
  },
};
```

### A note on condition naming

I chose `"@colinhacks/source"` as the condition name for the sake of uniqueness. You want to pick a condition name that will only be defined in your workspace packages _only_. If you choose something generic like `"source"`, you may have dependencies with the same condition defined in their `package.json`. In this case, VS Code will resolve imports of that package using its `"source"` files, instead of using it's pre-compiled `.d.ts` files from `"types"`. This can hurt performance of the TypeScript server and slow down your editor.

## Conclusion

Overall, my recommendation is #5: custom export conditions.

- It's clean, easy to configure, and works well with modern tooling.
- It doesn't require you to use `pnpm`, which is a non-starter for many projects.
- You don't need to worry about keeping runtime and TypeScript configurations in sync: you just set `"exports"` once using your custom condition, then tell your other tools (including TypeScript itself) to pay attention to that condition.

As usual the "comments section" for this post is on Twitter. Feel free to make comments or ask questions in the replies to this tweet:

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">new blog post 👇 it breaks down a few approaches to configuring &quot;live types&quot; in TypeScript monorepos. you should never need to run `build` while developing!<br><br>1. tsconfig paths<br>2. custom export conditions<br>3. publishConfig (*my recommended solution)<a href="https://t.co/sf6F3CfcsQ">https://t.co/sf6F3CfcsQ</a></p>&mdash; Colin McDonnell (@colinhacks) <a href="https://twitter.com/colinhacks/status/1796595123595378822?ref_src=twsrc%5Etfw">May 31, 2024</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

Happy monorepo hacking!


Live types in a TypeScript monorepo


I maintain Zod, which is a schema library. Zod lets people declare an email schema like this:

```ts
import {z} from 'zod';

z.string().email();
```

A lot of people think every _technically valid_ email address should pass validation by that schema. I [used to think that](https://github.com/colinhacks/zod/issues/639#issuecomment-917223045) too.

Over the years I've merged several PRs to make the email regex more "technically correct". Most recently, I merged a PR that adds support for _IPv6 addresses_ as the domain part of an email. They look like this and I hate them:

```
jonny@[ipv6:7e95:0559:10f2:21e9:9dab:7309:c116:ca3b]
```

Turns out that PR also broke plain old subdomains: `user@subdomain.domain.com`. That means Zod currently fails to parse my _mom's current email address_ but Jonny up there can parse his freakish IPv6 email.

This caused a spiritual crisis and made me re-evaluate my whole stance on what `z.string().email()` should do. Zod's users are mostly engineers who are building apps. When you're building an app, you want to make sure your users are providing normal-ass email addresses. So that's what `z.string().email()` is going do.

So I rewrote the regex from scratch to be simple and reasonable. Here's what I came up with:

```ts
/^(?!\.)(?!.*\.\.)([a-z0-9_'+\-\.]*)[a-z0-9_+\-]@([a-z0-9][a-z0-9\-]*\.)+[a-z]{2,}$/i;
```

Let's break that down in plain English:

**The username** (AKA "local part")

- can use letters, numbers, and these special characters: `_'+-.`
- can't have two dots in a row
- can't start or end with a dot

**The domain**

- must consist of at least 2 dot-separated segments consisting of letters, numbers, and hyphens
- can't have two dots in a row
- must end with a "TLD" that's 2+ letters

It is _not_ trying to be [RFC 5322](https://datatracker.ietf.org/doc/html/rfc5322) compliant. It's not going to check if the TLD is real. And it's not going to implement any of this craziness:

- No "printable" characters: `wtf-!#$%&'*/=?^_{|}~@mail.com`
- No quoted local parts: `"zod is cool"@mail.com`
- No comments: `regexiscool(kinda)@mail.com`
- No IPv4: `billie@[1.2.3.4]`
- No IPv6: `jonny@[ipv6:7e95:0559:10f2:21e9:9dab:7309:c116:ca3b]`
- No emoji: `👎@mail.com`

But for reasonable people, it'll do.


An email regex for reasonable people

> Spoiler alert: GitHub Pages and GitHub actions are not involved.

I recently set out to build a proper documentation site for [Zod](https://github.com/colinhacks/zod) with a short and eminently reasonable list of goals:

1. The site should be generated from the README. Zod's `README.md` is the "source of truth".
2. The site should automatically update whenever the `master` branch is updated.
3. I didn't want to worry about building or maintaining a complex build workflow in GitHub Actions, if possible.



Despite the eminent reasonability, I had a hard time finding a simple solution. After a lot of research and false starts, I've found what I think is the lowest-effort, highest-reward way to publish a documentation site for your GitHub-hosted project.

Let's jump in.

## 1. Add this `index.html` to your repo

Create a file called `index.html` in your project root and paste the following contents into it.

> Replace all occurrences of `user/repo` with your project's name. And maybe write up a slightly more detailed `meta` description :)

```
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>user/repo</title>
    <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
    <meta name="description" content="user/repo is cool!" />
    <meta name="viewport" content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0" />
    <link rel="stylesheet" href="//cdn.jsdelivr.net/npm/docsify/lib/themes/vue.css" />
    <style>
      .markdown-section { max-width: 700px; }
    </style>
  </head>
  <body>
    <nav
      style="display: flex; flex-direction: row; align-items: center; justify-content: space-between;"
    >
      
    </nav>
    <div id="app"></div>
    <script>
      window.$docsify = {
        subMaxLevel: 1,
        maxLevel: 3,
        auto2top: true,
        repo: 'user/repo',
        routerMode: 'history',
        // homepage: "README.md"
      };
    </script>
    <script src="//cdn.jsdelivr.net/npm/docsify@4.12.2/lib/docsify.min.js"></script>
    <script src="//cdn.jsdelivr.net/npm/prismjs@1.28.0/prism.min.js"></script>
  </body>
</html>
```

Those 1150 bytes are doing a lot, but most importantly it's loading [Docsify](https://docsify.js.org/). Docsify is a documentation generator. You may recognize it from its goofy looking gradient cover pages with big pill-shaped buttons.

![Docsify sample homepage](/docsify.png)

> I'm not a fan of these cover pages, so we won't be using one here. All in all, though, I think Docsify is extraordinarily well designed.

It also differs from some other options like Docusaurus and Gitbook in that it's a _fully client-side_ documentation generator. Your Markdown files don't get pre-compiled to HTML ahead of time; instead, Docsify fetches it asynchronously, parses the contents in-browser, and injects the rendered HTML into the page.

_"The horror! Multiple round-trip requests before the first meaningful paint!?"_ Turns out, it's not a big issue. Modern globe-spanning ultra-fast CDNs for static content, this still feels near-instant. Google agrees; the [Zod site](https://zod.dev) gets a 98 Performance score on Lighthouse.

![98 performance on Lighthouse](/lighthouse.png)

## 2. Run locally

There's no build step required with Docsify. You could deploy your `index.html` and a `README.md` to any static site hosting service and it would work. We're going to be using [Netlify](https://www.netlify.com/) to deploy this site, but before we get to that, let's see how the site is looking. We can use Netlify's CLI to start a simple static site server.

```bash
$ npx netlify dev
◈ Netlify Dev ◈
   ┌─────────────────────────────────────────────────┐
   │                                                 │
   │   ◈ Server now ready on http://localhost:8888   │
   │                                                 │
   └─────────────────────────────────────────────────┘
```

Open `http://localhost:8888` in your browser, and you should see a rendered version of your `README.md`!

> If `netlify dev` detects that you're using a framework, this may not work. In that case you'll need to disable framework detection and tell Netlify to act as a simple static file host.
>
> ```sh
> npx netlify dev --framework "#static"
> ```

> If your primary Markdown file isn't called `README.md`, you can specify a different name with the `homepage` setting in the Docsify configuration.
>
> ```js
> window.$docsify = {
>   subMaxLevel: 1,
>   maxLevel: 3,
>   auto2top: true,
>   repo: 'user/repo',
>   routerMode: 'history',
>   homepage: 'home.md', // <-- add this
> };
> ```

## 3. Configure rewrites

Docsify using client-side routing to determine which Markdown file to load and render. In other words, it's a pure single-page application; `index.html` loads Docsify, which then assumes control over your browsers URL bar, similar to React Router or other client-side routers.

If your project/site contains several Markdown files, you may encounter a strange behavior. Consider the following file structure:

```
├── index.html
├── README.md
└── CONTRIBUTING.md
```

If you start the development server and type `localhost:8888/CONTRIBUTING` into the URL bar, you'll see a 404. That's because there's no HTML file called `CONTRIBUTING.html`. We need all incoming requests to render `index.html`! Then Docsify will read the URL and load the appropriate Markdown file.

We can achieve this with a Netlify feature called [rewrites](https://docs.netlify.com/routing/redirects/). (All static file hosts have an equivalent feature.) Create a file called `_redirects` with no extension. (With Netlify, both redirects and rewrites are declared in `_redirects`.)

```
$ touch _redirects
```

Then paste in the following:

```
/*            /             200
```

This tells Netlify that all incoming requests (`/*`) should render the homepage (`/`). However, unlike a _redirect_, rewrites do not change the URL. Basically Netlify will sneakily render `index.html` under the hood, without telling the browser.

> Another solution is to use a _hash router_. In fact, this is the default behavior for Docsify. Unfortunately, it makes the URL messy; the URL would look like `domain.com/#/CONTRIBUTING` instead of simply `domain.com/CONTRIBUTING`. I prefer clean URLs so I chose the rewrites approach.

Now when we run `npx netlify dev`, we can open `localhost:8888/CONTRIBUTING` directly without seeing a 404.

## 4. Deployment

Firebase used to be my go-to for static file hosting, but these days Netlify leads the pack on developer experience. It only takes a couple minutes to set up Github-linked static site hosting.

First, let's push our new files to `master`.

```sh
$ git add .
$ git commit -am "Add docs site"
$ git push origin master
```

Then configure Netlify.

1. Log in or create an account at [netlify.com](https://www.netlify.com/).
2. From the dashboard, click `Add new site > Import an existing project`.
3. Authenticate with GitHub (or GitLab or Bitbucket) and select your repository.

When prompted for a `"Branch to deploy"` select `master` (the default) unless your primary branch has a different name.

Under "Build settings" you can simply leave everything blank:

- `"Base directory"` — leave blank (defaults to the project root)
- `"Build command"` — leave blank (no build step necessary!)
- `"Publish directory"` — leave blank (defaults to the project root)

![Netlify configuration settings](/netlify_build.png)

There is virtually no configuration required, because we're simply hosting static files from the root directory of our `master` branch. Couldn't be easier!

> Note that we're deploying the entire contents of our repository to Netlify. You could write a build command to [delete all unwanted files](https://answers.netlify.com/t/can-i-exclude-certain-files-like-readme-md/15416) but I didn't bother. The contents of my repo are already public on the internet, so...🤷‍♂️

Proceed with the deployment. Once Netlify finishes deploying, Netlify will provide a `*.netlify.app` subdomain for your site.

## 5. Final touches

There's plenty more to do before your site is truly ready.

- Connect a custom domain
- Add a `robots.txt` to your root directory
- Add links to the top `<nav>` in `index.html`
- Add [Open Graph](https://ahrefs.com/blog/open-graph-meta-tags/) meta tags
- Add [Twitter](https://developer.twitter.com/en/docs/twitter-for-websites/cards/overview/markup) meta tags and a banner image
- Generate a favicon ([RealFaviconGenerator](https://realfavicongenerator.net/) is great)

But I'll leave that as an exercise for the reader.

From README to documentation site in 10 minutes

In the mid-2010s, the pendulum of web development swung massively in the direction of increased type safety.

First, we all switched back to relational DBs after recovering from an embarrassingly long collective delusion that NoSQL was cool. Then TypeScript started its meteoric rise after years of quiet development. Shortly thereafter GraphQL was announced and quickly became the de facto way to implement typesafe APIs.

> TLDR: I published a new library called tRPC that makes it easy to build end-to-end typesafe APIs without code generation, by leveraging the power of modern TypeScript. To jump straight to the tRPC walkthrough, [click here](#trpc).

### GraphQL's TypeScript problem

GraphQL's biggest competitive advantage is that it's a _spec_, not a library. It's designed to be universal and language-agnostic. If you use different languages for your client and server, then by all means stick with GraphQL! It remains the best language-agnostic way to build typesafe data layers.

But the vast majority of projects that use GraphQL are JavaScript-based. The `graphql` NPM module has [5M downloads/week](https://www.npmjs.com/package/graphql) vs [~300k](https://pypistats.org/packages/graphene)) for the `graphene` Python module and [~230k](https://bestgems.org/gems/graphql) for the `graphql` Rubygem. It's not a perfect metric, but the signal is clear.

And a growing fraction of JavsScript projects are TypeScript-based. As TypeScript approaches near-universal adoption in the JS ecosystem, that means **most** GraphQL-based projects will soon be TypeScript-based (if they aren't already).

Here's the thing: most people use GraphQL as a massively over-engineered way to share the **type signature** of their API with your client. What if you could import that type signature _directly into your client_ using plain ole `import` syntax? Well...you can.

## A demonstrative example

We're going to build the world's simplest typesafe API—no codegen or GraphQL required. There's only one endpoint, with the following type signature:

`getUser(id: string) => { id: string, name: string }`

For the sake of simplicity, the API is implemented as an Express router. It should be easy to tell what's going on even if you aren't familiar with it.

### #1 Define `getUser` function

```ts
// server/routes.ts
export const getUser = async (id: string) => {
  return { id, name: 'Bilbo' };
};
```

### #2 Implement a simple RPC endpoint

We set up a special `/rpc` endpoint that accepts POST requests with a payload of type `{ endpoint: string; arguments: any[] }`. The `endpoint` property indicates the function to call (in our case, the only option is `"getUser"`). We take the `arguments` array, pass it into the appropriate function, and `res.send` the results.

```ts
// server/router.ts
import express from 'express';
import * as routes from './routes';

type Payload = { endpoint: string; arguments: any[] };

const app = express();
app.post('/rpc', async (req, res) => {
  const payload: Payload = req.body;
  if (payload.endpoint === 'getUser') {
    const user = await routes.getUser(...payload.args);
    return res.status(200).send(user);
  }

  return res.status(404).send('Endpoint not found');
});
```

> **A totally skippable aside about RPC**

> With a REST APIs, you use different URLs (`/users`, `/user/:id`) and HTTP request types (e.g GET, POST, etc) to access different functionality. For instance `GET /users` will fetch all users, `GET /user/abc123` will fetch a single user, and `POST /user` will create a new user.

> RPC strips all of that away. Endpoints are just named functions that accept some arguments and return a value. The RPC ethos is to make API calls look and feel like calling a function (albeit one that's defined on a remote server).

> GraphQL is an RPC protocol that's been augmented with a schema language and a mechanism for client-side field selection.

### #3 Import the type signature of `getUser` into the client

This is where the magic happens. We **directly import** the type signature of our API into out client. Yes, it really is this easy.

```tsx
// client/index.tsx
import type { getUser } from '../server/routes.ts';
```

Note that this works no matter where `routes.ts` lives in your file system! It could be in a different Git repo; it could be on your external hard drive. As long as the TypeScript file exists on your computer, you can import it from any other TypeScript file.

**_"But wait, I can't just import stuff from a different repo!"_**

This is often true...when you're importing code. For instance, Create React App (CRA) prohibits you from importing anything from outside of your `src` directory.

But that doesn't apply here. We're using `import type` syntax, which is a **purely static construct**. That is, it can only import _type signatures_, but not data, code, or any runtime construct. For most build systems powered by Babel (including CRA), `import type` statements (and all type annotations) just get stripped out before the bundling step.
If you're using Webpack with Babel, Parcel, or Rollup, this Just Works™. Don't believe me? Play around with this [sample repo](https://github.com/colinhacks/import-type-cra) where you can see this approach working just fine in an unejected CRA project.

You can use `import type` to import a type from any `.ts` or `.tsx` file anywhere on your filesystem. Period. This will work even if your client and server are in different repositories in different parts of your filesystem.

> Of course, if multiple developers are working on the same codebase, you'll need to standardize the relative locations of client and server repos, so relative imports work on everyone's machines. You can either establish a convention ("make sure the `server` and `client` repos are in the same folder") or use something like Git Submodules.



**_"But wait, isn't importing stuff from the server into the client a security vulnerability?!"_**

Yes—when you're importing code. But not here! Remember, we're using the `import type` syntax. It was introduced in early 2020 ([TypeScript 3.8](https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-8.html#type-only-imports-and-export)) and it comes with a very important guarantee:

![import type never leaves remnants at runtime](/import-type.png)

All `import type` declarations are removed **completely** when you compile your TypeScript code into a JavaScript bundle, no matter what. So _yes_ — you can safely use `import type` without worrying about accidentally importing sensitive data from your server.



### #4 Make an API call

Now that we've imported the type signature of `getUser`, how do we make strongly typed API call? Answer: a bit of TypeScript magic. (Don't worry, you don't need to understand how this works! Just know that it's possible!)

```tsx
// client/index.tsx
import type { getUser } from '../server/routes.ts';

type API = {
  getUser: getUser;
  // add additional routes here...
};

const query = <Endpoint extends keyof API>(
  endpoint: Endpoint,
  ...args: Parameters<API[Endpoint]>
): ReturnType<API[Endpoint]> => {
  return fetch('http://localhost:3000/api/rpc', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ endpoint, arguments: args }),
  }).then((response) => response.json()) as any;
};
```

We now have a **fully typesafe** `query` function, which you use like this:

```ts
const user = await query('getUser', 'user_abc73bd39ef');
// => Promise<{ id: "user_abc73bd39ef", name: "Bilbo" }>
```

It's a function called `query`. The first argument is the _name_ of the
endpoint; the second is the input of the function. (As implemented each
RPC endpoint can only accept a single input.)

TypeScript validates that the endpoint name is valid...

![invalid endpoint name error](/trpc-invalid-endpoint.png)

...typechecks the arguments...

![invalid arguments](/trpc-invalid-arg.png)

...and strongly types the result.

![typed payload](/trpc-typed-payload.png)

And if you update the definition of `getUser` on your server, the new type signature **automatically** propagates to your client at the speed of TypeScript, no code generation required. The developer experience is unreal.

By using a next-generation ORM to fetch data in your endpoints, you get most of the value of GraphQL — end-to-end typesafety, easy nested fetching, field selection, etc — in a TypeScript-native way.







If you want, you can use this approach exactly as described; all the code above is available as a Next.js project in [this repo](https://github.com/colinhacks/typesafe-api-no-graphql). But there's a better way!

<a name="trpc"></a>

## Introducing tRPC!

Around 6 months ago, I distilled these ideas into a proof-of-concept library called tRPC. Since then, the inimitable [@alexdotjs](https://twitter.com/alexdotjs) has turned it into a full-fledged library. Lets see how to implement our `getUser` example with tRPC. Spoiler alert: it's mindblowingly easy.

> If you want to jump straight to the README, check it out at [github.com/trpc/trpc](github.com/trpc/trpc) — and leave a star if you're feeling generous!

### Create a router

```ts
// server/index.ts
import * as trpc from '@trpc/server';

const appRouter = trpc.router();
export type AppRouter = typeof appRouter;
```

### Add a query endpoint

Let's implement `getUser`.

```ts
// server/index.ts
import * as trpc from '@trpc/server';

export const appRouter = trpc.router().query('getUser', {
  input: String,
  async resolve(req) {
    req.input; // string
    return { id: req.input, name: 'Bilbo' };
  },
});
```

A few things to note:

- You add "endpoints" by calling the `.query` method. The first argument is a _procedure name_. The second is an object containing two properties: `input` and `resolve`.

- The `input` is a function that **validates the input**. For simplicity, all tRPC procedures accept a _single argument_. The type signature of this argument is inferred from the return type of the `input` function.

  I recommend defining your inputs using a schema definition library (I hear [Zod](https://github.com/colinhacks/zod) is pretty cool 🙃). Far too few developers rigorously validate incoming API payloads; this is a huge security risk and source of bugs. So tRPC has validation baked-in from the start.

  For simplicity, I'm just using the built-in JavaScript `String` constructor, which accepts any value and converts it into a string.

- The endpoint logic is defined in the `resolve` function. A "pseudorequest" is passed into it, which contains `input` (automatically typed as `string` in this case) and `ctx` (more on that later). tRPC **infers the return type** of your resolver. So make sure the returned value is statically typed! We recommend using a typesafe ORM like Prisma or TypeORM that automatically type the results of database operations; otherwise you'll need to write TypeScript types manually.

### What about mutations?

Like GraphQL, tRPC makes a distinction between queries and mutations. Add mutations to your router with the `mutation` method:

```ts
// server/index.ts
import * as trpc from '@trpc/server';
import { z } from 'zod';
export const appRouter = trpc
  .router()
  .query('getUser', {
    input: String,
    async resolve(req) {
      req.input; // string
      return { id: req.input, name: 'Bilbo' };
    },
  })
  .mutation('createUser', {
    input: z.object({ name: z.string() }),
    async resolve(req) {
      return await prisma.user.create({
        data: req.input,
      });
    },
  });
```

The syntax for the `mutation` is **identical** to `query`. tRPC only makes a distinction because `query` payloads are cached differently than `mutation` payloads.

### Chainable methods

Important note: when you add multiple endpoints to a router, you **must chain the calls** to `query()` and `mutation()`! This lets tRPC infer the type signature of your whole API. If you're familiar with Express you may be tempted to do this:

```ts
export const appRouter = trpc.router();

// ❌ DON'T DO THIS!! ❌

appRouter.query('getUser', {
  input: String,
  async resolve(req) {
    req.input; // string
    return { id: req.input, name: 'Bilbo' };
  },
});
```

This doesn't work at all. All router methods are `immutable`—they return an entirely new instance of `TRPCRouter`, instead of _modifying_ the current instance. So you _must chain_ these methods.

### Merging routers

At some point, you'll want to split up your tRPC router into multiple files. If all methods must be chained together, how is this possible?

tRPC provides an easy way to **merge routers**. I recommend creating a root `appRouter` and several "child routers" that you merge into it.

```ts
import { userRouter } from './routers/user';
import { postRouter } from './routers/post';

export const appRouter = trpc
  .createRouter()
  .merge('users.', userRouter) // prefix userRouter endpoints with "users."
  .merge('posts.', postRouter); // prefix postRouter endpoints with "posts."
```

All procedure names will be prefixed with the string literal you pass as the first argument of `.merge`. So a `createUser` mutation on `userRouter` will be re-named to `users.createUser`. This is made possible by the incredible _string literal templates_ feature introduced in [TypeScript 4.1](https://devblogs.microsoft.com/typescript/announcing-typescript-4-1/#template-literal-types)!

### Handing incoming requests

Now lets use our router to handle some actual HTTP requests. tRPC ships "adapters" that easily let you generate Express middleware or Next.js API routes from your tRPC router.

#### Next.js integration

```ts
// pages/api/trpc/[trpc].ts
import { createNextApiHandler } from '@trpc/server/adapters/next';

const appRouter = /* ... */;

export default createNextApiHandler({
  router: appRouter,
  createContext: (opts) => {
    return { bearer: opts.req.headers.authorization };
  },
});
```

#### Express integration

If you're using Express, do this instead:

```ts
import { createExpressMiddleware } from '@trpc/server/adapters/express';

const appRouter = /* ... */;

const app = express();

app.use(
  '/trpc',
  createExpressMiddleware({
    router: AppRouter,
    createContext: () => null, // no context
  })
);

app.listen(80);
```

#### Koa, Hapi, Nest.js

Coming soon (and PRs welcome)!

### Request context

All server adapters accept `createContext` function, where you can generate a `ctx` object from the "raw" HTTP request and response objects (available as `opts.req` and `opts.res`). This `ctx` is available in all your resolvers.

### Client-side usage

So what about the client side? The `@trpc/client` module generates a typesafe "tRPC client"—analagous to the `query` method we implemented in our toy example. You can import the the type signature of your **entire API** at once with `import type { AppRouter } from '...'`. Then pass it into `createTRPCClient`, along with the URL of your API:

```ts
// pages/index.ts
import { createTRPCClient } from '@trpc/client';
import type { AppRouter } from './api/[trpc]';

const trpcClient = createTRPCClient<AppRouter>({
  url: 'http://localhost:3000/api',
});

trpcClient.query('getUser', 'user_1238823498');
// => Promise<{ id: string; name: string }>
```

#### React integration

If you use React, you can use the `@trpc/react` package to convert your tRPC client into React hooks powered by [React Query](https://react-query.tanstack.com/). This gets you a ton of cool features like request invalidation, SSR, and cursor-based pagination! 🚀

```ts
import { createReactQueryHooks } from '@trpc/react';

// path to your backend server where your AppRouter is defined
import type { AppRouter } from '../server/trpc';

export const trpc = createReactQueryHooks<AppRouter>();
```

#### Vue and Svelte

Get in touch if you'd like to build dedicated bindings for those frameworks! Create an issue on [the GitHub repo](https://github.com/trpc/trpc).

## Wrapping up

I've been using various iterations of tRPC in production for nearly a year, and it truly feels like living in the future. I'm immensely proud of what tRPC has become. Checkout the docs (and leave a star! 🤩) on [the GitHub repo](https://github.com/trpc/trpc).

There are a lot of other features I could highlight, but that'll do for now. Huge shoutout to [@alexdotjs](https://twitter.com/alexdotjs), who picked up a fledgling proof-of-concept four months ago and carried it to the finish line!

Building an end-to-end typesafe API — without GraphQL

Zod's v2 has been in beta since September 2020 (around 3 months as of this writing). The people are starting to talk.



## The background

The reason Zod 2 has been in beta for so long (well, one of the reasons anyway!) is that I’ve been increasingly unsatisfied with Zod’s implementation of transformers. Transformers are a mechanism within Zod to convert data from one type to another. For the sake of rigor, I decided it was paramount for transformers to encapsulate both pre- and post-transform validation; to create one, you gave it an input schema `A`, and output schema `B`, and a transformation function `(arg: A)=>B`.

Multiple issues have cropped up that have led me to re-evaluate the current approach. At best, transformers are a huge footgun and at worst they’re fundamentally flawed.





## Context

In version 1, a Zod schema simply let you define a data type you'd like to validate. Under the hood, it magically inferred the static TypeScript type for your schema. More technically, there is a base class (`ZodType`) that tracked the inferred type in a generic parameter called `Type`.

![Screen Shot 2020-12-08 at 5 09 36 PM](https://user-images.githubusercontent.com/3084745/101593584-03bc6f00-39a5-11eb-8a66-5e7e84103290.png)

Transformers break this assumption. The accept an `Input` of one type and return an `Output` of a different type. To account for this, every Zod schema now had to track both an input and output type. For non-transformers, `Input` and `Output` are the same. For transformers only, these types are different.

Let's look at `stringToNumber` as an example:

```ts
const stringToNumber = z.transformer(z.string(), z.number(), (val) =>
  parseFloat(val)
);
```

For `stringToNumber`, Input is `string` and Output is `number`. Makes sense.

What happens when you pass a value into `stringToNumber.parse`?

1. The user passes a value into `stringToNumber.parse`
2. The transformer passes this value through the `parse` function of its _input schema_ (z.string()). If there are parsing errors, it throws a ZodError
3. The transformer takes the output from that and passes it into the transformation function (`val => parseFloat(val)`)
4. The transformer takes the output of the transformation and validates it against the output schema (z.number())
5. The result of that call is returned

Here's the takeaway: for a generic transformer `z.transformer(A, B, func)`, where A and B are Zod schemas, the argument of `func` should be the `Output` type of A and the return type is the `Input` type of B. This lets you do things like this:

```ts
const stringToNumber = z.transformer(z.string(), z.number(), (val) =>
  parseFloat(val)
);
const numberToBoolean = z.transformer(
  z.number(),
  z.boolean(),
  (val) => val > 25
);
const stringToNumberToBoolean = z.transformer(
  stringToNumber,
  numberToBoolean,
  (val) => 5 * val
);
```

## The problems with transformers

After implementing transformers, I realized transformers could be used to implement another much-requested feature: default values. Consider this:

```ts
const stringWithDefault = z.transformer(
  z.string().optional(),
  z.string(),
  (val) => val || 'trout'
);
stringWithDefault.parse('marlin'); // => "marlin"
stringWithDefault.parse(undefined); // => "trout"
```

Voila, a schema that accepts `string | undefined` and returns `string`, substituting the default "trout" if the input is ever undefined.

So I implemented the `.default(val:T)` method in the ZodType base class as below (partially simplified)

```ts
default(def: Input) {
  return ZodTransformer.create(this.optional(), this, (x: any) => {
    return x === undefined ? def : x;
  });
}
```

Do you see the problem with that? I didn’t. Neither did anyone who read through the [Transformers RFC](https://github.com/colinhacks/zod/issues/100) which I left open for comment for a couple months before starting on implementation.

This implementation quickly gets wonky when you try to pass _an existing transformer_ in as the input or output. In other words, transformers aren't composable. Let's see what happens if we try to set a default value on `stringToNumber`.

```ts
stringToNumber.default('3.14');

// EQUIVALENT TO
const defaultedStringToNumber = z.transformer(
  stringToNumber.optional(),
  stringToNumber,
  (val) => (val !== undefined ? val : '3.14')
);

defaultedStringToNumber.parse('5');
/* { ZodError: [
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "number",
    "path": [],
    "message": "Expected string, received number"
  }
] */
```

Let’s walk through why this fails. The input ("5") is first passed into the transformer input (`stringToNumber.optional()`). This converts the string `"5"` to the number `5`. This is then passed into the transformation function. But wait: `val` is now `number | undefined`, but the transformer function needs to return a `string`. Otherwise, if we pass `5` into `stringToNumber.parse` it’ll throw. So we need to convert `5` back to `"5"`. That may seem easy in this toy example but it’s not possible in the general case. Zod can’t know how to magically undo the transformation function.

In practice, the current definition of `default` in ZodType shouldn’t have even been possible. The only reason the type checker didn’t catch this bug is because there are a regrettable number of `any`s floating around in Zod. It’s not a simple matter to switch them all to `unknown`s either; I’ve had to use `any` in several instance to get type inference and certain generic methods to work properly. I’ve tried multiple times to reduce the number of `any`s but I’ve never managed to crack it.

It’s possible this is a one-off issue. I could find some other way to implement `.default()` that doesn’t involve transformers. Unfortunately this isn’t even the only problem in Zod’s implementation.

## The `.transform` method

Initially the only way to define transformers was with `z.transformer(A, B, func)`. Eventually I implemented a convenience method you can use like this:

```ts
z.string().transform(z.number(), (val) => parseFloat(val));

// equivalent to
z.transformer(z.string(), z.number(), (val) => parseFloat(val));
```

Some users were executing multiple transforms in sequence without changing the actual data type:

```ts
z.string()
  .transform(z.string(), (val) => val.toLowerCase())
  .transform(z.string(), (val) => val.trim())
  .transform(z.string(), (val) => val.replace(' ', '_'));
```

To reduce the boilerplate here, it was recommended that I overload the method definition to support this syntax:

```ts
z.string()
  .transform((val) => val.toLowerCase())
  .transform((val) => val.trim())
  .transform((val) => val.replace(' ', '_'));
```

If the first argument is a function instead of a Zod schema, Zod should assume that the transformation doesn’t transform the type. In other words, `z.string().transform((val) => val.trim())` should be equivalent to `z.string().transform(z.string(), (val) => val.trim())`. Makes sense.

Consider using this method on a transformer:

```ts
stringToNumber.transform(/* transformation_func */);
```

What type signature do you expect for `transformation_func`?

Most would probably expect `(arg: number)=>number`. Some would expect `(arg: string)=>string`. Neither of those are right; it’s `(arg: number)=>string`. Yes really. The transformation function expects an input of `number` (the Output of `stringToNumber`) and a return type of `number` (the Input of `stringToNumber`). This type signature is a natural consequence of a series of logical design decisions, but the end result is dumb. Well done, self.

Intuitively, you should be able to append `.transform(val => val)` to any schema. Surely passing in the identity function to a method called `.transform()` should be a no-op? Unfortunately due to how transformers are implemented, that's not always possible.

### More complicated examples

These strange issues composability issues also make it difficult to write any generic functions on top of Zod (of which `.transform` and `.default` are two examples). Others have encountered similar issues, like [here](https://github.com/colinhacks/zod/issues/199) and [here](https://github.com/colinhacks/zod/issues/213). For the sake of simplicity, I won't break down the nuances of those cases; the underlying issue is the composability problem I described above.

## When should defaults even be applied?

Strangely, there doesn't appear to be a consensus expectation on whether default values should be "applied" pre- or post-transformations. Consider the simple `stringToNumber` example; should `stringToNumber.default(/* value */)` accept a number or a string?

As implemented in Zod 2 it should accept `string` (the `Input` of the output schema of `stringToNumber`). In this case, the schema looks at the value passed into parse, checks if it's `undefined`, and, if so, hot-swaps in the default value. That value is then passed through all the transformers and refinements downstream (in this case, being converted to a `number`) by the non-optional `stringToNumber`. Remember, as implemented, this is how `stringToNumber.default()` returns:

```ts
stringToNumber.default(val);

// equivalent to

z.transformer(stringToNumber.optional(), stringToNumber, (val) =>
  val !== undefined ? val : '3.14'
);
```

But some users expect `stringToNumber.default` to accept a number; they assume that a default value "short-circuits" the parsing logic. If you pass `undefined` into `.parse()`, the parsing should "return early" and return the value. In this case, taht value should be a `number` because that's the ultimate expected return type of the transformer returned by `stringToNumber.default(val)`.

## A path forward

> Note from March 2021: the approach described below also had issues! Ultimately the implementation used in Zod 3 was slightly different.

I just created [an RFC](https://github.com/colinhacks/zod/issues/264) describing a plan to improve this state of affairs. The idea was to have each Zod schema contain a list of post-parse transformation functions. When a value is passed into `.parse`, Zod will type check the value, then pass it through the transform chain. This is the approach Yup uses.

```ts
const schema = z
  .string()
  .transform((val) => val.length)
  .transform((val) => val > 100);

type In = z.input<typeof schema>; // string
type Out = z.input<typeof schema>; // boolean
```

Unlike before, Zod doesn’t validate the data type between each transform. We’re relying on the TypeScript engine to infer the correct type based on the function definitions. In this sense, Zod is behaving just like I intended; it’s acting as a reliable source of type safety that lets you confidently implement the rest of your application logic — including transforms. Re-validating the type between each transform is overkill; TypeScript’s type checker already does that.

Each schema will still have an Input (the inferred type of the schema) and an Output (the output type of the last transform in the chain). But because we’re avoiding the weird hierarchy of ZodTransformers everything behaves in a much more intuitive way.

One interesting ramification is that you could interleave transforms and refinements. Zod could keep track of the order each mod/refinement was added and execute them all in sequence:

```ts
const schema = z
  .string()
  .transform((val) => parseFloat(val))
  .refine((val) => val > 25, { message: 'Too short' })
  .transform((val) => `${val}`)
  .refine((val) => /^\d+$/.test(val), { message: 'No floats allowed' });
```

We'll have to wait and see how this turns out!

**Update from March 2021**

This is the general approach being used in the new Zod 3 alpha release. There are some small implementational details — refinements and transforms are now "housed" within the ZodEffects wrapper class instead of being attached directly to the schema.

A note on default values: Zod no longer uses transforms to implement default values. Transforms are only capable of implementing _post-parse defaults_: parse the input, then swap in the default value if the result is `undefined`. But I've decided that _pre-parse defaults_ make more sense (and are more expected by users). That is — if the input is `undefined` swap in the default value, and _then_ finish the parsing process (which includes refining and transforming). As such, the defaulting logic now lives in the ZodOptional class.

Why Zod 2 isn't leaving beta

> **Edit: 2022** The issues described here have been largely solved by [layouts](https://nextjs.org/blog/layouts-rfc) in Next.js 13.

I recently set out to implement a single-page application (SPA) on top of Next.js.

To clarify, I'm using the term "SPA" in a very specific way. I'm referring to the "classical" SPA model: an application that handles _all_ data fetching, rendering, and routing entirely client-side.

Turns out, Next.js does _not_ make this easy.

> To actually learn _how_ to build a single-page application on top of Next.js, checkout my tutorial! [Building a single-page application with Next.js](https://colinhacks.com/essays/building-a-spa-with-nextjs).

### Why not use Next's router

Next's built-in router isn't as flexible as React Router! React Router lets you nest routers hierarchically in a flexible way. It's easy for any "parent" router to share data and provide an outer layout to its "child" routes. This is true for both top-level routes (e.g. `/about` and `/team`) and nested routes (e.g. `/settings/team` and `/settings/user`).

This isn't possible with Next's built-in router! Instead, _all_ shared state and layout must be initialized in your custom `_app.tsx` component. It's equivalent to building an app with a single, top-level React Router. Nominally, Next.js lets you define "nested routes" (e.g. `pages/settings/team.tsx`), but in practice it "flattens" all those routes. The only code that's shared is the stuff you can fit into `_app.tsx`.

There are some established patterns for circumventing this limitation, notably those described in [this post](https://adamwathan.me/2019/10/17/persistent-layout-patterns-in-nextjs/) by Adam Wathan. If you are dead-set on using Next's routing system, you should look at those approaches. But in my opinion they're agregiously hacky.

### Why is this so hard?

I finally got React Router to work inside Next.js — check out [my tutorial](https://colinhacks.com/essays/building-a-spa-with-nextjs) for details — but there were some thorny issues along the way. The guidance in the documentation is...scant, to put it mildly. It boils down to "use a custom App".

![Next.js Use A Custom App](/nextjs_custom_app.png)

That's it. Not particularly enlightening. Or detailed.

I found this to be pretty strange. Aren't SPAs still the dominant model for building React-based applications? Didn't Next.js just run a wildly successful conference with 60,000 virtual attendees? Didn't they just raise a \$21M Series A? Am I the only person struggling with this? Why is this so hard?

My answer: Vercel doesn't really want you to build SPAs, at least not with client-side routing. Theyr're a server hosting company, so they want you to use servers. To that end, I believe they consciously emphasize the server-side rendering (SSR) approach and de-emphasize the SPA approach.

### What about SSG?

An astute reader may ask "Why would Next.js try to de-popularize SPAs but encourage static site generation (SSG)?" Like SPAs, statically generated sites don't involve servers and thus doesn't make Vercel much money. Good point. This is where my theory veers into conspiracy and madness.

I suspect SSG is a strategic marketing play. A huge fraction of developers will build a static website at some point in their career. Having built their personal website with Next.js, they're more likely to choose Next.js down the road when designing their shiny new SaaS app. And since it's against Vercel's fair use policy to run a commercial enterprise on the free tier, all these new companies will eventually convert to a paid plan. They're playing the long game.

I don't think the SSG functionality in Next.js was created solely for marketing reasons; that's ludicrous. I think Next.js is the best way to build static site; I wrote a [1500-word article](https://colinhacks.com/essays/devii) to that effect just 5 months ago. In fact, I believe Vercel is laser-focused on improving the developer experience, perhaps more than any other company. That's exactly why I don't believe the lack of documentation on client-side routing is merely an oversight. The strength of the rest of their documentation makes this hole even more glaring.

None of this is _bad_, per se. Vercel and Next.js originated, in part, as a reaction to the SPA-centricity of React development pre-2016. They're under no obligation to document, advertise, or encourage the usage of SPA paradigm on their platform, especially if it works against their own interests. And all told, their free tier is extremely generous.

But remember that, at the end of the day, Vercel is a hosting company with a vested financial interest in pushing server-side rendering and de-popularizing SPAs. If someone tried to learn full-stack web development by reading the Next.js documentation, they wouldn't even know that SPAs are an option. So keep that in mind as you design your next project!

---

> Addendum: The section below was added shortly after initial publication.

In response to this article the lead maintainer of Next.js, Tim Neutkens, provided this counterpoint on Twitter:

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">All our marketing and communication around this have been around making sure you don’t use SSR if you don’t need it, e.g. by adding revalidate (incremental generation) and fallback (on-demand generation)</p>&mdash; Tim (@timneutkens) <a href="https://twitter.com/timneutkens/status/1321906247722033157?ref_src=twsrc%5Etfw">October 29, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

This actually sheds light on another interesting aspect of Vercel's SSG push.

Recently Next.js has introduced support for "dynamic static generation" with the `fallback` flag and incremental static regeneration with the `revalidate` flag. Tim presented these features as evidence that Next.js is helping developers avoid server-side rendering if it isn't absolutely necessary. But using either of these features requires Vercel to spin up a serverless function for your app. In other words, Next is blurring the line between SSG and SSR.

If you rely on these features, you've left the "pure" SSG paradigm. Your app now requires a server to work as expected — which means Vercel stands to make money! You can no longer simply `next build && next export` your app and deploy to your static hosting service of choice. Moreover, if your project is commercial in nature, you now must upgrade to a Pro plan (if hosting on Vercel).

---

## Wrap up

If you want to actually learn _how_ to build a single-page application on top of Next.js, checkout my tutorial [Building a single-page application with Next.js
](https://colinhacks.com/essays/building-a-spa-with-nextjs).

Hopefully I didn't come off as too critical of Vercel. Next.js is phenomenal and I love it. If I didn't I would never have gone through all this trouble!

The "comments section" for this post is this Twitter thread. Jump in and yell at me for all the bad takes in this post!

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">✨ NEW BLOG POST ✨<br><br>It started as a simple &quot;How to build a SPA with Next.js&quot; tutorial.<br><br>It turned into a fevered rant about Vercel, Next.js, perverse incentives, and the future of the SPA paradigm. Oops.<a href="https://t.co/hrRtKojezH">https://t.co/hrRtKojezH</a></p>&mdash; Colin McDonnell (@colinhacks) <a href="https://twitter.com/colinhacks/status/1321871817095684097?ref_src=twsrc%5Etfw">October 29, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

If you're into Next.js or TypeScript, follow me [on Twitter @colinhacks](https://twitter.com/colinhacks)! I build and maintain open-source tools like [Zod](https://github.com/colinhacks/zod) (a TypeScript validation library) and [tRPC](https://github.com/colinhacks/trpc) (a tool for building end-to-end typesafe APIs with TypeScript). Or subscribe to the newsletter to get notified when I publish new posts!

<br/>



<br/>



<br/>

Why it's so hard to build a classic single-page application with client-side routing on top of Next.js

Next.js, nested routes, and the war on SPAs

> **Update 2023** — Updated for Next.js 13, React Router 6, and React 18.

> **Update 2022** — Next.js 13 now supports nested routes via the `/app` directory. However, it requires a server to render React Server Components, and thus doesn't qualify as a "single-page application" as defined in this post.

I recently set out to implement a single-page application (SPA) on top of Next.js.

To clarify, I'm using the term "SPA" in a very specific way. I'm referring to the "old school" SPA model: an application that handles _all_ data fetching, rendering, and routing entirely client-side.

Turns out, Next.js does _not_ make this easy.

## Why not use Next.js routing?

Next.js is not as flexible as React Router! React Router lets you nest routers hierarchically in a flexible way. It's easy for any "parent" router to share data with all of its "child" routes. This is true for both top-level routes (e.g. `/about` and `/team`) and nested routes (e.g. `/settings/team` and `/settings/user`).

This isn't possible with Next's built-in router! Instead, _all_ shared state and layout must be initialized in your custom `_app.tsx` component. It's equivalent to building an app with a single, top-level React Router. This is true despite the fact that Next.js nominally lets you define "nested routes" (e.g. `pages/settings/team.tsx`). In practice it "flattens" all those routes.

There are complex, hacky ways of achieving this in Next, including several described [here](https://adamwathan.me/2019/10/17/persistent-layout-patterns-in-nextjs/) by Adam Wathan. If you are dead-set on using Next's routing system, you should look at those approaches.



## Why use Next.js at all?

Because it's awesome! You can split your bundle with dynamic imports, which solves one of the biggest performance issues associated with large SPAs. You can easily deploy to Vercel. You can build your API using API Routes, instead of maintaining separate client and server codebases. Plus there's that snazzy new `<Image>` component!

But the killer feature of Next.js for me is the zero-config client-server codesharing.

The ability to share code and typings between my client and server is a _massive win_. The developer experience is 10x better than Lerna and 100x better than Yarn Workspaces (trust me). It's the enabling feature behind my new library [github.com/colinhacks/trpc](https://github.com/colinhacks/trpc), which lets you build an end-to-end typesafe data layer. Better yet, it abstracts away the API entirely and lets you call your server-side functions directly from your client. I'm biased, but it's extremely cool. Check it out at [github.com/colinhacks/trpc](https://github.com/colinhacks/trpc)! ✨



## Implementation

For highly interactive "dashboard-style" SaaS development, the dominant paradigm is still the SPA. Unfortunately Next.js is (nearly) incompatible with the SPA paradigm for one fundamental reason: routing. Next.js ships with its own built-in page-based routing system, whereas the typical SPA relies on client-side routing, typically using a library like [react-router](https://reactrouter.com/).

Below I will walk through all the errors I encountered while trying to use React Router inside a Next app and how I fixed them. For demonstration purposes, I'll be using the simple router below, which is a simplified version of the "Basic Routing" example from the [react-router docs](https://reactrouter.com/web/guides/quick-start#quick-start-1st-example-basic-routing).

```tsx
import React from 'react';
import {BrowserRouter as Router, Routes, Route, Link} from 'react-router-dom';

export default function App() {
  return (
    <Router>
      <div>
        <ul>
          <li>
            <Link to="/">Home</Link>
          </li>
          <li>
            <Link to="/about">About</Link>
          </li>
        </ul>

        <Routes>
          <Route path="/about" element={<h1>About</h1>} />
          <Route path="/" element={<h1>Home</h1>} />
        </Routes>
      </div>
    </Router>
  );
}
```

## `Browser history needs a DOM`

Let's start by creating a new Next.js app, installing React Router (`npm install react-router-dom`), and pasting the router code above into `pages/index.tsx`. When you run `next dev` you'll immediately get a not-so-subtle `Invariant failed: Browser history needs a DOM` error.

![Browser history needs a DOM](/nextjs_no_dom.png)

This happens because Next.js tries to pre-render your pages on the development server in development mode. React Router, on the other hand, requires access to the global `window` object provided by the browser. Because `window` isn't available in the server environment, React Router craps out.

To fix this, we need to (_drumroll please_) use a [custom App](https://nextjs.org/docs/advanced-features/custom-app)! Add a file called `pages/_app.tsx` (or `pages/_app.js` if you're a monster). Let's try a naive approach to fixing this problem. If `window` isn't defined, return `null`; otherwise, render the page.

```tsx
import {AppProps} from 'next/app';
import {useEffect, useState} from 'react';

function App({Component, pageProps}: AppProps) {
  if (typeof window === 'undefined') return null;
  return <Component {...pageProps} />;
}
export default App;
```

## `Hydration failed`

Reload the page and you'll see a new error.

![Hydration failed](/hydration_failed.png)

When you load a page of your Next.js app, Next.js 1) tries to pre-render it on the server, 2) sends the result to the browser, and 3) "re-hydrates" the page in the browser. Re-hydration means the page is re-rendered again on the browser and compared against the version that was rendered on the server. If they disagree, React throws an error.

Let's get a little more clever.

```ts
import {AppProps} from 'next/app';
import {useEffect, useState} from 'react';

function App({Component, pageProps}: AppProps) {
  const [render, setRender] = useState(false);
  useEffect(() => setRender(true), []);
  return render ? <Component {...pageProps} /> : null;
}
export default App;
```

This approach `useState` and `useEffect`. When Next.js renders a page on the server, it doesn't execute any `useEffect` calls; it just renders the page using default values and returns the result.

When this page is loaded by the browser will do the same thing: render the page using default values. Because the default value of `render` is `false`, the browser will initially render `null`. This is great, because it agrees with the server-rendered version of the page.

Immediately after, the browser will execute the `useEffect` call, which will set `render` to `true`. The page will now re-render in its full glory. We do waste a little time rendering `null` initially, but it's a microscopic amount of time: under a millisecond.

## `404 Page Not Found`

React Router and the Next.js router can interact in strange and unexpected ways. To understand the problem, you need to understand the different between client-side routing and "server-side routing".

Currently `pages/index.tsx` implements a router with 2 "pages": the home page (`/`) and an about page (`/about`).

Try this:

1. Go to `localhost:3000`
2. Click the "About" link

You should see the URL changes to `localhost:3000/about` and we see the About "page". Great! But:

3. Refresh the page

Now we get a 404. What gives? `localhost:3000/about` worked just fine before the reload.

As far as Next.js is concerned, the homepage (`localhost:3000`) is the _only_ page that exists. Once that page loads, react-router siezes control of the URL bar. When you click a react-router `<Link>`, it changes the URL programmatically, but the page is never fully reloaded.

But when you refresh the page, the browser asks Next.js to provide the content for `localhost:3000/about`. Since we haven't implemented `pages/about.tsx`, Next.js gives up and throws a 404.

The solution: [Rewrites](https://nextjs.org/docs/api-reference/next.config.js/rewrites)!

This feature was made available in Next.js 9.5. Rewrites are like "URL aliases"; you can tell Next.js to remap some "source" URL to a different "destination" URL. You can configure them in `next.config.js` file (if that file doesn't exist you should create it in the root of your project). Add the following contents:

```ts
// next.config.js
module.exports = {
  async rewrites() {
    return [
      {
        source: '/:any*',
        destination: '/',
      },
    ];
  },
};
```

This rewrite rule maps _all incoming requests_ (`/:any*`) to the homepage (`/`).

Now, if you restart the server and navigate to `localhost:3000/about` your `index.tsx` homepage should render. Yay!

## Mixing client- and server-side rendering

Important note: Next.js _ignores_ this rewrite rule if the incoming request corresponds to an existing page in the `pages` directory. For instance, if you add `pages/about.tsx` later, Next.js will stop rewriting `/about` to the homepage.

![Nextjs rewrite documentation](/nextjs_pages.png)

This is actually really cool! For starters, it lets you use API Routes without any additional configuration. But more importantly, it lets you mix-and-match the SPA, SSR, and SSG paradigms at will! For instance, to add a `/settings` page that is server-side rendered:

1. Create and implement `pages/settings.tsx`
2. Add a `getServerSideProps` fetcher
3. Add a link to `/settings` using the `<Link>` component from `next/link` (_not_ the one from `react-router-dom`!)

Now you have a single server-side rendered page inside your SPA! Can't do that with create-react-app 🙃

## Wrap up

Big shout out to Tanner Linsley's [GitHub Gist](https://gist.github.com/tannerlinsley/65ac1f0175d79d19762cf06650707830) on converting from CRA to Next.js, which inspired and informed this post.

You can see ALL the code I used here in the demo repo at [github.com/colinhacks/nextjs-spa](https://github.com/colinhacks/nextjs-react-router).

The "comments section" for this post is this Twitter thread. Chime in! I love discussing Next.js, TypeScript, open-source software, and just about everything else!

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">✨ NEW BLOG POST ✨<br><br>It started as a simple &quot;How to build a SPA with Next.js&quot; tutorial.<br><br>It turned into a fevered rant about Vercel, Next.js, perverse incentives, and the future of the SPA paradigm. Oops.<a href="https://t.co/hrRtKojezH">https://t.co/hrRtKojezH</a></p>&mdash; Colin McDonnell (@colinhacks) <a href="https://twitter.com/colinhacks/status/1321871817095684097?ref_src=twsrc%5Etfw">October 29, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

If you're into Next.js or TypeScript, follow me [on Twitter @colinhacks](https://twitter.com/colinhacks)! I build and maintain open-source tools like [Zod](https://github.com/colinhacks/zod) (a TypeScript validation library) and [tRPC](https://github.com/colinhacks/trpc) (a tool for building end-to-end typesafe APIs with TypeScript). Or subscribe to the newsletter to get notified when I publish new posts!

<br/>



<br/>



<br/>

Next.js is awesome. React Router is awesome. Why is it so hard to get them to play nice?

Building a single-page application with Next.js and React Router

> **Feb 24, 2021**: a previous version of this post contained a bug where a new cookie would be created each time you signed out and back in again. These (identical) cookies would stack up cause problems. If you're having issues, pass the `{ path: '/' }` option to `nookies.set` call — details below.

I had a hell of a time figuring out how to get Firebase Auth, Next.js, tokens, cookies, and `getServerSideProps` to play nice together. I'm hoping this breakdown will spare you the same suffering!

> To jump straight to the sample repo, head to https://github.com/colinhacks/next-firebase-ssr 🤙



I'm currently building a new app. My goal is to do _all_ data fetching inside `getServerSideProps` , just like the bad ole days of HTML templating. There are a lot of reasons for this.

- It reduces the cognitive overhead compared to the classic JAMstack approach. The client-server dichotomy is gone — it's just a server! So I can deploy my entire application with a single command.
- It solves the stale bundle problem. With a fully client-side rendered app, an user may use your application for days or weeks without doing a full page refresh. If the API changes underneath them, they may send API payloads that are no longer supported.
- It strikes a great balance between dynamism and search engine optimization.
- The user experience is unbeatable. With good caching and performant fetching, you can swap out the ubiquitous SPA spinner for snappy page loads.

As usual, Dan Abramov says it best:

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">We’ve reached peak complexity with SPA. The pendulum will swing a bit back to things on the server. But it will be a new take — a hybrid approach with different tradeoffs than before. Obviously I’m thinking React will be a part of that wave.</p>&mdash; Dan Abramov (@dan_abramov) <a href="https://twitter.com/dan_abramov/status/1290289129255624706?ref_src=twsrc%5Etfw">August 3, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

It seems like the SSR-centric approach to building apps with Next.js is gaining steam fast. So I'm extremely confused why the official Next.js [with-firebase-authentication example project](https://github.com/vercel/next.js/tree/canary/examples/with-firebase-authentication) doesn't demonstrate how to do it! It only demonstrates how to make authenticated POST requests to an API, not authenticate users inside `getServerSideProps` .

So below I explain how to use Next.js and Firebase Auth to:

- sign in users (duh)
- generate ID tokens
- store those ID tokens as a cookie
- auto-refresh the cookie whenever Firebase refreshes the ID token (every hour by default)
- implement authenticated routes
- authorize the user in `getServerSideProps`
- redirect unauthenticated users to a login page from within `getServerSideProps`

Let's dive in.

## #1 Configure your Firebase JS SDK

You've probably done this already. Should look something like this:

```ts
// firebaseClient.ts

import * as firebase from 'firebase/app';
import 'firebase/auth';

if (typeof window !== 'undefined' && !firebase.apps.length) {
  firebase.initializeApp({
    apiKey: 'APIKEY',
    authDomain: 'myproject-123.firebaseapp.com',
    databaseURL: 'https://myproject-123.firebaseio.com',
    projectId: 'myproject-123',
    storageBucket: 'myproject-123.appspot.com',
    messagingSenderId: '123412341234',
    appId: '1:1234123412341234:web:1234123421342134d',
  });
  firebase.auth().setPersistence(firebase.auth.Auth.Persistence.SESSION);
}

export { firebase };
```

> The `firebase.apps.length` check is a clever way of preventing Next.js from accidentally re-initalizing your SDK when Next.js hot reloads your application!

## #2 Configure your Firebase Admin SDK

You've probably done this already too. Check out the Firebase docs for guidance. It'll look something like this:

```ts
// firebaseAdmin.ts

import * as firebaseAdmin from 'firebase-admin';

// get this JSON from the Firebase board
// you can also store the values in environment variables
import serviceAccount from './secret.json';

if (!firebaseAdmin.apps.length) {
  firebaseAdmin.initializeApp({
    credential: firebaseAdmin.credential.cert({
      privateKey: serviceAccount.private_key,
      clientEmail: serviceAccount.client_email,
      projectId: serviceAccount.project_id,
    }),
    databaseURL: 'https://YOUR_PROJECT_ID.firebaseio.com',
  });
}

export { firebaseAdmin };
```

Alternatively you can store the `secret.json` file elsewhere on your file system and tell Firebase where you find it using the `GOOGLE_APPLICATION_CREDENTIALS` environment variable. For details check out the Firebase Admin setup documentation [here](https://firebase.google.com/docs/admin/setup#initialize-sdk).

## #3 Create a Context and provider

Now we're getting to the meat and potatoes. We're going to use React Context to implement the authentication logic. Let's start by writing a simple provider:

```ts
import { createContext } from 'react';

const AuthContext = createContext<{ user: firebase.User | null }>({
  user: null,
});

export function AuthProvider({ children }: any) {
  const [user, setUser] = useState<firebase.User | null>(null);

  // handle auth logic here...

  return (
    <AuthContext.Provider value={{ user }}>{children}</AuthContext.Provider>
  );
}
```

In the `AuthProvider` component we initialize a `user` state and pass it into our React component hierarchy using React Context. If you haven't seen this pattern before, I recommend reading the [official docs](https://reactjs.org/docs/context.html) on Context before continuing.

### Listen for token changes

Now let's add a `useEffect` hook that initializes our Firebase authentication listener:

```ts
import nookies from 'nookies';

const AuthContext = createContext<{ user: firebase.User | null }>({
  user: null,
});

export function AuthProvider({ children }: any) {
  const [user, setUser] = useState<firebase.User | null>(null);

  useEffect(() => {
    return firebase.auth().onIdTokenChanged(async (user) => {
      if (!user) {
        setUser(null);
        nookies.set(undefined, 'token', '', { path: '/' });
      } else {
        const token = await user.getIdToken();
        setUser(user);
        nookies.set(undefined, 'token', token, { path: '/' });
      }
    });
  }, []);

  return (
    <AuthContext.Provider value={{ user }}>{children}</AuthContext.Provider>
  );
}
```

This is where the magic happens. We're hooking into the `firebase.auth().onIdTokenChanged` event listener; if you've never used it, it's identical to `onAuthStateChanged` but it _also_ fires when the user's ID token is refreshed.

In the `onIdTokenChanged` callback, we check if the user is still signed in. If they are, we set the user with `setUser` .

Here's the most important part: we also set a `token` cookie that contains the user's ID token. (To accomplish this, I'm using the excellent [ `nookies` ](https://github.com/maticzav/nookies) package by the great and powerful [@maticzav](https://github.com/maticzav).)

> Feb 24, 2021: passing the `{ path: '/' }` option to `nookies.set` prevents your app from creating a new token each time this code runs!

Now all outgoing requests — both API requests and page navigations! — will contain the user's ID token as a cookie! There's an example of this further down the page.

### Token refresh

A previous version of this post incorrectly assumed that Firebase automatically refreshes the ID token on an hourly basis. As it turns out, that's not true. Firebase only does so if it is maintaining an active connect to Firestore or Realtime Database. If you aren't using one of these, services we need to refresh the tokens ourselves.

Below is the final version of our AuthProvider. Note the new `useEffect` hook that automatically force-refreshes the Firebase token every 10 minutes.

```ts
import nookies from 'nookies';

const AuthContext = createContext<{ user: firebase.User | null }>({
  user: null,
});

export function AuthProvider({ children }: any) {
  const [user, setUser] = useState<firebase.User | null>(null);

  // listen for token changes
  // call setUser and write new token as a cookie
  useEffect(() => {
    return firebase.auth().onIdTokenChanged(async (user) => {
      if (!user) {
        setUser(null);
        nookies.set(undefined, 'token', '', { path: '/' });
      } else {
        const token = await user.getIdToken();
        setUser(user);
        nookies.set(undefined, 'token', token, { path: '/' });
      }
    });
  }, []);

  // force refresh the token every 10 minutes
  useEffect(() => {
    const handle = setInterval(async () => {
      const user = firebaseClient.auth().currentUser;
      if (user) await user.getIdToken(true);
    }, 10 * 60 * 1000);

    // clean up setInterval
    return () => clearInterval(handle);
  }, []);

  return (
    <AuthContext.Provider value={{ user }}>{children}</AuthContext.Provider>
  );
}
```

#### Why use React Context?

You may be wondering why you need to use React Context at all. Let's consider a more naive approach to writing this hook.

```ts
// DONT DO THIS

export const useAuth = () => {
  const [user, setUser] = useState<firebase.User | null>(null);

  useEffect(() => {
    return firebase.auth().onIdTokenChanged(async (user) => {
      if (user) {
        setUser(user);
      } else {
        setUser(null);
      }
    });
  }, []);

  return { user };
};
```

This actually works fine. However it would create a new `user` state variable and a new listener _every time you use the hook_. We want to make sure the `user` variable is the same everywhere throughout our application to avoid difficult-to-debug synchronization issues. We only want a _single reference_ to the currently signed in user that is shared throughout our app.

## #4 Add your provider to a custom App

The documentation for setting up a Custom App page with Next.js are at https://nextjs.org/docs/advanced-features/custom-app. It's the ideal place to add "wrapper" code that you want to exist on all other pages — like Context providers.

```tsx
// pages/_app.tsx

import type { AppProps } from 'next/app';
import { AuthProvider } from '../auth';

function App({ Component, pageProps }: AppProps) {
  return (
    <AuthProvider>
      <Component {...pageProps} />
    </AuthProvider>
  );
}
export default App;
```

## #5 Create the useAuth hook

Now that the context and provider is set up, our actual `useAuth` hook is dead simple:

```ts
export const useAuth = () => {
  return useContext(AuthContext);
};
```

## #6 Check the token in getServerSideProps

Below is a complete working example of how to implement an authenticated route. If the user isn't properly signed in (e.g. if the `token` cookies doesn't exist or if the token verification fails) the user is redirected to the `/login` page.

Otherwise, you can confidently fetch and return data belonging to the user!

```tsx
// /pages/authenticated.tsx
import React from 'react';
import nookies from 'nookies';
import { InferGetServerSidePropsType, GetServerSidePropsContext } from 'next';

import { firebaseAdmin } from '../firebaseAdmin';

export const getServerSideProps = async (ctx: GetServerSidePropsContext) => {
  try {
    const cookies = nookies.get(ctx);
    const token = await firebaseAdmin.auth().verifyIdToken(cookies.token);

    // the user is authenticated!
    const { uid, email } = token;

    // FETCH STUFF HERE!! 🚀

    return {
      props: { message: `Your email is ${email} and your UID is ${uid}.` },
    };
  } catch (err) {
    // either the `token` cookie didn't exist
    // or token verification failed
    // either way: redirect to the login page
    ctx.res.writeHead(302, { Location: '/login' });
    ctx.res.end();

    // `as never` prevents inference issues
    // with InferGetServerSidePropsType.
    // The props returned here don't matter because we've
    // already redirected the user.
    return { props: {} as never };
  }
};

export default (
  props: InferGetServerSidePropsType<typeof getServerSideProps>
) => (
  <div>
    <p>{props.message}</p>
  </div>
);
```

## The full solution

If you've made it this far you must think the ideas in this post are pretty useful! Consider retweeting this to get the word out: 🙌

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Wrote up a guide to authenticated server-side rendering with Nextjs and Firebase Auth. <br><br>There&#39; s a surprising lack of documentation or sample code for this use case, given that it may well become a best practice over the next couple years. 1/n<a href="https://t.co/22hVqp8Fy5">https://t.co/22hVqp8Fy5</a></p>&mdash; Colin McDonnell (@colinhacks) <a href="https://twitter.com/colinhacks/status/1295486862899859456?ref_src=twsrc%5Etfw">August 17, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

The full code of this project — plus some other goodies! a sign up form! a logout button! — is available at https://github.com/colinhacks/next-firebase-ssr. Clone that repo to hit the ground running. 🏃🏽‍♂️💨

By the way, I'm currently working on an extremely rad library that lets you define your API with TypeScript and auto-generate a strongly typed SDK that you can safely use on the client. I think it could be a gamechanger for full-stack TypeScript/Next.js development. Join the mailing list below to stay updated.

Authenticated server-side rendering with Next.js and Firebase

> Check out the HN discussion <a target="_blank" rel="noopener noreferrer" href="https://news.ycombinator.com/item?id=23981563">here</a>.
>
> This article has been translated into [Japanese](https://itnews.org/news_contents/a-new-funding-model-for-open-source-software) and [Russian](https://pngset.com/ru-open-source-software).

The existing open-source funding models don't work well for small projects.

Big projects — operating systems, frameworks, CMSs, or fully self-hostable applications — are in a privileged position to extract more value from their users, especially corporate ones. Since entire APIs and products are built on top of them, they inspire enough appreciation (or, more likely, fear of obsolescence!) to net a sustainable income from one-off or monthly donations.¹

But most OSS projects aren't big. The typical project on GitHub is better described as a **utility** — an apt term, given the [infrastructural role](https://www.fordfoundation.org/work/learning/research-reports/roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure/) these projects play in the world. It is a small tool that does one thing really well. Over the course of building a complex application, you may end up using dozens of these utilities — and they'll save you hundreds of hours of development time.

Unfortunately, utilities like these rarely bring in any meaningful amount of money from donations, no matter how widely used or beloved they are. Consider [react-router](https://github.com/ReactTraining/react-router). Even with 41.3k stars on GitHub, 3M weekly downloads from NPM, and nearly universal adoption in React-based single-page applications, it only brings in [~\$17k](https://opencollective.com/react-router) of donations annually.

The root of the problem is that open-source donations are made on a per-project basis². To support a project via GitHub Sponsors or OpenCollective, you must create yet another auto-renewing monthly subscription for each project you want to support. Moreover, at the moment of truth (_"I'm gonna sponsor X!"_) it's easy to "logic" yourself out of donating (_"But what if it gets replaced by some new hotness next week?!"_). This has a huge dampening effect on total donations to open source. And in the end, only the projects that are massively, insanely, indisputably useful get funded. And those projects are usually the "big stuff" — frameworks, self-hostable software, etc.

A new model is needed; one that works for the small- and medium-sized projects, not just the huge ones. So I propose a novel(-ish) approach to OSS sustainability.

## Introducing "sponsor pools"

1. Every month, you donate some amount into a "wallet".
2. Your funds are then distributed to the projects in your "sponsor pool". Your sponsor pool is just the set of open-source projects you want to support.
3. Adding new projects to your pool should require **one click** — as easy as starring the repo on GitHub.

That's it. It's hardly ingenious, which is why it's surprising that no major player in OSS has implemented it for facilitating open source donations.³

This would achieve the holy grail of OSS sustainability: the one-click sponsorship.⁴ Once a person has funded their sponsor pool, it would take _a single click_ to financially support another project. <mark>The marginal cost — both psychological and financial — of supporting additional projects would drop to zero.</mark> That's a game changer.

### Why this works better

Why do I think this will dramatically increase the total amount donated to open source? To answer that, consider a hypothetical question.

_How much of your income would you be willing to donate to open-source software every year?_

I suspect the typical overpaid HN lurker reading this post (👋) came up with a number in the hundreds of dollars. Compare that with how much you’re actually donating — is there a difference? There is for me. <mark>That's because there's currently no way to make a donation to the abstract concept of "open source software".</mark> But with sponsor pools, donation amounts will reflect an honest answer to the question posed above.

### GitHub

The best case scenario, in my opinion, is for GitHub to natively support this model as an extension of GitHub Sponsors. It's where most projects live so it's best positioned to create a zero-friction donation system like this.

Of course, if GitHub implemented something like this, they would likely divorce the donation mechanism from stars. Perhaps instead, users who have created and funded a sponsor pool can be presented with an "Add to Pool" button in place of the current "Sponsor" button.

![Add to pool button](https://colinhacks.com/add_to_pool.png)

GitHub has a financial incentive to switch to this approach — sort of. They're currently eating the cost of all processing fees for transactions on GitHub Sponsors. So if you sponsor a project at $1/mo, the maintainer gets $1/mo...and GitHub pays \$0.30 to the credit card companies. Larger donation sizes means GitHub pays a proportionally smaller amount as fees.

Note that I said _proportionally smaller_. If more total donations are being processed (which is the goal of this proposal!), they may still end up paying more in total fees, even if the ratio of fees-to-donations is smaller. If the sponsor pool concept is massively successful — say, cumulative donations of $1B annually — GitHub will be eating nearly $20M in card fees. I suspect that would ruffle some feathers at Microsoft.

As a final addendum — I'd love to see embeddable "badges" that are unlocked at different levels of donorship. Imagine a world where you see a "GitHub Sponsor Gold Badge" in the footer of someone’s website, indicating that they donate, say, \$1000+ annually to open source. Clicking on this badge would bring you to a trusted site that lets you verify the claim. Here's my quick-and-dirty attempt at a badge design:

![order of the gold octocat](https://colinhacks.com/gold_octocat.jpg)

Some may cry "virtue signaling" but approaches like this are a proven way to establish positive reinforcement loops that a) increase awareness and b) encourage more folks to donate to open source.

## Wrapping up

This is a hard problem with a vast array of potential solutions. I don't mean to criticize any existing approaches; they've all done a lot for maintainers, including myself, and I don't mean to minimize that. There may also be major implementational or regulatory hurdles to building something like this that I'm not aware of. This is merely a hypothetical exercise.

_I write proposals like this one every month or so. Put your email in the box to get future proposals delivered to your inbox. ✌️_





### Footnotes

¹ Some examples of financially successful large projects.

>

- operating systems: Red Hat, Elementary OS
- frameworks: Livewire, Next.js
- CMSs: Ghost, Wordpress
- Full-featured applications: Sentry, Blender, Discourse

<br>

² It's worth noting that currently GitHub Sponsorships actually correspond to a given user or organization, NOT a particular repository or project.

> I think it's important that the sponsorship pools themselves are comprised of projects, _not_ users or organizations. This gets complicated if multiple entities/organizations are listed as the maintainers of a particular project. There would need to be a mechanism in place to split the incoming donation money among the set of maintainers as they see fit.

<br>

³ It's been pointed out to me that [Flattr](https://flattr.com) uses this exact model! Unfortunately they're not intended for open source software specifically and simply don't have enough awareness to really move the needle. For this model to reach it's potential, it needs to be implemented by a visible, trusted, established player.

> Other organizations trying interesting things: [Snowdrift](https://snowdrift.coop) ("crowdmatching"...very cool concept), [Flossbank](https://flossbank.com/) (a Tidelift-like model), [Gitcoin](https://gitcoin.co/).

<br>

⁴ The even holier grail: a zero click sponsorship! An example of this approach is [Brave Browser's](https://brave.com/) token-based micropayments system. Brave lets you fund your "wallet" with various cryptocurrencies, which are then distributed to creators proportional to the time you spend on their site.

> It’s a great idea, but surveillance is necessary to gather the requisite information. This is implemented ethically by Brave, but that’s only possible because they control the full browser experience. For this to work across browsers would require an extension with full access to your browsing patterns. So probably a non-starter.

GitHub Sponsors is a step forward, but it's far from a panacea. I propose an scalable alternative funding model for open-source software called sponsor pools.

Or, achieving the holy grail of one-click sponsorships

Sponsor pools: a new funding model for open source software

> A maintainer of Emotion published a very thoughtful response to a version of this post! I encourage you to check that out [here](https://github.com/emotion-js/emotion/issues/1883#issuecomment-637827074). The essay below has modified to address some of his points.

Emotion is my favorite CSS-in-JS library.

It's easy to define style classes — both inline or in separate files. You can powerfully compose classes in powerful ways with the `cx` utility (the Emotion equivalent of Jed Watson's [classnames](https://github.com/JedWatson/classnames)). You sprinkle in your styles using the standard `className` attribute. There's no need to modify your markup/JSX — as it should be! You only need to install a single module (`yarn add emotion`). And there's no complicated Babel plugin or config file to set up.

```
import { css, cs } from 'emotion';

const redBorder = css({ border: '1px solid red' });
const blueText = css({ color: 'blue' });

const MyComp = () => {
  return <div className={cx(redBorder, blueText)}>hello</div>;
};
```

I'm currently building a Tailwind-style CSS-in-JS utility styling library with a maniacal focus on brevity and developer experience. If you want to hear about that when it's ready, join my mailing list.



### The "but"

But unfortunately everything I just said only applies to the "vanilla" `emotion` module ([docs](https://emotion.sh/docs/emotion)), _not_ the confusingly-named `@emotion/core` module.

`@emotion/core` is the React-centric Emotion wrapper that gives you some extra goodies, specifically server-side rendering and theming. It's also the official recommendation of the Emotion project for any React project.

![Emotion's recommending emotion core](https://colinhacks.com/emotion_recommendation.png)

So why exactly is Emotion recommending this for React devs?

## Minimal upsides

The main three advertised benefits of using `@emotion/core` are server-side rendering (SSR), theming, and customizability. Let's dig into those.

### Out-of-the-box SSR

There's no denying this is a remarkable technical achievement. Getting SSR to "just work" with both Next.js, Gatsby, and the classic `ReactDOMServer.renderToString` approach is very impressive. I'd be lying if I claimed to understand the complexities involved.

I don't have data on this, but — in my experience — SSR isn't a consideration for a hefty majority of React projects. If you started a project/website in the last 7 years where SEO/SEO/pageload speed/bundle size was an important design consideration, you probably didn't choose React. Website builders, static site generators, and HTML templating still dominate that arena. Take it from someone who got [torn apart on HN](https://news.ycombinator.com/item?id=23309002) for advocating the use of React/Next.js for personal developer websites 😘



For the people who do need SSR, the guidance is a little slim.

#### Next.js

There's no explicit documentation from Next.js on how to set up SSR with vanilla `emotion`. Next.js provides a sample project [here](https://github.com/vercel/next.js/tree/master/examples/with-emotion). Notably, this project a) has a very uninformative Readme and b) is built with `@emotion/core`! So it's not immediately obvious that the approaches on display will even transfer to a vanilla project.

Enough buildup. Here's the internet's first comprehensive guide to setting up SSR with vanilla Emotion and Next.js:

1. `yarn add emotion-server`
2. create `_document.tsx` in your `pages` directory and copy [this gist](https://gist.github.com/colinhacks/c40519a6a050a99091862319151377ec) into it. ⚠️ This gist is intended for Emotion v10. Version 11 is the latest version. ⚠️
3. ok ur done

#### Gatsby

For completeness, here's some instructions for Gatsby users, too.

1. `yarn add gatsby-plugin-emotion`
2. add `'gatsby-plugin-emotion'` to your `plugins` list in `gatsby-config.js`

If you're using @emotion/core to avoid the complexities of SSR configuration, you may want to reconsider.

### Theming

In the era of React Context and Hooks, there's no reason for libraries to use prop or high-order components to handle theming. Emotion provides a useTheme hook, but it still requires adding an additional library (`emotion-theming`).

This isn't a controversial claim; the next version of Emotion [will explicitly recommend](https://github.com/emotion-js/emotion/pull/973/files) using a Context/Hook based solution, so I won't belabor this point.





Even Context/Hooks might be overkill for many projects. Just define your theme values as variables and import them into components as needed. If you're using TypeScript here's some code to get you started:

```tsx
// theme.ts
export const primaryColor = "blue";
export const serif = `'Encode Sans', Times New Roman, Times, serif`;

// anydamnfile.ts
import { css } from 'emotion';
import * as theme from './theme.ts';

export const MyComponent = ()=>{
  return <p className={css({ color: theme.primaryColor, fontFamily: theme.serif })}>
}
```

If you want to import your theme with a `useTheme` hook, here's a workable implementation that took me several seconds to write:

```tsx
import * as theme from './theme.ts';
export const useTheme = () => theme;
```

### Customization

> Edit: This section was added after the fact to address a concern raised by the maintainer of Emotion in [his response](https://github.com/emotion-js/emotion/issues/1883#issuecomment-637827074) to an earlier version of this post.

@emotion/core provides a `CacheProvider` component that lets you customize low-level aspects of it's behavior. This customization isn't possible with vanilla emotion. I'll let the maintainer of Emotion explain it:

> Because [@emotion/core] doesn't give you string class names back, but rather just opaque objects, the rule insertion is lazy. This might not be as interesting on its own, but it allows us to customize injection through `<CacheProvider>`. Use cases for this are not as common - but they exist and once you ever come across one then you might be very thankful for the path we've chosen...Main things that can be customized:
>
> - prefixing rules
> - parser plugins
> - container element (very important for rendering into iframes)
> - nonce (very important for CSP)
>
> — Andarist ([Github](https://github.com/Andarist))

If you absolutely need this degree of customizability, then @emotion/core is probably right for you.

For everyone else, let's look at the downsides.

## Serious downsides

### The `css` prop

Emotion [recommends using](https://emotion.sh/docs/css-prop) their non-standard `css` prop to style your components, instead of React's built-in `className`. This causes me immeasurable emotional pain.



This approach destroys the portability of your React components. Your components are now unusable in any codebase that isn't configured to use `@emotion/core`.

The portability and encapsulation of React components is one of the most powerful and wondrous achievements in web development in the last last decade. Don't give that up without a good reason!

### Installation woes

Unfortunately to get that non-native `css` prop to work, Emotion core entirely replaces your project's JSX parser. It substitutes the built-in `React.createElement` function with Emotion's custom `jsx` function.

![Replacing React's createElement function](https://colinhacks.com/jsx_replacement.png)

There are a couple ways to set this up.

Option #1: install the `@emotion/babel-preset-css-prop` Babel plugin and add it to your `.babelrc`. If you're using Create React App, this isn't impossible. If you're using TypeScript, you probably don't have a `.babelrc` in your project.

If you're in one of those buckets, there's option #2: copy these two lines at the top of every React component you want to style with Emotion:

```tsx
/** @jsx jsx */

import { jsx } from '@emotion/core';
```





If your TypeScript config or linter doesn't allow unused imports, you'll have to disable those rules to get rid of the warning. Check out [this issue](https://github.com/emotion-js/emotion/issues/1046) if you want to see dozens of TypeScript users being sad about this.

![Sad Typescripters with linter warnings](https://colinhacks.com/sad_emotion_small.png)

### Lack of composability

Perhaps the most damning issue with @emotion/core is that it makes the simple things harder. If you want to define a new class or use `cx`, you have to wrap your component with the `ClassNames` render prop. But with @emotion/core, these basic functions — found in nearly all CSS-in-JS libraries — requires you to modify your markup. In my humble opinion, requiring markup modifications is a cardinal sin for a styling library.

Here's the example from the top of this post, reimplemented with @emotion/core:

```tsx
import { ClassNames } from '@emotion/core';

const MyComp = () => {
  return (
    <ClassNames>
      {({ css, cx }) => {
        const redBorder = css({ border: '1px solid red' });
        const blueText = css({ color: 'blue' });

        return <div className={cs(redBorder, blueText)}>hello</div>;
      }}
    </ClassNames>
  );
};
```

## Wrapping up

I understand how this happened. Vanilla emotion was undoubtedly getting flooded with GitHub issues by frustrated developers hitting against subtle limitations of its design. @emotion/core fixes these issues. But because @emotion/core is now the officially recommended approach for _all_ React projects (the vanilla option isn't even mentioned on the Readme anymore), I suspect thousands of developers using it who would be better served by plain ol' `emotion`.

And finally: a huge thank you to the Emotion team for all their exceptional work and contributions to the open-source community.

_I'm an open sourcer and ride-or-die React fan. Put your email in the box to hear when I write new things._

If you're using Emotion in your React project, @emotion/core is the offically recommended approach. But it's probably not ideal for your project.

Why you shouldn't use @emotion/core

> Check out the HN <del>roast</del> discussion <a href="https://news.ycombinator.com/item?id=23309002">here</a>! 🤗

> This article has been [translated into Russian](https://howtorecover.me/vybor-tekhnicheskogo-steka).

I recently set out to build my personal website — the one you're reading now, as it happens!

Surprisingly, it was harder than expected assemble a "tech stack" that met my criteria. My criteria are pretty straightforward; I would expect most React devs to have a similar list. Yet it was surprisingly hard to put all these pieces together.

Given the lack of a decent out-of-the-box solution, I worry that many developers are settling for static-site generators that place limits on the interactivity and flexibility of your website. We can do better.

<div style="padding:14px 20px; border: 0px solid #555; margin:15px 0px; border-left: 4px solid #4BBAAB">
  <b>tl;dr</b> 
  <br/>
  Clone the repo here to get started with this setup: <a href="https://github.com/colinhacks/devii">https://github.com/colinhacks/devii</a>.
</div>

Let's quickly run through my list of design goals:

### React (+ TypeScript)

I want to build the site in React and TypeScript. I love them both wholeheartedly, I use them for my day job, and they're gonna be around for a long time. Plus writing untyped JS makes me feel dirty.

I don't want limitations on what my personal website can be/become. Sure, at present my site consists of two simple, static blog posts. But down the road, I may want to build a page that contains an interactive visualization, a filterable table, or a demo of a React component I'm open-sourcing. Even something simple (like the email newsletter signup form at the bottom of this page) was much more pleasant to implement in React; how did we use to build forms again?

Plus: I want access to the npm ecosystem and my favorite UI, animation, and styling libraries. I sincerely hope I never write another line of raw CSS ever again.

### Good authoring experience

If it's annoying to write a blog post, I won't do it. That's a regrettable law of the universe. Even writing blog posts in plain HTML — a bunch of `<p>` tags in a div — is just annoying enough to bug me. The answer: Markdown of course!

Static site generators (SSGs) like Hugo and Jekyll provide an undeniably wonderful authoring experience. All you have to do is `touch` a new .md file in the proper directory and get to writing. Unfortunately all Markdown-based SSGs I know of are too restrictive. Mixing React and Markdown on the same page is either impossible or tricky. If it's possible, it likely requires some plugin/module/extension, config file, blob of boilerplate, or egregious hack. Sorry Hugo, I'm not going to re-write my React code using `React.createElement` like it's 2015.

Well, that doesn't work for me. I want my website to be React-first, with a sprinkling of Markdown when it makes my life easier.

### Static generation

As much as I love the Jamstack, it doesn't cut it from an SEO perspective. Many blogs powered by a "headless CMS" require two round trips before rendering the blog content (one to fetch the static JS bundle and another to fetch the blog content from a CMS). This degrades page load speeds and user experience, which accordingly degrades your rankings on Google.

Instead I want every page of my site to be pre-rendered to a set of fully static assets, so I can deploy them to a CDN and get fast page loads everywhere. You could get the same benefits with server-side rendering, but that requires an actual server and worldwide load balancing to achieve comparable page load speeds. I love overengineering things as much as the next guy, even I have a line. 😅

## My solution

I describe my final architecture design below, along with my rationale for each choice. I distilled this setup into a website starter/boilerplate available here: https://github.com/colinhacks/devii. Below, I allude to certain files/functions I implemented; to see the source code of these, just clone the repo `git clone git@github.com:colinhacks/devii.git`

### Next.js

I chose to build my site with Next.js. This won't be a surprising decision to anyone who's played with statically-rendered or server-side rendered React in recent years. Next.js is quickly eating everyone else's lunch in this market, especially Gatsby's (sorry Gatsby fans).

Next.js is by far the most elegant way (for now) to do any static generation or server-side rendering with React. They released their next-generation (pun intended) static site generator in the [9.3 release](https://nextjs.org/blog/next-9-3) back in March 2020. In the spirit of using technologies [in the spring of their life](https://www.youtube.com/watch?v=eBAX8MbRYFA), Next.js is a no-brainer.

Here's a quick breakdown of the project structure. No need to understand every piece of it; but it may be useful to refer to throughout the rest of this post.

```
.
├── README.md
├── public // all static files (images, etc) go here
├── pages // every .tsx component in this dir becomes a page of the final site
|   ├── index.tsx // the home page (which has access to the list of all blog posts)
|   ├── blog
|       ├── [blog].md // a template component that renders the blog posts under `/md/blog`
├── md
|   ├── blog
|       ├── devii.md // this page!
        ├── whatever.md // every MD file in this directory becomes a blog post
├── components
|   ├── Code.tsx
|   ├── Markdown.tsx
|   ├── <various others>
├── loader.ts // contains utility functions for loading/parsing Markdown
├── node_modules
├── tsconfig.json
├── package.json
├── next.config.js
├── next-env.d.ts
├── .gitignore
```



### TypeScript + React

Both React and TypeScript are baked into the DNA of Next.js, so you get these for free when you set up a Next.js project.

Gatsby, on the other hand, has a special plugin for TypeScript support, but it's not officially supported and seems to be [low on their priority list](https://github.com/gatsbyjs/gatsby/issues/18983). Also, after messing with it for an hour I couldn't get it to play nice with hot reload.

### Markdown authoring

Using Next's special `getStaticProps` hook and glorious [dynamic imports](https://nextjs.org/docs/advanced-features/dynamic-import#with-no-ssr), it's trivial to import a Markdown file and pass its contents into your React components as a prop. This achieves the holy grail I was searching for: the ability to easily mix React and Markdown.

#### Frontmatter support

Every Markdown file can include a "frontmatter block" containing metadata. I implemented a simple utility function (`loadPost`) that loads a Markdown file, parses its contents, and returns a TypeScript object with the following signature:

```ts
type PostData = {
  path: string; // the relative URL to this page, can be used as an href
  content: string; // the body of the MD file
  title?: string;
  subtitle?: string;
  date?: number;
  author?: string;
  author_image?: string;
  tags?: string[];
  banner_image?: string;
  thumb_image?: string;
};
```

I implemented a separate function `loadPosts` that loads _all_ the Markdown files under `/md/blog` and returns them as an array (`PostData[]`). I use `loadPosts` on this site's home page to render a list of all posts I've written.

### Medium-inspired design

I used the wonderful [`react-markdown`](https://github.com/rexxars/react-markdown) package to render Markdown as a React component. My Markdown rendered component (`/components/Markdown.tsx`) provides some default styles inspired by Medium's design. Just modify the `style` pros in `Markdown.tsx` to customize the design to your liking.

### GitHub-style code blocks

You can easily drop code blocks into your blog posts using triple-backtick syntax. Specify the programming language with a "language tag", [just like GitHub](https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks)!

To achieve this I implemented a custom `code` renderer (`/components/Code.tsx`) for `react-markdown` that uses [react-syntax-highlighter](https://github.com/conorhastings/react-syntax-highlighter#readme) to handle the highlighting. So this:



<pre>
```ts
// pretty neat huh?
const test = (arg: string) => {
  return arg.length > 5;
};
```</pre>

turns into this:

```ts
// pretty neat huh?
const test = (arg: string) => {
  return arg.length > 5;
};
```

### RSS feed generation

An RSS feed is auto-generated from your blog post feed. This feed is generated using the `rss` module (for converting JSON to RSS format) and `showdown` for converting the markdown files to RSS-compatible HTML. The feed is generated during the build step and written as a static file to `/rss.xml` in your static assets folder. It's dead simple. That's the joy of being able to easily write custom build scripts on top of Next.js's `getStaticProps` hooks!

### SEO

Every blog post page automatically populated meta tags based on the post metadata. This includes a `title` tag, `meta` tags, `og:` tags, Twitter metadata, and a `link` tag containing the canonical URL. You can modify/augment this in the `PostMeta.ts` component.

### Static generation

You can generate a fully static version of your site using `yarn build && yarn export`. This step is entirely powered by Next.js. The static site is exported to the `out` directory.

After its generated, use your static file hosting service of choice (Firebase Hosting, Vercel, Netlify) to deploy your site.

### Insanely customizable

There's nothing "under the hood" here. You can view and modify all the files that provide the functionality described above. Devii just provides a project scaffold, some Markdown-loading loading utilities (in `loader.ts`), and some sensible styling defaults (especially in `Markdown.tsx`).

To start customizing, modify `index.tsx` (the home page), `BlogPost.tsx` (the blog post template), and `Markdown.tsx` (the Markdown renderer).

## Get started

Head to the GitHub repo to get started: [https://github.com/colinhacks/devii](https://github.com/colinhacks/devii). If you like this project, leave a ⭐️star⭐️ to help more people find Devii! 😎

To jump straight into the code, clone the repo and start the development server like so:

```sh
git clone git@github.com:colinhacks/devii.git mysite
cd mysite
yarn
yarn dev
```

_I write essays like this one every month. Put your email in the box to get future proposals delivered to your inbox._

Choosing a tech stack for my personal dev blog in 2020

> **Update 2023** — Many of the problems with other libraries have been solved since this post was originally published.

There are a handful of wildly popular validation libraries in the Javascript ecosystem with thousands of stars on GitHub.

When I started my quest to find the ideal schema declaration/validation library, I assumed the hardest part would be identifying the Most Great option from a sea of excellent ones.

But as I dug deeper into the many beloved tools in the ecosystem, I was surprised that none of them could provide the features and developer experience I was looking for.

> **tl;dr**: I made a new Typescript validation library that has static type inference and the best DX this side of the Mississippi. To jump straight to README, head over to https://github.com/colinhacks/zod

### A pinch of context

I'm building an API for a healthcare application with Typescript. Naturally I want this mission-critical medical software to be rock-solid, so my goal is to build a fully end-to-end typesafe data layer.

All data that passes from the client to the server AND server to client should be validated at runtime. Moreover, both the client and server should have static type definitions of all payloads, so I can catch more errors at compile-time.

I also don't hate myself, so I don't want to keep my static types and runtime type validators in sync by hand as my data model changes. Which means we need a tool that supports 🚀StAtIc TyPe InFeReNcE!!🚀 _vuvuzela sounds_

Let's look at our options!

## The existing options

### <a href="https://github.com/hapijs/joi">Joi</a> (14.3k stars)

Doesn't support static type inference. Boo.

### <a href="https://github.com/jquense/yup">Yup</a> (8.6k stars)

Yup is jquense's popular, Joi-inspired validation library that was implemented first in vanilla JS, with Typescript typings added later.

Yup supports static type inference! But with a small caveat: the typings are wrong.

For instance, the yup package treats all object properties as optional by default.

```ts
const schema = yup.object({
  asdf: yup.string(),
});
schema.validate({}); // passes
```

Yet the inferred type indicates that all properties are required. 😕

```ts
type SchemaType = yup.InferType<typeof schema>;

// returns { asdf: string }
// should be { asdf?: string }
```

Yup also mis-infers the type of "required" arrays.

```ts
const numList = yup.array().of(yup.string()).required();

// .required() is used to indicate a non-empty list
numList.validateSync([]); // fails

// yet the inferred type doesn't reflect this
type NumList = yup.InferType<typeof numList>;
// returns string[]
// should be [string,...string[]]
```

Finally, Yup doesn't explicitly support generic union and intersection types. This is a recurring [source](https://github.com/jquense/yup/issues/593) of [frustration](https://github.com/jquense/yup/issues/321) for the Yup community.

These may sound like nitpicks. But it's very uncool for the inferred type to not actually reflect the actual type of the validator it came from.

Moreover, there are no plans to fix these issues in Yup's type inference because those changes would be backwards compatible. See more about this [here](https://github.com/DefinitelyTyped/DefinitelyTyped/issues/42360).

### <a href="https://github.com/gcanti/io-ts">io-ts</a> (2.7k stars)

Finally, a library that was designed for Typescript from the ground up!

Look, `io-ts` is excellent library. It's creator, gcanti, has done more than anyone to bring proper higher-order functional programming to Typescript with his `fp-ts` library.

But in my situation, and I think many others', `io-ts` prioritizes functional programming purity at the expense of developer experience. Functional purity is a valid and admirable design goal, but it makes `io-ts` particularly hard to integrate into an existing codebase with a more procedural or object-oriented bias. It's difficult to progressively integrate `io-ts` without entirely refactoring your code with a more functional flavor.

For instance, consider how to define an object schema with optional properties in `io-ts`:

```ts
const A = t.type({
  foo: t.string,
});

const B = t.partial({
  bar: t.number,
});

const C = t.intersection([A, B]);

type C = t.TypeOf<typeof C>;
/*
{
  foo: string;
  bar?: number | undefined
}
*/
```

You must define the required props with a call to `t.type({...})`, define your optional props with a call to `t.partial({...})`, and merge them with `t.intersection()`.

Spoiler alert: Here's the equivalent in my new library:

```ts
const C = z.object({
  foo: z.string(),
  bar: z.number().optional(),
});

type C = t.TypeOf<typeof C>;
/* {
  foo: string;
  bar?: number | undefined
} */
```

`io-ts` also requires the use of gcanti's functional programming library `fp-ts` to parse results and handle errors. From the `io-ts` docs, here is the most basic example of how to run a validation:

```ts
import * as t from 'io-ts';
import {pipe} from 'fp-ts/lib/pipeable';
import {fold} from 'fp-ts/lib/Either';

// failure handler
const onLeft = (errors: t.Errors): string => `${errors.length} error(s) found`;

// success handler
const onRight = (s: string) => `No errors: ${s}`;

pipe(t.string.decode('a string'), fold(onLeft, onRight));
// => "No errors: a string"
```

The functional approach here is alienating for developers who aren't accustomed to it.

Again: `fp-ts` is _fantastic_ resource for developers looking to keep their codebase strictly functional. But depending on `fp-ts` necessarily comes with a lot of intellectual overhead; a developer has to be familiar with functional programming concepts, `fp-ts`'s nomenclature, and the Either monad to do a simple schema validation. It's alienating for devs who don't have a functional background, and it drives them to libraries like Yup, which returns incorrect types.

## Introducing Zod

So, in the tradition of many a nitpicky programmer, I decided to build my own library from scratch. What could go wrong?

The final result: <a href="https://github.com/colinhacks/zod">https://github.com/colinhacks/zod</a>.

Zod is a validation library designed for optimal developer experience. It's a Typescript-first schema declaration library with rigorous (and correct!) inferred types, incredible developer experience, and a few killer features missing from the existing libraries.

- Uses Typescript generic inference to statically infer the types of your schemas
- Eliminates the need to keep static types and runtime validators in sync by hand
- Has a composable, declarative API that makes it easy to define complex types concisely

Zod was also designed with some core principles designed to make all declarations as non-magical and developer-friendly as possible:

- Fields are required unless explicitly marked as optional (just like Typescript!)
- Schemas are immutable; methods (i.e. `.optional()`) return a new instance.
- Zod schemas operate on a <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/">Parse, don't validate!</a> basis!

> To jump straight to README, head over to https://github.com/colinhacks/zod. If you're feeling frisky, leave a star 🌟👍

### Primitives

```ts
import * as z from 'zod';

const stringSchema = z.string(); // => ZodType<string>
const numberSchema = z.number(); // => ZodType<number>
const booleanSchema = z.boolean(); // => ZodType<boolean>
const undefinedSchema = z.undefined(); // => ZodType<undefined>
const nullTypeSchema = z.null(); // => ZodType<null>
```

### Parsing

```ts
// every ZodType instance has a .parse() method
const stringSchema = z.string();
stringSchema.parse('fish'); // => "fish"
stringSchema.parse(12); // throws Error('Non-string type: number');
```

### Type inference

Like in `io-ts`, you can extract the Typescript type of any schema with `{'z.TypeOf<>'}`.

```ts
const A = z.string();
type A = z.TypeOf<typeof A>; // string

const u: A = 12; // TypeError
const u: A = 'asdf'; // compiles
```

We'll include examples of inferred types throughout the rest of the documentation.

### Objects

```ts
// all properties are required by default
const dogSchema = z.object({
  name: z.string(),
  age: z.number(),
  neutered: z.boolean(),
});

type Dog = z.TypeOf<typeof dogSchema>;
/* equivalent to:
type Dog = {
  name:string;
  age: number;
  neutered: boolean;
} 
*/

const cujo = dogSchema.parse({
  name: 'Cujo',
  age: 4,
  neutered: true,
}); // passes, returns Dog

const fido: Dog = {
  name: 'Fido',
  age: 2,
}; // TypeError: missing required property 'neutered'
```

### Arrays

```ts
const dogsList = z.array(dogSchema);

dogsList.parse([{name: 'Cujo', age: 3, neutered: true}]); // passes
dogsList.parse([]); // passes
```

Plus you can explicitly define a non-empty array schema, something
`io-ts` doesn't support.

```ts
// Non-empty lists

const nonEmptyDogsList = z.array(dogSchema).nonempty();
nonEmptyDogsList.parse([]); // throws Error("Array cannot be empty")
```

### Unions (including nullable and optional types)

Zod includes a built-in `z.union` method for composing "OR"
types.

```ts
const stringOrNumber = z.union([z.string(), z.number()]);

stringOrNumber.parse('foo'); // passes
stringOrNumber.parse(14); // passes
```

Unions are the basis for defining nullable and optional values.

```ts
/* Optional Types */ // "optional string" === the union of string and undefined
const A = z.union([z.string(), z.undefined()]);
A.parse(undefined); // => passes, returns undefined
type A = z.TypeOf<typeof A>; // string | undefined
```

There is also a shorthand way to make a schema "optional":

```ts
const B = z.string().optional(); // equivalent to A

const C = z.object({
  username: z.string().optional(),
});
type C = z.TypeOf<typeof C>; // { username?: string | undefined };
```

```ts
/* Nullable Types */ const D = z.union([z.string(), z.null()]);

const E = z.string().nullable(); // equivalent to D
type E = z.TypeOf<typeof D>; // string | null
```

You can create unions of any two schemas.

```ts
/* Custom Union Types */
const F = z.union([z.string(), z.number()]).optional().nullable();
F.parse('tuna'); // => tuna
F.parse(42); // => 42
F.parse(undefined); // => undefined
F.parse(null); // => null
F.parse({}); // => throws Error!

type F = z.TypeOf<typeof F>; // string | number | undefined | null;
```

### Intersections

Intersections are useful for creating "logical AND" types.

```ts
const a = z.union([z.number(), z.string()]);
const b = z.union([z.number(), z.boolean()]);

const c = z.intersection(a, b);
type c = z.TypeOf<typeof C>; // => number

const neverType = z.intersection(z.string(), z.number());
type Never = z.TypeOf<typeof stringAndNumber>; // => never
```

This is particularly useful for defining "schema mixins" that you can apply to multiple schemas.

```ts
const HasId = z.object({
  id: z.string(),
});

const BaseTeacher = z.object({
  name: z.string(),
});

const Teacher = z.intersection(BaseTeacher, HasId);

type Teacher = z.TypeOf<typeof Teacher>;
// { id:string; name:string };
```

### Object merging

In the examples above, the return value of `z.intersection` is an instance of `ZodIntersection`, a generic class that wraps the two schemas passed in as arguments.

But if you're trying to combine two object schemas, there is a shorthand:

```ts
const Teacher = BaseTeacher.merge(HasId);
```

The benefit of using this shorthand is that the returned value is a new object schema (`ZodObject`), instead of a generic `ZodIntersection` instance. This way, you're able to fluently chain together many `.merge` calls:

```ts
// chaining mixins
const Teacher = BaseTeacher.merge(HasId).merge(HasName).merge(HasAddress);
```

### Tuples

These differ from arrays in that they have a fixed number of elements, and each element can have a different type.

```ts
const athleteSchema = z.tuple([
  // takes an array of schemas
  z.string(), // name
  z.number(), // jersey number
  z.object({
    pointsScored: z.number(),
  }), // statistics
]);

type Athlete = z.TypeOf<typeof athleteSchema>;
// type Athlete = [string, number, { pointsScored: number }]
```

### Recursive types

You can define a recursive schema in Zod, but because of a limitation of Typescript, their type can't be statically inferred. If you need a recursive Zod schema you'll need to define the type definition manually, and provide it to Zod as a "type hint".

```ts
interface Category {
  name: string;
  subcategories: Category[];
}

const Category: z.ZodType<Category> = z.lazy(() => {
  return z.object({
    name: z.string(),
    subcategories: z.array(Category),
  });
});

Category.parse({
  name: 'People',
  subcategories: [
    {
      name: 'Politicians',
      subcategories: [{name: 'Presidents', subcategories: []}],
    },
  ],
}); // passes
```

### Function schemas

Zod also lets you define "function schemas". This makes it easy to validate the inputs and outputs of a function without intermixing your validation code and "business logic".

You can create a function schema with `z.function(args, returnType)` which accepts these arguments.

- `args: ZodTuple` The first argument is a tuple (created with
  `z.tuple(\[...\])` and defines the schema of the arguments to
  your function. If the function doesn't accept arguments, you can pass an
  empty tuple (`z.tuple([])`).
- `returnType: ZodType` The second argument is the function's return type. This can be any Zod schema.

```ts
const args = z.tuple([
  z.object({nameStartsWith: z.string()}),
  z.object({skip: z.number(), limit: z.number()}),
]);

const returnType = z.array(
  z.object({
    id: string(),
    name: string(),
  })
);

const FetcherFactory = z.function(args, returnType);
```

`z.function` returns a higher-order "function factory". Every "factory" has `.validate()` method which accepts a function as input and returns a new function. The returned function automatically validates both its inputs and return value against the schemas provided to `z.function`. If either is invalid, the function throws.
This lets you confidently execute business logic in a "validated function" without worrying about invalid inputs or return types, mixing your validation and business logic, or writing duplicative types for your functions.
Here's an example.

```ts
const validatedQueryUser = FetchFunction.validate((filters, pagination) => {
  // the arguments automatically have the appropriate types
  // as defined by the args tuple passed to `z.function()`
  // without needing to provide types in the function declaration

  filters.nameStartsWith; // autocompletes
  filters.ageLessThan; // TypeError

  // Typescript statically verifies that value returned by
  // this function is of type { id: string; name: string; }[]

  return 'salmon'; // TypeError
});

const users = validatedQueryUser(
  {
    nameStartsWith: 'John',
  },
  {
    skip: 0,
    limit: 20,
  }
); // => returns { id: string; name: string; }[]
```

This is particularly useful for defining HTTP or RPC endpoints that accept complex payloads that require validation. Moreover, you can define your endpoints once with Zod and share the code with both your client and server code to achieve end-to-end type safety.

## Onwards and upwards

I think there's a LOT of room for improvement in how we handle the complexity of data modeling, validation, and transport as developers. Typescript, GraphQL, and Prisma are huge steps towards a future where our tooling can provide guarantees of data integrity. But there's still a long way to go.

If you like what you see, head over to <a href="https://github.com/colinhacks/zod">https://github.com/colinhacks/zod</a> and leave a star.

_I'm an open sourcer and ride-or-die TypeScripter. Put your email in the box to hear when I publish new essays or open source libraries._

There are a handful of wildly popular validation libraries in the Javascript ecosystem with thousands of stars on GitHub. But as I dug deeper into the many beloved tools in the ecosystem, I was surprised that none of them could provide the features and developer experience I was looking for. Until now.

Hi! I'm Colin.

Writing