---
title: "llms.txt and the shards"
description: "The curated index a model should read first, the four topic shards that exist because context windows are finite, and the size budget that governs them."
url: "https://opsinjs.pensievelabs.org/agents/llms-txt"
source: "https://opsinjs.pensievelabs.org/agents/llms-txt.md"
section: "Agents & automation"
kind: "guide"
reviewed: "2026-09-20"
reviewer: "engineering"
aliases: ["llms.txt", "llms-full.txt", "shards", "context budget"]
---
> Elements written as `` below are opsinjs documentation
> components. Their attributes are the content: the values they render are
> generated from `tokens/*.json` and `registry/catalogue.ts` and are
> published separately at https://opsinjs.pensievelabs.org/r/index.json and under the Reference
> section.
> Nothing is missing from this page. The data simply does not live in
> the prose.
## Overview [#overview]
[`llms.txt`](https://llmstxt.org/) is a convention: a single markdown file at the
root of a site that tells a model what the site contains and where to find it,
in a form that fits in a prompt. opsinjs publishes one, plus a full corpus dump,
plus four topic shards.
The reason for six files rather than one is arithmetic. This corpus is large
enough that pasting all of it into a prompt wastes most of a context window on
pages irrelevant to the task, and a model that has read everything about
typography and nothing about clinical status will still get the status wrong.
Shards let a caller spend context where it changes the answer.
## The six files [#the-six-files]
| Path | Contains | Use it when |
| ------------------------------------------------ | ----------------------------------------------------------------------------------------- | ------------------------------------------ |
| [`/llms.txt`](/llms.txt) | Curated index: one line per page, grouped under `##` section headers, with absolute URLs | Always. Start here |
| [`/llms-full.txt`](/llms-full.txt) | The whole corpus as one document, size-capped | You have a large window and a broad task |
| [`/llms-components.txt`](/llms-components.txt) | Component specifications and screens | Generating or reviewing UI code |
| [`/llms-health.txt`](/llms-health.txt) | Health doctrine, accessibility and content guidance | Anything with a clinical or copy dimension |
| [`/llms-foundations.txt`](/llms-foundations.txt) | Foundations and theming: what a token means and how to change it | Theming, contrast, motion, spacing |
| [`/llms-reference.txt`](/llms-reference.txt) | The generated reference: every token, CSS variable, measured pair, term and exported type | Resolving a name to a value |
Every URL in every one of these files is absolute. A relative URL in an index is
useless to a client that fetched the file from somewhere else and no longer
knows where it came from. That is the normal case for a model.
The four topic shards do not cover the whole tree, so "use the shards" misleads
for the half they miss. Between them they carry eight of the sixteen sections:
Components, Screens, Health, Accessibility, Content & language,
Foundations, Theming & tokens and Reference. Start here, Recipes, Patterns,
Handbook, Agents, Registry & distribution, Project and the introduction have no
shard, and neither do the package pages, which sit outside the sixteen groups.
`/llms.txt` names the sections with no shard rather than leaving you to work it
out, and for a page in one of them the markdown twin is the route: one request,
exactly the page you asked for.
## What is in the index [#what-is-in-the-index]
`/llms.txt` is generated from the page tree, so it cannot drift from the site.
Each entry carries the page title, its one-sentence description, and its search
aliases, because an alias is what a model was actually thinking of. Sections
match the sidebar's sixteen groups.
A component page's status is included too, and it is the most load-bearing field
in the file. Sixty component ids are built, and a built id has source in the
registry item and an API that may change in any release. Every one of the sixty
has been audited against WCAG 2.2 AA, but that audit was run by the authors of
the components, not by an independent reviewer, and no component has had a
clinical review; see
[ADR 0025](../project/decisions/0025-the-audit-is-author-run.mdx) for the
record. `shipped` means the source installs and passed that author-run audit,
and it does not mean either an independent or a clinical review has happened.
An index that did not carry the field at all would be an invitation to import a
specification. Each
entry is one line, and the tail in brackets carries the status where there is
one, the kind and the aliases:
```txt
## Components
- [Card](https://opsinjs.pensievelabs.org/components/card): A bounded group of related content with a border and consistent padding is the ordinary box that most of a screen is made of. (status: shipped · kind: component · also known as: panel, container, box, tile group, content box)
- [Select](https://opsinjs.pensievelabs.org/components/select): Choosing one option from a list. (status: shipped · kind: component · also known as: dropdown, picker)
```
## The size budget [#the-size-budget]
Two limits, both applied by the generator, and both now reported by
`pnpm run check:llms` when it is pointed at a running server.
**`/llms-full.txt` is capped.** When the corpus outgrows the cap the generator
keeps whole pages, in the order the sidebar presents them, and stops when the
next one would not fit. The generator never truncates mid-page. A markdown
document ending in the middle of a table is worse than a shorter complete one,
because a model will happily finish the table from imagination. The header
counts what survived (`Pages: N of M.`) and a `## Truncated` block at the foot
of the file counts what was dropped, names the shards to fetch instead, and then
lists every dropped page by title with its `.md` URL and its status. Nothing is
hidden: the manifest is budgeted for before any page is, and measured over the
whole corpus so the reservation can never be an underestimate. That costs a few
per cent of the cap, in exchange for a file that never drops a page without
naming it.
**Every shard carries its sections whole, and a shard that does not fails the
build.** That was not true until recently. At a 400 kB cap the three shards
carried 65%, 46% and 32% of their own sections, which is not a smaller answer
but a quieter wrong one: ask for the health doctrine, get half of it, with no
way to know which half. A shard at its budget is a signal that it needs
splitting rather than trimming, so it was split. Reference left the foundations
shard and became a shard of its own, because Reference is 899 kB of which eight
generated table pages are 800 kB, and a combined shard therefore spent its whole
budget on tables and truncated before it reached the doctrine it was named for.
The four shards are now 599, 630, 576 and 899 kB against a 1.2 MB cap.
Run against a server, `check:llms` reads the `Pages: N of M.` line out of every
corpus file, and a shard that dropped even one page is a build failure naming
the count and the coverage. It fails separately when an implemented component is
missing from `/llms-components.txt`. That second check matters more than it
sounds, because truncation drops pages in corpus order, so the components that
fall off are the ones at the end rather than a random sample.
`/llms-full.txt` is the exception and is reported rather than failed. It is
capped on purpose: the whole corpus is 3.8 MB, which is more context than most
readers of that file have, so the cap is what makes the file usable at all. It
still names every page it dropped, and `check:llms` fails if it ever drops one
without printing that manifest.
If you need a specific page rather than a shard, fetch its markdown twin. See
[Raw markdown API](./raw-markdown-api.mdx). Fetching one page is almost always
cheaper than fetching a shard and hoping the page is in it.
## Verify it worked [#verify-it-worked]
### The index parses as markdown [#the-index-parses-as-markdown]
```bash
curl -s https://opsinjs.pensievelabs.org/llms.txt | head -40
```
Section headers, list items, absolute URLs, no HTML.
### A URL from the index resolves [#a-url-from-the-index-resolves]
Take any link from the file and fetch it with `.md` appended. If the twin 404s,
the index and the router disagree and `check:llms` should have caught it.
### The shards partition the corpus [#the-shards-partition-the-corpus]
The four topic shards plus the pages that are in none of them should account
for every page in the tree. `check:llms` asserts this; a page that belongs to no
shard is usually a page in a group nobody assigned.
## Troubleshooting [#troubleshooting]
**`/llms-full.txt` is missing pages I can see on the site.** The cap. The header
counts how many pages survived and the `## Truncated` block at the foot lists
every page that was dropped, with the `.md` URL to fetch it from. Read that
block first rather than diffing the file against `/llms.txt`. Then fetch the
twin, or take the shard that covers its section, remembering that eight of the
sixteen sections have no shard at all.
**A page's aliases are missing from the index.** Aliases come from frontmatter.
A page with none contributes none. That is a documentation gap, reportable as a
docs issue.
**The index is stale after I added a page.** It is generated at build time.
Rebuild, or in development restart. `pnpm run generate` does not rebuild the
index on its own.
**I want an index of only the built components.** Filter the Components section
on `status: shipped`; there are sixty in total. `/r/index.json` answers the same
question in a field named `implemented` and is cheaper to parse than a text
index.
## Next [#next]
* [Raw markdown API](./raw-markdown-api.mdx) covers fetching one page instead of
a shard.
* [Offline docs bundle](./offline-docs-bundle.mdx) is the same corpus as one
versioned JSON file.
* [Writing docs for agents](./writing-docs-for-agents.mdx) says what makes a
page survive the trip into an index entry.