Compile your snapshots: what four code generators and two production specs taught me

osury started as a narrow tool: point it at an OpenAPI spec, get ReScript types with Sury runtime schemas. It has since grown into a small compiler. The spec is parsed once into a typed AST, transforms extract unions, deduplicate structures and detect cycles, and an intermediate representation makes every decision as data. Four backends then print that IR: ReScript + Sury, Effect TS, OCaml and Rust.

This post is about the release where I checked whether those four targets were really equal, and about what happened when I stopped trusting fixtures and ran the generator on real specs.

What it generates

A discriminated union whose wire tag is not the case name:

"Pet": {
  "oneOf": [
    { "$ref": "#/components/schemas/Cat" },
    { "$ref": "#/components/schemas/Dog" }
  ],
  "discriminator": {
    "propertyName": "kind",
    "mapping": { "cat": "#/components/schemas/Cat", "dog": "#/components/schemas/Dog" }
  }
}

ReScript, with a Sury schema derived by the ppx:

@genType
@tag("kind")
@schema
type pet = @as("cat") Cat({
  name: string,
  lives: int
}) | @as("dog") Dog({
  name: string,
  owner: @s.null Nullable.t<string>
})

Rust, the same union as serde sees it:

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "kind")]
pub enum Pet {
    #[serde(rename = "cat")]
    Cat {
        name: String,
        lives: i64,
    },
    #[serde(rename = "dog")]
    Dog {
        name: String,
        owner: Option<String>,
    },
}

Effect TS maps the wire tag to _tag with Schema.decodeTo, and OCaml gets plain types with hand-written-style yojson codecs, no ppx. You can try all four on your own spec in the playground; it runs the same code as the CLI, in the browser.

Green tests, broken code

Every target had a golden test: a synthetic spec covering every construct, with the output compared byte for byte. All green. But only the ReScript output was ever compiled. When I ran the other goldens through their own toolchains:

• Rust: pub parent: Option<Node> — "recursive type has infinite size". The golden had been pinning code that does not build.
• Effect TS: Schema.Array(Node) inside Node's own initializer. The module crashed on import.
• Mutual recursion (A → B → A) worked in none of the four targets, ReScript included.
• A field-level union with a discriminator but no mapping used the schema name ("IssueLoop") as its tag, while the wire says "loop_found". In all four targets at once, because the bug lived in the shared IR.

A snapshot test answers "did the output change?". It says nothing about "is the output right?", and it guards broken code exactly as faithfully as correct code.

What replaced trust

There is now a parity test. The same spec, under every combination of flags, through every target, and the generated code must pass two checks:

1. build with the target's own compiler — rescript with sury-ppx, ocamlfind, cargo, tsc --strict;
2. decode sample data generated from the same spec, one sample per schema.

The second check turned out to matter as much as the first. It caught bugs in the sample generator itself, and it is the reason a "compiles fine" type cannot quietly reject every real value.

Each language now breaks a reference cycle its own way, and that is a decision in the IR rather than an accident of printing:

Then I pointed it at production

A synthetic spec only contains what you thought to put in it. So for 2.11 I ran the same check — four targets, four flag combinations, compile and decode — against two production specs from the product osury was built for, about 1,100 types in total. It found three bugs no fixture had:

• Literal[...] | str. Pydantic writes a string field that suggests some values but accepts any as anyOf: [enum, string]. osury turned it into a tagged variant expecting {"_tag": "String", ...}, so every real value failed to decode. The literals add nothing a string does not accept, and it is now a plain string in every target.
• An enum value "3pl". It is all letters and digits, so it passed the identifier check, yet a Rust identifier cannot start with a digit and the crate did not build. Variants now get a V prefix (OCaml already did this) and #[serde(rename)] keeps the wire value.
• A _tag with a default. Pydantic gives the tag a default, so it is not in required[], but it is always sent. The sample generator skipped it, and its samples were ones the real API never produces.

An earlier pilot on another live contract had already found an enum value "open", which ReScript only accepts as #"open". Both specs now compile and decode every sample in every target.

Also in this release

Stable Sury 11. In Sury 11.0.0 a format schema such as S.uuid decodes to its own branded type instead of string. osury now prints @s.matches(S.uuid->S.to(S.string)) string under --refinements: the format is still checked and the field stays a plain string.

--exact-optional. A field that is neither required nor non-null has three wire states: a value, null, or no key at all. For decoding a response that rarely matters; for a PATCH body it is the whole point, because leave it, clear it and set it are three different instructions. Under the flag each target spells it natively:

Errors that say where, what and how to fix. Every error carries a JSON path, a typed kind and a hint, and the parser collects all of them instead of stopping at the first.

The takeaway

It is not really about osury. If you maintain a code generator with snapshot tests, compile your snapshots, at least once. And then run them against a spec you did not write.

npm i -D osury
npx osury openapi.json src/Api.res
npx osury openapi.json --target rust -o src/api.rs

Source on GitHub, package on npm, playground at osury.coreofkeen.com. Thanks to Dmitry Zakharov for Sury, which all of this started from.