Lineage and the recorded recipe

Think of a recipe card sealed inside the box with the cake. The card lists the ingredients, the oven temperature, and even which batch of flour went in, so anyone who opens the box can see exactly how the cake was made. Crucially, there is no live chef hiding inside: the card describes the cooking, but it cannot cook. Every derived record in MTRL carries exactly such a card — its recipe — sealed in with the result itself.

One producing step per record

Every derived record carries a lineage: a record of when the checkpoint was created, which checkpoint is its parent, and the one step that produced it.

A lineage records exactly one producing step: a transformation (an operation that turns one record into another, such as normalisation) or an aggregation (an operation that combines several records, such as a mean) — never both. A raw checkpoint, like the spectrum straight from ftir-replicate-01, has no parent and no producing step at all.

The lineage also names its exact inputs, in order, as RecordRefs — the paired checkpoint and record identities from Two kinds of identity. A transform records one RecordRef for its source. An aggregate records one RecordRef for each selected member, in selection order. So the normalised spectrum from lesson 4 names the exact raw record it consumed, and the mean over ftir-replicate-01 to ftir-replicate-03 names each replicate record precisely — never a bare checkpoint name that could silently drift to a newer version.

Corrections and replacements get their own slots. When a record is a correction, correction_of holds the exact prior RecordRef; when it is a replacement, replacement_of does. A lineage cannot hold both. These relationships are independent of the producing step. One honest limit: a correction that supplies hand-built data, like the lesson 5 correction, records what it corrects but no operation recipe, because no operation produced the new values. Only records an operation produced carry the full recipe described below.

You can read those edges for one sample without inspecting the archive internals: print(sample.history()) lists each active record’s producing step and, one hop only, the exact versions it consumed, marking a version that a later correction superseded — see one complete analysis.

The provenance envelope

The lineage says which step ran. A second, richer object says everything about how it ran: the OperationProvenance envelope, a frozen bundle of data attached to the record. It holds:

• the qualified operation identifier and its contract version;
• the implementation package name and version;
• the effective parameters, including defaults you never typed;
• the ordered RecordRef inputs;
• the input and output schemas and units;
• the execution status, success or failed;
• the installed dependency versions the operation actually used;
• a canonical digest of the output payload — a fingerprint binding the recipe to the exact values, shape, and dtype it produced.

Data only, never code

The envelope’s most important property is what it refuses to contain. It holds data only: no import path, no callable, no pickle, no executable source. An archive is evidence, not a programme — opening one can never run code. This is enforced, not merely promised. Callables, custom objects, and non-string mapping keys are rejected when the envelope is built, and every value is validated, detached from the caller’s objects, and frozen at construction. Once sealed, a recipe cannot change after the fact.

The same caution governs the future. When a newer version of MTRL writes provenance fields your version does not know, loading preserves them as inert data. They are never resolved, imported, registered, or executed — they ride along, untouched, for a future reader.

Why it matters

Because the recipe travels with the record, “how did you get this number?” is always answerable from the record itself. You do not need to trust a lab notebook, a colleague’s memory, or a script that has since been edited. The record says which operation ran, with which parameters, on which exact inputs, and whether it succeeded.

The recipe is not only for reading. It is exactly what the replay feature consumes later: the operation identity, the effective parameters, the exact input references, and the recorded output digest to compare against. The how-to guides cover replay in detail.

Where next

Each sealed recipe becomes one step in a longer history — Revisions: the archive timeline shows how those steps accumulate into a verifiable archive. To record a recipe of your own, revisit Transform and plot a spectrum.