One complete, provenance-preserving analysis

In this lesson you will carry one polymer film through measurement, transformation, correction, replication and a claim, then save and reopen the whole analysis. By the end you will have an archive preserving the history and a CSV of the final mean. Allow about fifteen minutes to run the example and follow the relationships it keeps.

Download: one-complete-analysis.ipynb

Use an environment containing MTRL, as described in lesson 1. The complete example needs only the base package. Plotting is optional and is covered separately in lesson 3.

Follow one sample from the first measurement

The earlier lessons built one continuous session, adding a little new work to the same Sample each time. Here the whole story appears again, broken into short stages so each block stays small enough to read at a glance. Run the blocks in order — each one carries straight on from the last, exactly like the session you followed.

That is also how real work accumulates: a mistake is discovered midway, and later results need to say which values they used. The spectrum is still synthetic. Its initial narrow peak stands in for the detector mistake, while the corrected broad peak stands in for the accepted values for that same measurement. The later polymer label is a hypothetical operator statement, not a conclusion calculated from these curves.

Write the two scientific choices

Two functions express your choices before anything is recorded: making the synthetic spectrum and normalising its peak. Everything after them records results, their exact inputs and the operator’s assertion.

from pathlib import Path
from tempfile import TemporaryDirectory

import numpy as np

from mtrl import Sample


def make_spectrum(scale=1.0, *, corrected=False):
    axis = np.linspace(2000.0, 3000.0, 101)
    data = np.empty(axis.size, dtype=[("wavenumber", "<f8"), ("absorbance", "<f8")])
    data["wavenumber"] = axis
    data["absorbance"] = 0.015 * (axis - 2000.0) / 1000.0
    if corrected:
        data["absorbance"] += scale * 0.6 * np.exp(-((axis - 2450.0) / 60.0) ** 2)
    else:
        data["absorbance"] += scale * np.exp(-((axis - 2500.0) / 35.0) ** 2)
    return data


def normalise_peak(data, *, target=1.0):
    peak = float(data["absorbance"].max())
    if not np.isfinite(peak) or peak == 0.0:
        raise ValueError("absorbance has no finite, non-zero maximum")
    output = data.copy()
    output["absorbance"] *= target / peak
    return output

Record, transform, and correct the first measurement

Record the raw spectrum and normalise it. Then the detector mistake comes to light, so you correct the record — the checkpoint keeps its identity, but the version changes — and normalise the corrected values. raw stays readable afterwards; its first normalisation stays linked to the original values.

units = {"wavenumber": "cm^-1", "absorbance": "a.u."}
sample = Sample("polymer-film-01")
initial_revision_count = len(sample.core.revisions)

raw = sample.record(make_spectrum(), name="ftir-replicate-01", units=units)
normalised_old = sample.transform(
    raw, normalise_peak, name="normalised-before-correction"
)
corrected = sample.correct(raw, make_spectrum(corrected=True))
normalised = sample.transform(corrected, normalise_peak, name="normalised")

assert corrected.checkpoint_id == raw.checkpoint_id
assert corrected.record_id != raw.record_id
assert sample.select(record_id=raw.record_id) == ()
assert sample.core.get_record(raw.record_id) is not None
print(f"original peak: {round(float(raw.to_numpy()['absorbance'].max()), 4)}")
print(f"normalised peak: {float(normalised.to_numpy()['absorbance'].max())}")

original peak: 1.0075
normalised peak: 1.0

The correction kept the checkpoint identity, changed the record version, and left the old record readable — the quiet asserts above confirm all three. The corrected values normalise to the target peak, while the original peak keeps its own value.

Add the replicates and the mean

Two more films are measured, and the mean combines the corrected first replicate with the two new ones. Every member is an exact recorded version, not a numerically similar curve.

replicate_02 = sample.record(
    make_spectrum(0.95, corrected=True), name="ftir-replicate-02", units=units
)
replicate_03 = sample.record(
    make_spectrum(1.05, corrected=True), name="ftir-replicate-03", units=units
)
members = [corrected, replicate_02, replicate_03]
mean = sample.aggregate(
    members,
    axis="wavenumber",
    value="absorbance",
    method="mean",
    name="mean-spectrum",
)

assert tuple(mean.checkpoint.lineage.input_refs) == tuple(item.ref for item in members)
print(f"mean peak: {round(float(mean.to_numpy()['absorbance'].max()), 5)}")

mean peak: 0.60675

The mean’s recorded inputs are exactly the three handles you passed, in your order.

Record the operator’s claim

The physical sample gets a portable identity, then the operator’s statement about the polymer. Both commits count against the running total.

sample.core.assign_sample_uid()
claim = sample.claim("polymer.identity", "PET")

print(f"commits added: {len(sample.core.revisions) - initial_revision_count}")
print(f"active records: {len(sample.records)}")
print(f"claims: {len(sample.claims(property_id='polymer.identity'))}")

commits added: 9
active records: 6
claims: 1

Nine commits cover the first recording, the first normalisation, the correction, the new normalisation, two more recordings, the mean, the UID assignment and the claim. Selection, reading claims, saving, verification, loading and exporting add no revisions: they inspect or share the committed work without inventing new scientific results.

Recap the state and the history

Before saving, two views recap what the sample holds. The summary answers “what is here now”; the history answers “how did it get here”.

print(sample.summary())
print(sample.history())

Sample 'polymer-film-01' — 6 active records; 1 active claim
Records (commit order):
  [0] 'normalised-before-correction' 101 values, shape=(101,)
      fields: wavenumber (cm^-1), absorbance (a.u.)
  [1] 'ftir-replicate-01' 101 values, shape=(101,)
      fields: wavenumber (cm^-1), absorbance (a.u.)
  [2] 'normalised' 101 values, shape=(101,)
      fields: wavenumber (cm^-1), absorbance (a.u.)
  [3] 'ftir-replicate-02' 101 values, shape=(101,)
      fields: wavenumber (cm^-1), absorbance (a.u.)
  [4] 'ftir-replicate-03' 101 values, shape=(101,)
      fields: wavenumber (cm^-1), absorbance (a.u.)
  [5] 'mean-spectrum' 101 values, shape=(101,)
      fields: wavenumber (cm^-1), absorbance (a.u.)
Sample 'polymer-film-01' — record history (6 active records, commit order)
  [0] 'normalised-before-correction'
      step: transformation custom:normalise_peak
      inputs: 'ftir-replicate-01' (original version, corrected)
  [1] 'ftir-replicate-01'
      step: recorded values (no producing step)
      note: corrected version; corrects the original record
  [2] 'normalised'
      step: transformation custom:normalise_peak
      inputs: 'ftir-replicate-01' [1]
  [3] 'ftir-replicate-02'
      step: recorded values (no producing step)
  [4] 'ftir-replicate-03'
      step: recorded values (no producing step)
  [5] 'mean-spectrum'
      step: aggregation mean of 3 members
      inputs: 'ftir-replicate-01' [1], 'ftir-replicate-02' [3], 'ftir-replicate-03' [4]

Read the history against the work above: the old normalisation still names the stale original version, the correction note marks the superseded record, and the mean lists the exact three record versions it combined — the corrected first replicate, not the normalised result.

File output stays in a temporary directory so you can rerun the example without overwriting earlier work. Keep the save, reload and export together as shown; the loaded history’s backing archive stays available until that directory closes.

with TemporaryDirectory() as directory:
    root = Path(directory)
    archive_path = sample.save(root / "polymer-film-01.mtrl")
    result = sample.verify()
    print(result.format_summary())

    reloaded = Sample.load(archive_path)
    restored_mean = reloaded.one(name="mean-spectrum")
    assert restored_mean.ref == mean.ref
    assert len(reloaded.core.revisions) == len(sample.core.revisions)
    assert len(reloaded.claims(property_id="polymer.identity")) == 1
    assert reloaded.core.get_record(raw.record_id) is not None
    assert tuple(restored_mean.checkpoint.lineage.input_refs) == tuple(
        item.ref for item in members
    )

    csv_path = reloaded.export(restored_mean, root / "mean-spectrum.csv")
    exported = np.genfromtxt(csv_path, delimiter=",", names=True)
    assert np.allclose(exported["absorbance"], restored_mean.to_numpy()["absorbance"])
    print("archive and CSV:", archive_path.is_file(), csv_path.is_file())
    print("exact mean restored:", restored_mean.ref == mean.ref)

structural: valid
hash: consistent
anchor: no-receipt
coverage: full
source closure: not-applicable
evidence availability: complete
issues: 0
archive and CSV: True True
exact mean restored: True

Verification prints its separate facts, including structural valid, hash consistent, anchor no-receipt and coverage full. The final lines report that the archive and the CSV exist, and that reloading restored the mean’s exact references, not merely a numerically similar curve. The temporary directory disappears when the example finishes. In a real analysis, choose an archive destination you retain and back up. Share the archive when a colleague needs the history, and the CSV when their tool needs a table of values.

What just happened?

One sample kept the mistaken measurement and its correction, plus both normalisations. The old normalisation still names the old input; the new one names the correction. The mean combines the corrected first replicate with the two other exact replicate versions. You can explain all of those relationships from the archive, without remembering which notebook cells happened to run first.

The claim sits beside that measurement history as an operator statement about the physical sample. It is neither another spectrum nor an integrity verdict. Reloading preserved its identity and the result relationships, and let you read the stored results without executing the custom function again.

Where next

You have followed the whole path once. From here, choose the guide that fits your work:

Prepare a machine-learning dataset — assemble saved archives into aligned features and claim-derived targets, with grouped splits and provenance.
The concept pages — understand exact identity, immutable commits, lineage, revision history and the limits of verification.