Generated from who-iris/skills/iris-cdn-exporter.md — do not edit here.

✎ Edit this page’s source

The IRIS CDN Exporter

The exporter is an agent role plus a set of deterministic scripts. The scripts (scripts/iris-cdn/) move bytes. The agent plans, reviews and reports. It never holds the signing key and never pushes to a WHO repository.

Processes, all in processes/:

ProcessWhenWhat
iris-cdn-initial-load.bpmnonce, run by hand with a stated budgetevery record and every bitstream, in resumable batches
iris-cdn-nightly.bpmnevery night, 02:00 UTConly the items IRIS reports as changed
iris-cdn-reconcile.bpmnSundays, 04:00 UTCcatches withdrawals, which REST does not report, and checks for drift
iris-cdn-sign-publish.bpmncalled by the three aboveBagIt bag, detached JWS (ES256), did:web key, verify

What each process does is in docs/iris-cdn-export.md; where it runs, what it writes where, and its secrets are in docs/iris-cdn-deployment.md; signing is skill iris-cdn-signing; reading changes from IRIS is skill iris-rest-feed.

The no-churn rule

Content that has not changed is not rewritten, re-uploaded or re-signed. Each step says how:

  1. Canonical records. A record is written as JSON with sorted keys, LF line endings and no fetch timestamp inside it. If the canonical bytes equal the held ones, the file is not rewritten.
  2. Content-addressed bitstreams. A bitstream is stored under its SHA-256. An object that already exists is never downloaded, uploaded or overwritten again.
  3. Sorted graph output. The graph is rebuilt in full, and N-Quads partitions are written sorted, so a partition with no changed item comes out byte-identical and is not re-uploaded.
  4. Date-free manifests. bag-info.txt has no Bagging-Date, so the tag manifest of an unchanged payload is byte-identical.
  5. An empty diff ends the run. No commit, no new signature.

A step that cannot meet its rule reports it. It never hides it.

What the agent does

  • Plan the initial load as one batch per collection page, with the checkpoint in state/initial-load.json in the Cloudflare git repository.
  • Review each run report and open a bean for each anomaly: a change count far from the recent median, a quarantined bitstream, a verification whose verdict is unknown, or a partition over its size ceiling.
  • Never widen scope. Only the owner changes what is exported, its budget or where it is stored.

What the agent must not do

  • Push to any WHO/* repository.
  • Treat a short listing as a mass withdrawal (see iris-rest-feed).
  • Delete a published object. A withdrawn item is marked withdrawn, and its bytes stay addressable because a citation to them may exist.
  • Put an upstream count into prose. Counts come from the catalogue and the run reports.

Processes that run this skill

processstep(s) that name it
IRIS CDN initial loadState scope and budget, with denominators; Plan batches by community and collection; Fetch bitstreams not already held; Store bytes under SHA-256 (skip if held); Commit the batch checkpoint; Quarantine and report the mismatch; Review the load report; open beans for gaps
IRIS CDN nightly change ingestFetch only bitstreams whose checksum changed; Write canonical records into the library; Review the run report; open beans for anomalies
IRIS CDN weekly reconciliationTreat the listing as faulty; escalate
Sign and publish an IRIS CDN releaseStore new objects (never overwrite)

folio-assistant — code under the Apache License 2.0, documentation under CC BY 3.0. Generated schema and API references are rebuilt from source on every push to main.

deployed dev