Generated from who-iris/processes/iris-cdn-initial-load.bpmn by gen-processes-viz.ts — do not edit here. All processes · who-iris processes

IRIS CDN initial load

Process_IrisCdnInitialLoad · strict (defaulted) · 10 step(s)

The first full export of WHO IRIS onto Cloudflare (git on Artifacts, bytes in R2): every item record and every bitstream: metadata plus PDFs. Run in batches with a checkpoint, so a run that stops resumes where it stopped and never re-fetches what it already holds. Ends in one signed, verified release.

BPMN diagram: IRIS CDN initial load

How it connects

Lanes — who acts

lanerolewhat it does here
Owneriris-cdn-ownerA person: decides scope, budget and the storage target. Nothing downstream widens what the owner approved.
IRIS CDN Exporter (agent)iris-cdn-exporterThe agent role. It plans, reviews and reports; it does not hold the signing key, and it never pushes to a WHO repository.
Build pipelinebuild-pipelineDeterministic scripts in scripts/iris-cdn/, entry point run.ts. Same input, same bytes.

Steps

3 of 10 step(s) carry no documentation — activity-documented lists them.

steplaneskill / sub-processwhat it does
State scope and budget, with denominators
Task_StateBudget
Owneriris-cdn-exporterItems, files and bytes upstream (273,559 items, 1,057,223 files, 361.55 GiB as last transcribed), the storage target for each kind of file, and a request rate IRIS will tolerate. A budget with no denominator is a preference, not a limit.
Plan batches by community and collection
Task_PlanBatches
IRIS CDN Exporter (agent)iris-cdn-exporter
iris-dspace
One batch per collection (large collections split by page).
Harvest a batch of item records (REST)
Task_HarvestBatch
Build pipelineiris-rest-feed
iris-dspace
DSpace REST: the item, its metadata, bundles and bitstream descriptors. Written as canonical JSON (sorted keys, LF, no fetch timestamp inside the record) so the same item always serialises to the same bytes.
Fetch bitstreams not already held
Task_FetchBitstreams
Build pipelineiris-cdn-exporterA bitstream is stored under its SHA-256. If that object already exists it is not downloaded again; a re-run of a batch therefore moves no bytes.
Store bytes under SHA-256 (skip if held)
Task_StoreBytes
Build pipelineiris-cdn-exporter—
Commit the batch checkpoint
Task_Checkpoint
Build pipelineiris-cdn-exporterstate/initial-load.json in the Cloudflare git repository. A restarted load reads it and skips finished batches.
Quarantine and report the mismatch
Task_Quarantine
Build pipelineiris-cdn-exporterNot published, not retried silently. The record keeps its reference to IRIS and the bitstream is marked unknown.
Build the graph partitions
Call_BuildGraph
Build pipelinecalls Package and distribute partitioned W3C N-Quads and named queries
iris-oxigraph
nquads-distribution
The platform process nquads-distribution (cat-harness): spine plus one partition per community, zero blank nodes, sorted N-Quads.
Sign and publish the release
Call_SignPublish
Build pipelinecalls Sign and publish an IRIS CDN release
iris-cdn-signing
—
Review the load report; open beans for gaps
Task_ReviewLoad
IRIS CDN Exporter (agent)iris-cdn-exporter—

Decisions

1 of 2 decision(s) carry no documentation — gateway-documented lists them.

decisionwhat decides itbranches
Bytes match the IRIS checksum?
GW_Fixity
DSpace reports an MD5 checksum per bitstream. The downloaded bytes must match it before their SHA-256 is recorded.yes → Store bytes under SHA-256 (skip if held)
no → Quarantine and report the mismatch
More batches?
GW_More
—yes → Harvest a batch of item records (REST)
no → Build the graph partitions

folio-assistant — code under the Apache License 2.0, documentation under CC BY 3.0. Generated schema and API references are rebuilt from source on every push to main.

deployed dev