How the IRIS catalogue is compiled into named graphs with Oxigraph,
checked, published, and queried. How to build and query it is in the
iris-oxigraph skill.
[!IMPORTANT] Binary assets are never embedded in the graph. PDFs, image bitstreams and covers stay on the CDN. The graph holds typed entities, Dublin Core values, containment, and each bitstream's file name, media type, byte size and copyright verdict.
The pipeline is modelled in the cat-harness BPMN process files, which are the source of truth:
Package and distribute partitioned N-Quads (spine, community partitions,
manifest, prepared queries):
processes/kg/nquads-distribution.bpmn
Run a prepared SPARQL query in Oxigraph (browser WASM or local engine):
processes/kg/named-query-execution.bpmn
How this instance feeds the pipeline from IRIS and publishes to Cloudflare is
in IRIS CDN Exporter: processes; its nightly full
rebuild calls nquads-distribution.
catalogue/nodes/*.json (communities, collections, items, bitstreams) and
site/dublin-core/*.dc.jsonld (Qualified Dublin Core records conforming to
dublin-core.ts); or--records <dir>: DSpace records exported by scripts/iris-cdn/run.ts.scripts/build-iris-oxigraph.ts (--out <dir>, default
dist/oxigraph/).https://iris.who.int/entity/item/{handle}#{prop}_{idx}) for every
anonymous compound Dublin Core object. The build fails if any blank node
(_:) remains.<https://iris.who.int/graph/spine> — hierarchy,
item Handles, title, first creator, issued date, and bitstream
descriptors with copyright verdicts. who-iris-spine.nq and .nq.gz.<https://iris.who.int/graph/community/{id}> —
the full Dublin Core of each community's items (MeSH authorities,
abstracts, spatial tags, govdoc, ISBN).
iris_who_int_graph_community_{id}.nq and .nq.gz.iris_who_int_graph_catalogue.nq(.gz) and
iris_who_int_graph_metadata.nq(.gz), merged in who-iris-dataset.nq
and .nq.gz.subgraph-manifest.json (partition
registry) and queries.json (prepared SPARQL).bun run scripts/build-iris-oxigraph.ts
bun test scripts/tests/iris-oxigraph.test.ts
The test builds the dataset and checks: zero blank nodes; multi-condition
search, date-range search and facet counts; additive community mounting; the
discoverySearch prepared query; that the manifest's partition files exist;
and that a rebuild over exported records leaves unchanged partitions
byte-identical.
Not yet checked by any test: every item declares a collection and community
path; every bitstream declares bytes and media type; every item has a
10665/... Handle.
Exported data is published to Cloudflare: R2 behind the Cloudflare CDN, with release history in a git repository on Cloudflare Artifacts (see IRIS CDN Exporter: deployment).
scripts/iris-cdn/run.ts rebuilds the dataset over the exported records and
lists each output file in the release manifest as graph/<file>:
who-iris-spine.nq.gz (Tier 1 spine)subgraph-manifest.json (partition registry)iris_who_int_graph_community_{id}.nq.gz (Tier 2 partitions)who-iris-dataset.nq.gz (merged graph)queries.json (prepared queries)sha256/<aa>/<digest>.Cache-Control: public, max-age=31536000, immutable on
sha256/*. A changed file has a new digest, so a cached object never goes
stale.The intended client pattern (no page in this repository loads Oxigraph yet; the replica's home page filters an index embedded in the page with plain JavaScript):
const store = new oxigraph.Store();who-iris-spine.nq.gz) and subgraph-manifest.json.store.load(decompressedNq, { format: 'application/n-quads' })..nq.gz partition from the
manifest and store.load() it; existing quads are kept.| ID | Requirement | Verification Method |
|---|---|---|
| SR-1 | No Binary Embeddings in Graph: No RDF literal or URI may represent raw file bytes (base64 or blob data). Bitstreams are nodes reached by dspace:hasBitstream, with fileName, fileBytes (integer) and mediaType. | Not automated. |
| SR-2 | Upstream Skolemization at the Extractor (Zero Blank Nodes): Extractors and ingestion pipelines assign deterministic URIs (https://iris.who.int/entity/item/{handle}#{prop}_{idx}) at extraction for all compound nodes (creators, subjects, dates). No blank node (_:) may be emitted. | The build throws on any residual blank node; iris-oxigraph.test.ts asserts none in the dataset. |
| SR-3 | Multi-Graph Isolation: Catalogue containment and rights gates reside in <https://iris.who.int/graph/catalogue>; bibliographic records in <https://iris.who.int/graph/metadata>. | Not automated. The catalogue graph also carries rdf:type, rdfs:label and dcterms:title. |
| SR-4 | Sub-50ms Query Execution: Standard discovery and complex search queries over the materialized dataset complete in under 50 milliseconds in WASM. | Not measured by any test. |
| SR-5 | Additive Subgraph Mounting: store.load() of an additional named subgraph must not clobber, mutate, or duplicate loaded quads. | iris-oxigraph.test.ts: a query before mounting returns N items; after mounting a community, N + M. |
| SR-6 | Handle Authority: Every item subject URI uses the Handle system: https://hdl.handle.net/10665/{id}. | Graph check: every a dspace:Item subject starts with https://hdl.handle.net/. |
| SR-7 | Compressed Transport Ceiling: At full scale (300,000 items), a community partition must not exceed 25 MB compressed. | Not measured in CI. |
| SR-8 | Prepared Query Equivalence: Every entry in queries.json is valid SPARQL 1.1 and runs against the dataset with 0 errors. | iris-oxigraph.test.ts runs discoverySearch only. |
| SR-9 | Append-Only Streaming Partitioning: The build writes partitions as an append-only stream, with memory bounded regardless of corpus size (300,000 items). | Not met: buildIrisDataset holds and sorts every quad in memory; StreamingPartitionWriter is unused. Target: heap under 250 MB for the whole corpus. |
WHO IRIS holds 273,559 items across 1,057,223 files, about
361.55 GiB of PDFs, reports and images (see
iris-dspace §Scale). The figures below are
estimates for about 300,000 items.
| Metric | Raw Full-Text in Oxigraph | Pure KG Metadata Extract (Architecture Standard) |
|---|---|---|
| Content Included | Full OCR extracted page text + bibliographic metadata | Dublin Core, MeSH authorities, bitstream descriptors, access gates, hierarchy paths |
| Binary Blobs in RDF | 0 (strictly excluded) | 0 (strictly excluded) |
| Quads / Triples | ~45,000,000 quads (including sentence/paragraph chunks) | ~12,000,000 quads (~40 quads per item) |
| Uncompressed N-Quads | ~45 – 60 GB | ~1.2 – 1.5 GB |
| Brotli / Gzip Payload | ~8 – 12 GB compressed | ~95 – 140 MB compressed (>90% compression ratio) |
| Partition Size (per Community) | ~1.2 – 1.8 GB compressed | ~12 – 20 MB compressed |
| WASM In-Memory RAM | > 4 GB (Exceeds 32-bit WASM limit → Browser Crash) | ~35 – 65 MB RAM per loaded partition |
| Load Time over CDN (4G/Fiber) | Impossible in browser | 150 ms – 350 ms |
| Query Latency | Slow regex/filter scans (> 5,000 ms) | < 15 ms index-backed graph pattern matching |
VALUES ?item { ... } for facets, MeSH
hierarchies, collection constraints and copyright gates.scripts/build-iris-oxigraph.ts: N-Quads compiler, skolemizer and gzip
writer.scripts/iris-oxigraph-search.ts: IrisOxigraphEngine — boolean search,
facet counts and incremental subgraph mounting.scripts/gen-iris-pages.ts: static site generator; the home page's facet
chips and filters run over an index embedded in the page, not Oxigraph.bun test scripts/tests/iris-oxigraph.test.ts.bunx playwright test cat-harness-tools/test/who-iris-search.e2e.ts
(browser search, facet chips, open-access filters, multilingual UI).skills/iris-oxigraph.md: requirements OX-1 through OX-9.docs/oxigraph-pipeline-requirements.md: system requirements SR-1 through
SR-9.