Generated from smart-base/L1/skills/smart-l1/indicator-extraction.md — do not edit here.

✎ Edit this page’s source

indicator-extraction

Skill id: indicator-extraction · Package: authoring-who-smart-guidelines · Follows dak-l1-library, which ingests the source. Sibling of recommendation-extraction: that one takes a source’s recommendations, this one its indicators.

It is a step of L1 → L2 extraction for a DAK or DTH, bound to Task_Indicators in l2-dak-authoring and Call_Data in dth-authoring.

Two layers, kept apart

layerwhatkind / modelwritten by
L1the indicator as the source prints it, verbatimindicator block (folio type smart-base:Indicator; KG L1 indicator, L1/schemas/l1.ts)L1/scripts/l1-indicator-sheets.py → candidates.json → gen-library-jsonld
L2the DAK’s or DTH’s own operational measure, a draft that points backprogram-indicator (sgkg-l2), logical model ProgramIndicatorscripts/l2-program-indicators.py

sgkg-l2 keeps them apart: merging them would merge a WHO definition with a country’s implementation of it. The L2 record is a draft with derivedFrom its L1 node, and the L2 author adopts, edits or drops it.

Rules

  • Ids come from the source. number / indicatorId is the printed indicator number (“1-01”, “4-01.3”). A sheet whose header prints none is reported and skipped, never numbered by an agent. The L2 id is <prefix>.<number> (“NHWA.1-01”); the prefix only namespaces it.
  • Verbatim. Every field is the printed text: lines joined, letter-spacing artefacts closed up, nothing reworded. A printed list (glossary terms, data sources) is one string joined with “; “.
  • Every printed field, not only name, numerator and denominator. The L1 node’s meta holds indicatorId, module, abbreviatedName, name, numerator, denominator, disaggregation, definition, glossary, reportingFrequency, reportingFrameworks, dataSources, furtherInformation and dimension where printed; a sub-indicator adds parentIndicator and guidingQuestions. These are the keys the structured-block cards read.
  • No gap is filled. ProgramIndicator makes definition, numerator, denominator and disaggregation 1..1. Where the source prints none (a composite or self-assessed indicator), the L2 draft lists it in gaps and the L2 author decides. Nothing is copied from another source to fill it.
  • A composite indicator keeps its sub-indicators as their own L1 nodes; in L2 they are the parent’s components, not program indicators of their own.
  • Read the page layout, not the section text, when the source prints sheets as a label column and a value column: Stage A’s text reads side bars and columns out of order.

Steps

  1. The source is in the library (dak-l1-library), with its PDF in uploads/<doc-id>/.
  2. L1 candidates. From the library repository root:

    python3 smart-base/L1/scripts/l1-indicator-sheets.py --entry library/<doc-id> \
      --pdf uploads/<doc-id>/<doc-id>.pdf --pages <first>-<last> --json   # look first
    python3 smart-base/L1/scripts/l1-indicator-sheets.py --entry library/<doc-id> \
      --pdf uploads/<doc-id>/<doc-id>.pdf --pages <first>-<last> --write
    

    --write merges into candidates.json as the indicators aspect, replacing an indicator candidate with the same number (re-runs are idempotent). Check the count against the source’s own list of indicators (NHWA: the synthesis table, 29 indicators).

  3. L1 nodes. bun cat-harness-tools/pipeline/gen-library-jsonld.ts --entry library/<doc-id> writes blocks/indicator-<section>-NN.{jsonld,md}, each derivedFrom its section, provenance: ingested, with the fields in meta. Existing indicator IRIs do not change when the first candidate of a section stays the same indicator.
  4. L2 drafts.

    python3 smart-base/scripts/l2-program-indicators.py --entry library/<doc-id> \
      --prefix <PREFIX> --out <where the L2 author will review them>
    
  5. Into the DAK or DTH. The L2 author takes the drafts into the guide’s indicators component (DAK component 8; a DTH’s data elements, indicators and requirements), through the guide’s own review (l2-component-edit-review) and, for a DTH, a change set.

Validated against

  • NHWA handbook, 2nd edition (who-iris 9789240081291-eng): 29 indicators, 33 sub-indicators, PDF pages 51-91.

Not yet run on a guideline whose indicator sheets use a different layout (HIV SI 2022’s reference sheets: “What it measures”, “Rationale”, “Method of measurement”). Those labels are in the script’s label list, but the header number pattern may need a per-source ID_PATTERN.

Input and output contracts

What this skill takes and gives, as a class diagram drawn from its JSON Schema contracts by the schemas viewer’s own renderer. Each box links to its definition; each type to its specification.

input [1..1]output [1..1]indicator-extraction«skill»indicator-extractioninput : IndicatorExtractionInput [1..1]output : IndicatorExtractionOutput [1..1]IndicatorExtractionInput«json-schema»IndicatorExtractionInputentry : string [1..1]pdf : string [1..1]pages : string [0..1]prefix : string [1..1]IndicatorExtractionOutput«json-schema»IndicatorExtractionOutputentry : string [1..1]indicators : integer [1..1]subIndicators : integer [1..1]programIndicators : string [1..1]gaps : string[] [0..*]warnings : string[] [0..*]

Processes that run this skill

processstep(s) that name it
DTH authoringData elements, indicators, requirements (calls a sub-process)
L2 DAK authoringIndicators and requirements (calls a sub-process)

folio-assistant — code under the Apache License 2.0, documentation under CC BY 3.0. Generated schema and API references are rebuilt from source on every push to main.

deployed dev