A latent semantic index places every unit of a prose graph — a library section, a skill, a bean — in a space built from which words occur together, so units that discuss the same thing in different words sit close. It is a retrieval aid: every neighbour and finding it reports is a proposal, never a relation the graph asserts.
The method is Latent Semantic Indexing (node lsi), with
correspondence analysis (node correspondence-analysis) as its parallel track;
how to build, query and audit an index is the skill lsi-indexing.
Which graphs need an index
A graph needs one at 100 units and 20,000 words — a house threshold, with its
basis in scripts/lsi.ts. Below it a graph is not judged, which is not
the same as fine. The same verdict is kg:audit’s tool-downstream-fresh for the lsi-index Tool.
| graph | needs one | verdict | detail |
|---|---|---|---|
bootstrap-tools/bootstrap-tools-skills | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
bootstrap/skills | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
cat-harness-tools/beans | state graph — indexed on demand, never committed | n/a | state graph — indexed on demand, never committed |
cat-harness/beans | state graph — indexed on demand, never committed | n/a | state graph — indexed on demand, never committed |
cat-harness/docs | yes | fail | needs an LSI index and has none — run bun run cat lsi index --instance cat-harness --graph docs |
cat-harness/folio | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
cat-harness/library | yes | pass | fresh |
cat-harness/memory | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
cat-harness/methodologies | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
cat-harness/policies | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
cat-harness/site-chrome | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
cat-harness/skills | yes | pass | fresh |
fhir-harness/fhir-harness-docs | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
fhir-harness/fhir-ig-skills | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
fhir-harness/library | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
folio-assistant-core/beans | state graph — indexed on demand, never committed | n/a | state graph — indexed on demand, never committed |
folio-assistant-core/core-library | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
folio-assistant-core/core-methodologies | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
folio-assistant-core/core-skills | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
folio-assistant-core/folio-assistant-core-docs | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
folio-assistant-core/folios | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
folio-assistant-core/glossary | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
folio-assistant-core/memory | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
folio-assistant-sci/folio-assistant-sci-docs | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
folio-assistant-sci/library | yes | pass | fresh |
folio-assistant-sci/sci-methodologies | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
folio-assistant-sci/skills | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
smart-base/library | yes | pass | fresh |
smart-base/methodologies | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
smart-base/smart-base-docs | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
smart-base/smart-base-findings | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
smart-base/smart-base-site-docs | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
smart-base/smart-base-skills | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
smart-immunizations/library | yes | pass | fresh |
smart-trust/smart-trust-docs | yes | fail | needs an LSI index and has none — run bun run cat lsi index --instance smart-trust --graph smart-trust-docs |
who-iris/folio | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
who-iris/glossary | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
who-iris/library | yes | pass | fresh |
who-iris/who-iris-docs | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
who-iris/who-iris-site | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
who-iris/who-iris-skills | below the need-an-index threshold — not judged | n/a | below the need-an-index threshold — not judged |
cat-harness / library
2140 units · 14687 terms · k = 100 · retains 44.3 % of the weighted matrix · weighting log-entropy · sidecar cat-harness/test/results/lsi/cat-harness/library.lsi.json
Dimension 1 has no negative pole: it most likely measures unit length and term frequency (a margin), not a theme. Read the themes from dimension 2 on.
Each dimension is a contrast between two poles, shown by their highest-loading terms. It is not named here: naming it is a reader’s act.
| dim | σ | one pole | the other pole |
|---|---|---|---|
| 1 | 52.04 | arxiv, models, language, model, code, wang, zhang, preprint | (none) |
| 2 | 37.43 | arxiv, zhang, wang, preprint, liu, chen, yang, proceedings | prov, entity, prefix, activity, term, property, type, value |
| 3 | 31.99 | prov, entity, activity, prefix, rdfs, qualified, xsd, owl | algorithm, matrix, reward, arm, error, query, best, prompt |
| 4 | 27.34 | quot, nbsp, code, x7b, string, query, assistant, prompt | arm, algorithm, reward, matrix, algorithms, bound, approximation, instances |
| 5 | 25.48 | quot, json-ld, string, nbsp, node, type, x7b, iri | query, assistant, email, code, injections, defeating, prompt, side-effects |
| 6 | 25.24 | quot, nbsp, x7b, arm, algorithm, reward, assistant, string | enterprise, software, agents, zta, systems, engineering, publication, multi-agent |
| 7 | 23.92 | quot, nbsp, x7b, x5b, x5f, inherited, client, meta | json-ld, node, term, iri, graph, expanded, document, value |
| 8 | 23.84 | quot, nbsp, x7b, multi-agent, code, django, agents, sympy | health, security, attacks, injection, enterprise, attack, zta, defense |
Findings — 0 narrow dimension(s), 900 near-duplicate pair(s).
Near-duplicates (cosine ≥ 0.95) — similar is not duplicate; read both:
- 0.952 —
cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-002.md~cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-005.md - 0.952 —
cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-012.md~cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-016.md - 0.955 —
cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-016.md~cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-022.md - 0.969 —
cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-017.md~cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-020.md - 0.956 —
cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-017.md~cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-018.md - 0.980 —
cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-018.md~cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-020.md - 0.954 —
cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-018.md~cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-021.md - 0.957 —
cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-020.md~cat-harness/library/deerwester-1990-indexing-by-lsa/sections/page-024.md - 0.960 —
cat-harness/library/arxiv-2507.23348v1/sections/sec-000-abstract.md~cat-harness/library/arxiv-2507.23348v1/sections/sec-001-1-introduction.md - 0.952 —
cat-harness/library/arxiv-2507.23348v1/sections/sec-000-abstract.md~cat-harness/library/arxiv-2507.23348v1/sections/sec-026-82-repository-level-issue-resolution.md - 0.961 —
cat-harness/library/arxiv-2507.23348v1/sections/sec-001-1-introduction.md~cat-harness/library/arxiv-2507.23348v1/sections/sec-005-31-problem-formulation.md - 0.974 —
cat-harness/library/arxiv-2507.23348v1/sections/sec-019-55-case-study.md~cat-harness/library/arxiv-2507.23348v1/sections/sec-021-61-strengths.md - 0.973 —
cat-harness/library/w3c-2024-prov-jsonld/sections/page-003.md~cat-harness/library/w3c-2024-prov-jsonld/sections/page-065.md - 0.970 —
cat-harness/library/w3c-2024-prov-jsonld/sections/page-003.md~cat-harness/library/w3c-2024-prov-jsonld/sections/page-036.md - 0.970 —
cat-harness/library/w3c-2024-prov-jsonld/sections/page-003.md~cat-harness/library/w3c-2024-prov-jsonld/sections/page-063.md - … and 885 more in the sidecar
cat-harness / skills
258 units · 6578 terms · k = 100 · retains 76.0 % of the weighted matrix · weighting log-entropy · sidecar cat-harness/test/results/lsi/cat-harness/skills.lsi.json
Dimension 1 has no negative pole: it most likely measures unit length and term frequency (a margin), not a theme. Read the themes from dimension 2 on.
Each dimension is a contrast between two poles, shown by their highest-loading terms. It is not named here: naming it is a reader’s act.
| dim | σ | one pole | the other pole |
|---|---|---|---|
| 1 | 50.30 | merge, session, page, directory, declaration, kind, path, block | (none) |
| 2 | 23.11 | queue, prs, merge, watcher, sibling, commits, session, ledger | iri, subgraph, declaration, harness, node, publishes, site, directory |
| 3 | 20.31 | steward, train, merge, head, conflict, owed, conflicted, green | slot, block, watcher, chapter, project, objection, proof, audit |
| 4 | 17.95 | page, theme, tile, card, glass, avatar, board, sticky | steward, subgraph, cutover, member, iri, members, mount, module |
| 5 | 16.91 | preview, page, steward, head, staging, blocks, pages, slot | task, beans, store, session, goals, plan, sibling, goal |
| 6 | 16.39 | tile, glass, card, avatar, board, icon, sticky, tiles | slice, preview, deploy, payloads, payload, github, tombstone, heavy |
| 7 | 15.92 | actor, role, lane, model, roles, llm, train, agentic | commits, main, coordination, prs, sibling, idle, ledger, root |
| 8 | 15.64 | slice, term, claim, payloads, payload, content-addressed, sidecar, iri | consent, principal, phase, mounted, feedback, staging, cutover, hand-over |
Findings — 0 narrow dimension(s), 0 near-duplicate pair(s).
folio-assistant-sci / library
116 units · 2735 terms · k = 100 · retains 98.7 % of the weighted matrix · weighting log-entropy · sidecar cat-harness/test/results/lsi/folio-assistant-sci/library.lsi.json
Dimension 1 has no negative pole: it most likely measures unit length and term frequency (a margin), not a theme. Read the themes from dimension 2 on.
Each dimension is a contrast between two poles, shown by their highest-loading terms. It is not named here: naming it is a reader’s act.
| dim | σ | one pole | the other pole |
|---|---|---|---|
| 1 | 32.96 | formulas, formula, arxiv, pdf, matching, conference, zhang, math | (none) |
| 2 | 24.11 | url, doi, abs, arxiv, wang, conference, corr, volume | formulas, boxes, formula, detection, ssd, matching, character, bounding |
| 3 | 18.88 | ssd, default, boxes, iou, detection, region, detector, features | parsed, api, gpu, parsing, gemini, robust, accessed, perfect |
| 4 | 15.82 | proof, lean, blueprint, theorem, statement, formal, autoformalization, mathlib | conference, recognition, proceedings, detection, international, formulas, ieee, formula |
| 5 | 13.17 | node, blueprint, leanarchitect, nodes, mynat, latex, lake, declaration | forml, autoformalization, evaluation, test, real, datasets, training, pda |
| 6 | 12.19 | link, group, let, element, homotopy, links, trivial, groups | ieee, recognition, document, girshick, pattern, vol, documents, vision |
| 7 | 12.14 | link, group, element, ieee, let, homotopy, recognition, product | node, blueprint, default, url, leanarchitect, nodes, matching, declaration |
| 8 | 11.32 | quantum, url, arxiv, fault-tolerant, codes, merlean, balanced, paper | corr, html, annual, usa, agarwal, zhijiang, add, node |
Findings — 3 narrow dimension(s), 0 near-duplicate pair(s).
Narrow dimensions — carried by very few units; usually boilerplate, specimen text or a bad page:
- dimension 1:
folio-assistant-sci/library/arxiv-2512.09874v3/sections/sec-000-document.md,folio-assistant-sci/library/arxiv-2003.08005v1/sections/sec-front-matter.md,folio-assistant-sci/library/arxiv-2406.01940v2/sections/sec-016-conclusion.md - dimension 2:
folio-assistant-sci/library/arxiv-2406.01940v2/sections/sec-016-conclusion.md,folio-assistant-sci/library/arxiv-2003.08005v1/sections/sec-front-matter.md,folio-assistant-sci/library/arxiv-2601.22554v1/sections/sec-012-limitations.md - dimension 3:
folio-assistant-sci/library/arxiv-2512.09874v3/sections/sec-000-document.md,folio-assistant-sci/library/arxiv-2003.08005v1/sections/sec-front-matter.md
smart-base / library
839 units · 7506 terms · k = 100 · retains 55.3 % of the weighted matrix · weighting log-entropy · sidecar cat-harness/test/results/lsi/smart-base/library.lsi.json
Dimension 1 has no negative pole: it most likely measures unit length and term frequency (a margin), not a theme. Read the themes from dimension 2 on.
Each dimension is a contrast between two poles, shown by their highest-loading terms. It is not named here: naming it is a reader’s act.
| dim | σ | one pole | the other pole |
|---|---|---|---|
| 1 | 50.58 | registry, clinical, governance, facility, record, required, dpi-h, service | (none) |
| 2 | 26.89 | registry, dpi-h, requirement, consuming, limr, mandatory, authoritative, governed | user, intervention, training, accessed, pcposs, testing, interventions, handbook |
| 3 | 21.66 | provider, transmit, healthcare, registration, manage, facility, location, event | limr, fhir, models, computable, smart, cdse, semantic, standards |
| 4 | 20.53 | lhr, cdse, surveillance, hmis, phsp, alerts, clinical, transmit | product, master, supplier, identifier, mandatory, chain, canonical, date |
| 5 | 19.32 | accessed, world, organization, website, geneva, international, pdf, january | pcposs, user, worker, decision-support, logic, intervention, counselling, activity |
| 6 | 18.30 | accessed, website, world, adaptation, organization, geneva, pcposs, roll-out | architecture, product, infrastructure, goals, chain, supply, dhsc, shared |
| 7 | 17.87 | healthcare, transmit, provider, manage, fhir, clinical, models, limr | phsp, configurable, indicators, surveillance, lhr, outbreak, threats, investigation |
| 8 | 16.90 | intervention, limr, mhealth, project, evaluation, axis, consuming, versioning | pcposs, registry, identity, consent, roll-out, foundational, handbook, dpi |
Findings — 0 narrow dimension(s), 72 near-duplicate pair(s).
Near-duplicates (cosine ≥ 0.95) — similar is not duplicate; read both:
- 0.995 —
smart-base/library/9789240116191-eng/sections/page-003.md~smart-base/library/9789241511766-eng/sections/page-002.md - 0.985 —
smart-base/library/9789240116191-eng/sections/page-013.md~smart-base/library/9789240120747-eng/sections/page-016.md - 0.987 —
smart-base/library/9789240116191-eng/sections/page-014.md~smart-base/library/9789240120747-eng/sections/page-017.md - 0.957 —
smart-base/library/9789240116191-eng/sections/page-021.md~smart-base/library/9789240116191-eng/sections/page-022.md - 0.954 —
smart-base/library/9789240116191-eng/sections/page-043.md~smart-base/library/9789240116191-eng/sections/page-044.md - 0.986 —
smart-base/library/9789240116191-eng/sections/page-052.md~smart-base/library/9789240116191-eng/sections/page-054.md - 0.981 —
smart-base/library/9789240116191-eng/sections/page-056.md~smart-base/library/9789240120747-eng/sections/page-064.md - 0.988 —
smart-base/library/9789240093362-eng/sections/sec-005-12-smart-guidelines.md~smart-base/library/9789240116191-eng/sections/page-014.md - 0.982 —
smart-base/library/9789240093362-eng/sections/sec-005-12-smart-guidelines.md~smart-base/library/9789240120747-eng/sections/page-017.md - 0.977 —
smart-base/library/9789240093362-eng/sections/sec-005-12-smart-guidelines.md~smart-base/library/9789240101197-eng/sections/page-016.md - 0.997 —
smart-base/library/9789240081949-eng/sections/page-002.md~smart-base/library/9789240081949-eng/sections/page-008.md - 0.994 —
smart-base/library/9789240081949-eng/sections/page-004.md~smart-base/library/9789240116191-eng/sections/page-003.md - 0.993 —
smart-base/library/9789240081949-eng/sections/page-004.md~smart-base/library/9789241511766-eng/sections/page-002.md - 0.967 —
smart-base/library/9789240081949-eng/sections/page-005.md~smart-base/library/9789240081949-eng/sections/page-014.md - 0.951 —
smart-base/library/9789240081949-eng/sections/page-032.md~smart-base/library/who-rhr-1806-eng/sections/page-014.md - … and 57 more in the sidecar
smart-immunizations / library
130 units · 3985 terms · k = 100 · retains 99.0 % of the weighted matrix · weighting log-entropy · sidecar cat-harness/test/results/lsi/smart-immunizations/library.lsi.json
Dimension 1 has no negative pole: it most likely measures unit length and term frequency (a margin), not a theme. Read the themes from dimension 2 on.
Each dimension is a contrast between two poles, shown by their highest-loading terms. It is not named here: naming it is a reader’s act.
| dim | σ | one pole | the other pole |
|---|---|---|---|
| 1 | 59.94 | schedule, recommendations, dose, system, record, recommended, vaccines, client | (none) |
| 2 | 33.44 | epid, years, months, weeks, recommends, weekly, booster, ipv | information, eir, system, process, systems, date, business, requirements |
| 3 | 29.83 | immz, client, personas, worker, decision-support, workflows, nfxnreq, clinic | eir, program, individualized, system, information, project, ethical, registries |
| 4 | 22.19 | ddcc, digital, care, certificate, hcid, covid, specifications, pha | eir, duplicate, program, report, variables, immz, evaluation, contraindications |
| 5 | 18.35 | ddcc, pha, certificate, hcid, scenario, verification, solution, authority | piri, catch-up, routine, sia, doses, canada, home-based, kenya |
| 6 | 16.78 | ethical, challenge, geneva, journal, american, united, benefit, research | system, solution, questions, activity, doses, information, figure, routine |
| 7 | 16.00 | catch-up, piri, age, eligibility, ddcc, doses, vaccinations, missed | eir, adaptation, kit, indicators, requirements, recommendations, dak, decision-support |
| 8 | 15.09 | nfxnreq, retention, resilience, disaster, recovery, archiving, accessibility, regulation | yes, pha, registry, dose, certificate, hcid, scenario, signed |
Findings — 0 narrow dimension(s), 2 near-duplicate pair(s).
Near-duplicates (cosine ≥ 0.95) — similar is not duplicate; read both:
- 0.995 —
smart-immunizations/library/immunization-summary-table-1/sections/sec-001-table-1-summary-of-who-position-papers-recommend.md~smart-immunizations/library/immunization-summary-table-2/sections/sec-002-table-2-summary-of-who-position-papers---recomme.md - 0.996 —
smart-immunizations/library/immunization-summary-table-1/sections/sec-front-matter.md~smart-immunizations/library/immunization-summary-table-2/sections/sec-front-matter.md
who-iris / library
336 units · 3827 terms · k = 100 · retains 69.2 % of the weighted matrix · weighting log-entropy · sidecar cat-harness/test/results/lsi/who-iris/library.lsi.json
Dimension 1 has no negative pole: it most likely measures unit length and term frequency (a margin), not a theme. Read the themes from dimension 2 on.
Each dimension is a contrast between two poles, shown by their highest-loading terms. It is not named here: naming it is a reader’s act.
| dim | σ | one pole | the other pole |
|---|---|---|---|
| 1 | 23.69 | evidence, review, group, quality, health, interest, development, systematic | (none) |
| 2 | 13.83 | imprecision, effect, confidence, inconsistency, estimate, studies, rated, indirectness | organization, november, accessed, grc, world, planning, group, doi |
| 3 | 13.06 | accessed, november, pmid, cochrane, website, doi, oxman, org | interests, interest, members, conflict, coi, financial, individuals, chair |
| 4 | 12.14 | names, international, nomenclature, used, name, republic, style, social | gdg, doi, grade, interest, interests, review, evidence, coi |
| 5 | 11.85 | gender, determinants, social, equity, rights, services, human, right | name, names, republic, style, full, text, nomenclature, english |
| 6 | 11.57 | doi, coi, financial, accessed, november, conflict, interests, cois | rapid, proposal, canada, advice, planning, updating, key, review |
| 7 | 10.79 | republic, name, local, approved, english, haiti, addis, abu | nomenclature, symbols, world, international, abbreviations, text, used, organization |
| 8 | 10.54 | determinants, interests, canada, conflict, effect, declarations, social, conflicts | gdg, cost, key, questions, condition, steering, diabetes, effectiveness |
Findings — 0 narrow dimension(s), 9 near-duplicate pair(s).
Near-duplicates (cosine ≥ 0.95) — similar is not duplicate; read both:
- 0.953 —
who-iris/library/who-pub-tps-931/sections/page-039.md~who-iris/library/who-pub-tps-931/sections/page-040.md - 0.999 —
who-iris/library/who-pub-tps-931/sections/page-116.md~who-iris/library/who-pub-tps-931/sections/page-119.md - 0.975 —
who-iris/library/wpr-rdo-2020-003-eng/sections/page-006.md~who-iris/library/wpr-rdo-2020-003-eng/sections/page-008.md - 0.971 —
who-iris/library/wpr-rdo-2020-003-eng/sections/page-006.md~who-iris/library/wpr-rdo-2020-003-eng/sections/page-010.md - 0.965 —
who-iris/library/wpr-rdo-2020-003-eng/sections/page-006.md~who-iris/library/wpr-rdo-2020-003-eng/sections/page-007.md - 0.961 —
who-iris/library/wpr-rdo-2020-003-eng/sections/page-007.md~who-iris/library/wpr-rdo-2020-003-eng/sections/page-008.md - 0.955 —
who-iris/library/wpr-rdo-2020-003-eng/sections/page-007.md~who-iris/library/wpr-rdo-2020-003-eng/sections/page-010.md - 0.968 —
who-iris/library/wpr-rdo-2020-003-eng/sections/page-008.md~who-iris/library/wpr-rdo-2020-003-eng/sections/page-010.md - 0.970 —
who-iris/library/wpr-rdo-2020-003-eng/sections/page-020.md~who-iris/library/wpr-rdo-2020-003-eng/sections/page-021.md
Generated by bun run cat lsi:viz -- --detail during the docs-site build, with each index’s freshness and detail read from the checkout. The committed page in the repository carries the first two columns only — those are a function of the tree, and these are a function of the store. Bean tqjj.