Rank candidate glossary terms in prose by C-value

The scoring step of the automatic-term-extraction method: count the word n-grams of one or more Markdown or text files (not across punctuation, front matter or code), and rank them by C-value as Frantzi & Ananiadou 1996 define it, so nested multi-word terms are not swamped by the longer phrases that contain them. Tested against the paper’s own worked example (Table 2). Every row is a candidate; it writes nothing and promotes nothing.

Skills it satisfies

A skill states the capability; this Tool is one way to exercise it. Each skill’s declared input and output contract follows, where it has one.

glossary-terms

Add terms to a layer’s glossary: local SKOS terms (folio-glossary/v1) with a definition, a code and links to external SKOS concepts, and references to whole external schemes. Rendered on the glossary/ page; gated by check:glossary. Authored terms go to gettext (.pot) for translation; a paper’s glossary is one more scheme.

Source: folio-assistant-core/skills/library/cataloguing/glossary-terms.md.

Also satisfied by: glossary-from-source.

The skill declares no I/O contract (schemas/skills/glossary-terms/{input,output}.schema.json).

ontologist

Semantic Ontologist — scans narrative text for ambiguous terms, generates a formal glossary/registry, and produces a mapping linking narrative strings to formal identifiers. Acts as the “Gatekeeper” ensuring every term has a unique type/identity assignment before formalization proceeds.

Source: folio-assistant-core/skills/library/cataloguing/ontologist.md.

The skill declares no I/O contract (schemas/skills/ontologist/{input,output}.schema.json).

Inputs and outputs

Each port names a shared type by IRI, so two Tools for one skill can be checked for interchangeability.

porttyperequireddescriptionconstraint
infilesRepoPathyesOne or more Markdown or text files whose prose is read.string · pattern ^[A-Za-z0-9._][A-Za-z0-9._/-]*$ · A path relative to the repository root. No .. segments.
inmaxLengthCountLongest n-gram, in words; 1 or more. Default 5.integer · A non-negative count — how many of something, or how many to show. Zero is a real answer.
inminFreqCountFewest occurrences an n-gram needs to be scored. Default 3, the paper’s cut-off; 1 or more.integer · A non-negative count — how many of something, or how many to show. Zero is a real answer.
intopCountHow many candidates to print; 1 or more. Default 50.integer · A non-negative count — how many of something, or how many to show. Zero is a real answer.
inedgeStopwordsFlagDrop n-grams that begin or end with a common English function word. Off by default, because the 1996 method has no such filter.boolean · A boolean switch — projects to a bare flag, with no value word
injsonFlagPrint the candidates as JSON, each with its frequency, nesting counts and C-value.boolean · A boolean switch — projects to a bare flag, with no value word
outcandidatesTextA ranked table, or JSON with status: candidate, of n-grams with a C-value above 0.string · A short single-line human string. Not admissible as a command-line argument.

Processes and tasks

Steps this Tool can carry out, because each names a skill it satisfies:

processsteplanevia skill
Ingestion subprocess — build the L1 knowledge graphRead glossary into candidate termsIngestion Engine (agent, runs unattended)glossary-terms
Classify and code contentRun reuse-first lookup and record candidatesAuthoring agentglossary-terms
Classify and code contentAdjudicate slot codings and glossary candidatesTerminologist / ontologist reviewer (person)ontologist

How it is deployed and reached

  • Command line. From the repository root: bun run folio-assistant-core/scripts/c-value.ts.
  • GitHub Actions. No workflow in this checkout runs its command.

As an API: its operation in the OpenAPI document and the MCP tool list of its layer.

installnone
runtimebun
networkno
when to choose it
whenA person or agent wants candidate glossary terms from a body of prose, ranked, to review before writing any definitions.
limitsStatistics only, as the 1996 paper: no part-of-speech filter (the later C-value papers that add one are not held), English word splitting, and no reference corpus. On ordinary prose without –edge-stopwords the top rows are function phrases. A person decides what is a term.
costMilliseconds for a chapter; linear in words times –max-length. No network.

What it is made of

Linked data

The Tool is published as its own JSON-LD document at its IRI, https://litlfred.github.io/folio-assistant/folio-assistant-core/tools/c-value-terms.jsonld, partOf its layer’s document. The node below is the definition this page was drawn from, and is embedded in the page as application/ld+json.

{
  "@context": "https://litlfred.github.io/folio-assistant/folio-assistant-core/folio-assistant-core.context.jsonld",
  "@id": "https://litlfred.github.io/folio-assistant/folio-assistant-core/tools/c-value-terms.jsonld",
  "@type": "https://litlfred.github.io/cat-harness/v0/ns/Tool",
  "partOf": "https://litlfred.github.io/folio-assistant/folio-assistant-core/folio-assistant-core.jsonld",
  "name": "c-value-terms",
  "title": "Rank candidate glossary terms in prose by C-value",
  "description": "The scoring step of the `automatic-term-extraction` method: count the word n-grams of one or more Markdown or text files (not across punctuation, front matter or code), and rank them by C-value as Frantzi & Ananiadou 1996 define it, so nested multi-word terms are not swamped by the longer phrases that contain them. Tested against the paper's own worked example (Table 2). Every row is a candidate; it writes nothing and promotes nothing.",
  "install": {
    "none": true
  },
  "invoke": {
    "shell": "bun run folio-assistant-core/scripts/c-value.ts"
  },
  "io": {
    "inputs": [
      {
        "name": "files",
        "schema": "https://litlfred.github.io/folio-assistant/cat-harness/tool-types.schema.json#/$defs/RepoPath",
        "description": "One or more Markdown or text files whose prose is read.",
        "required": true,
        "arg": {
          "positional": 0
        }
      },
      {
        "name": "maxLength",
        "schema": "https://litlfred.github.io/folio-assistant/cat-harness/tool-types.schema.json#/$defs/Count",
        "description": "Longest n-gram, in words; 1 or more. Default 5.",
        "required": false,
        "arg": {
          "flag": "--max-length"
        }
      },
      {
        "name": "minFreq",
        "schema": "https://litlfred.github.io/folio-assistant/cat-harness/tool-types.schema.json#/$defs/Count",
        "description": "Fewest occurrences an n-gram needs to be scored. Default 3, the paper's cut-off; 1 or more.",
        "required": false,
        "arg": {
          "flag": "--min-freq"
        }
      },
      {
        "name": "top",
        "schema": "https://litlfred.github.io/folio-assistant/cat-harness/tool-types.schema.json#/$defs/Count",
        "description": "How many candidates to print; 1 or more. Default 50.",
        "required": false,
        "arg": {
          "flag": "--top"
        }
      },
      {
        "name": "edgeStopwords",
        "schema": "https://litlfred.github.io/folio-assistant/cat-harness/tool-types.schema.json#/$defs/Flag",
        "description": "Drop n-grams that begin or end with a common English function word. Off by default, because the 1996 method has no such filter.",
        "required": false,
        "arg": {
          "flag": "--edge-stopwords"
        }
      },
      {
        "name": "json",
        "schema": "https://litlfred.github.io/folio-assistant/cat-harness/tool-types.schema.json#/$defs/Flag",
        "description": "Print the candidates as JSON, each with its frequency, nesting counts and C-value.",
        "required": false,
        "arg": {
          "flag": "--json"
        }
      }
    ],
    "outputs": [
      {
        "name": "candidates",
        "schema": "https://litlfred.github.io/folio-assistant/cat-harness/tool-types.schema.json#/$defs/Text",
        "description": "A ranked table, or JSON with `status: candidate`, of n-grams with a C-value above 0."
      }
    ]
  },
  "satisfies": [
    "glossary-terms",
    "ontologist"
  ],
  "requirements": {
    "runtime": [
      "bun"
    ],
    "network": false
  },
  "selection": {
    "when": "A person or agent wants candidate glossary terms from a body of prose, ranked, to review before writing any definitions.",
    "limits": "Statistics only, as the 1996 paper: no part-of-speech filter (the later C-value papers that add one are not held), English word splitting, and no reference corpus. On ordinary prose without --edge-stopwords the top rows are function phrases. A person decides what is a term.",
    "cost": "Milliseconds for a chapter; linear in words times --max-length. No network."
  }
}

folio-assistant — code under the Apache License 2.0, documentation under CC BY 3.0. Generated schema and API references are rebuilt from source on every push to main.

deployed dev