RAG · Responsible AI · Computational philology
Reconstructing absence without pretending certainty
Lacuna is a local, inspectable workbench for generating ranked hypotheses for a missing section of Óláfsdrápa Tryggvasonar. Its central design rule is simple: source evidence has authority; model output does not.
Executive takeaway
Auditability is the product.
In a domain where no generated stanza can be verified as recovered historical truth, a fluent answer is the wrong success criterion. The useful system preserves why a candidate was proposed, which sources support its diction, where it may be copying, why alternatives were rejected and what a specialist should challenge next.
Authority is explicit
Checked source stanzas are primary evidence. Research notes guide method. Run memory is cautionary. Model judgement is only a ranking signal.
Failure is retained
Rejected attempts, guardrail reasons, critic pushback and pruning events remain first-class artifacts rather than disappearing from the interface.
Claims stay bounded
The output is a ranked hypothesis under a stated corpus, model and configuration—not a singular reconstruction and never proof of historical wording.
Leadership decisions
Design around plausible fabrication.
The project began as a language-generation problem but matured into a data-governance and research-workflow problem. I rebuilt the engine around a Rust service, Qdrant retrieval, local Ollama roles and a React/TypeScript workbench, with the evidential hierarchy expressed in artifacts and API boundaries rather than left in a prompt.
Distil evidence before prompting
Retrieval writes a source-linked scaffold of supported terms, cautions and copy risks. Generation receives a constrained plan rather than a dump of attractive source lines.
Bound the proposer
An eight-line structured contract, planned lexical handles and a conservative realization layer reduce the space in which a model can manufacture authority.
Preserve independent pushback
Hard filters, critic and assessor roles record copy risk, unsupported diction, weak form and narrative problems separately from the proposal score.
System & evidence
Every step leaves a trail.
The engine parses and normalizes Old Norse corpora, creates stanza and multi-stanza shards, and indexes separate style, event and morphology lanes. Hybrid retrieval also supports continuity, research and run-memory queries. Each beam step then writes the intermediate objects a researcher needs to inspect.
Prompts and hashes are retained; partial reconstructions are checkpointed after each completed beam step; and pruning memory records signatures and reasons so later steps do not blindly repeat known failures.
Product evidence
Diagnosis, not theatre.
The interface places runs, candidates, reconstructed stanzas and event evidence side by side. It is deliberately operational: reviewers can see where a run is, what survived and which signals contributed. A separate public artifact mode exposes selected run reads without allowing run creation, index rebuilds or model access.
Evaluation & QA
Where support becomes invention.
Schema validity is only the transport boundary. A syntactically valid response can still copy a source, repeat itself, invent unsupported diction or fail the target form. Lacuna therefore uses independent checks before and after generation, and it treats model scores as one diagnostic among many.
| Layer | Question | Evidence retained |
|---|---|---|
| Structure | Does the proposal satisfy the eight-line contract and line-level budgets? | Parser diagnostics, line plan and rejection reason |
| Lexical support | Can content diction be traced to permitted source or morphology handles? | Lexicon IDs, source references and unsupported-token checks |
| Copy risk | Is a line too close to retrieved text or a previous candidate? | Containment, overlap and prune-memory signals |
| Form & continuity | Is the candidate plausible in structure and coherent with its boundaries? | Surface scores, critic objections and continuity diagnostics |
| External validity | Can the method recover or rank masked-known material? | Masked-known evaluation plus specialist philological review |
Outcome & lessons
Honesty made the system more useful.
Lacuna now has a functioning local retrieval and run-artifact path, a responsive workbench, public read-only presentation and a detailed failure atlas. Its current value is not that it has solved a centuries-old lacuna; it is that it makes the research boundary visible enough to improve methodically.
- Provenance must survive transformation. A source reference is only useful if it stays attached through planning, generation, scoring and review.
- Rejected output is research data. Failure patterns reveal whether the next investment belongs in retrieval, morphology, metre, prompting or model choice.
- Do not let a judge become evidence. Model critique can rank and challenge; it cannot create historical authority.
- Public artifacts need a narrower surface. A read-only viewer can share the method without exposing model services or write operations.