PatrickNDevelopment

Work / Case study · Published · 2026-10-15

SourceTrace

Claim and citation review with exact passages and a human-only approval gate.

SourceTrace side-by-side review of a claim and its cited passage
SourceTrace side-by-side review of a claim and its cited passage. Screenshot of the implemented application with its synthetic sample data.

The problem

Research notes drift away from their sources: a quote gets paraphrased, a citation points at the wrong paragraph, and nobody can say who approved a claim. Reviewers need every claim tied to the exact passage that bears on it, and a record of every decision.

What it does

  • Sources from pasted text, Markdown or text files, or text PDFs via the optional local backend; each stored with a SHA-256 content hash.
  • Exact passages selected by mouse or keyboard; locators store code-point offsets and a context hash, so a changed source can never silently re-point a citation.
  • A review state machine only a person can drive; approval is refused with precise reasons, and refused attempts are recorded in an append-only audit trail.
  • Heuristic checks, labelled as heuristics: unsupported claims, mismatched citations, quotes or dates missing from the cited passage, duplicates.
  • Traceable evidence report export (Markdown and JSON); optional opt-in AI assistant that can only suggest drafts.

Engineering choices

  • All review logic in one TypeScript engine so the static demo is fully usable; the backend only stores snapshots, extracts PDF text and brokers optional AI calls.
  • Offsets in Unicode code points, not UTF-16 units or bytes; source text is immutable.
  • Anything duplicated between TypeScript and Python is specified, covered by golden vectors and fuzzed.
  • PDF extraction runs in an isolated, time-limited process because PDFs are untrusted input.

Significant challenge: An audit trail that cannot break the workspace

The independent review found that unbounded audit growth could make a saved workspace impossible to load, that the Markdown report rendered markup from sources, and that the TypeScript and Python JSON layers disagreed on deep nesting and huge integers. All were fixed with regression tests; the audit trail of a claim is now capped at 500 events.

Tested results

  • 221 Vitest and 51 pytest tests; ruff and mypy --strict clean.
  • 218 of 218 golden vectors agree between the TypeScript and Python engines; differential fuzz with 0 mismatches.
  • 29 Playwright tests under /source-trace/.
  • Independent acceptance review: 7 defects fixed with regression tests.
  • Published 2026-10-15: CI green on the first run (three jobs), Pages deployed, live check passed.

Last recorded verification 2026-10-15; latest recorded commit 4f2ac8d. The full commands and outcomes are in the repository's docs/test-report.md.

SourceTrace list of sources with provenance
SourceTrace list of sources with provenance
SourceTrace review stacked on a phone
SourceTrace review stacked on a phone

Stack

  • TypeScript
  • React
  • Vite
  • zod
  • Python
  • FastAPI
  • SQLite
  • pypdf
  • Vitest
  • pytest
  • Playwright

Candid limitations

  • It does not verify truth; its checks are heuristics.
  • Text-only PDFs (no OCR); one workspace per browser; the local backend has no authentication.
  • The AI assistant was tested only with a fake provider and mocked transport.

How it was made

Directed and reviewed by the owner; specification, code, tests and documentation written with Claude Code (AI). A separate Claude Code session that did not write the code then re-ran every check and fixed what it found; that is an AI review, not a human code audit. See the repository's AI_ASSISTANCE.md.