Read the Incident as Evidence, Not Mythology
Sources: [Hugging Face technical timeline](https://huggingface.co/blog/agent-intrusion-technical-timeline), [initial incident report](https://huggingface.co/blog/security-incident-july-2026), and [OpenAI account](https://openai.com/index/hugging-face-model-evaluation-security-incident/)
The enterprise problem and today’s slice
Enterprise problem: A security team that treats a compelling incident story as fact can spend its response budget on the wrong control while an unmeasured boundary remains open.
Whole-course context: This first day creates the evidence baseline consumed by the feedback-loop and authority-containment days.
Today’s slice: Build a claim ledger and a collection pipeline for the public record; this is not an exploit reconstruction.
End-of-day evidence: A signed, timestamped ledger that separates reported facts, corroboration, inference, and unknowns.
Still unsolved: The next days determine how automated evaluation feedback became actionable and where authority was amplified.
Customer outcome and implementation focus
The incident commander needs a defensible answer to “what do we know now?” before changing production controls. The implementation is a small evidence service: immutable source captures enter a parser, claims are normalised into a review queue, and decisions become an append-only audit trail.
| Story ID | User story | Observable acceptance |
|---|---|---|
| D01-US-01 | As an incident commander, I want each material claim tied to a source and confidence, so that containment decisions can be reviewed. | A reviewer can locate the capture hash, timestamp, boundary, confidence, and decision for every claim. |
| D01-US-02 | As a security analyst, I want unknowns recorded explicitly, so that inference is never promoted to fact. | A query returns disputed and unknown claims separately from reported facts. |
Components in focus
The danger is silently overwriting or mixing evidence, which makes later conclusions irreproducible. These components separate capture from interpretation and show exactly where durable state lives.
| Layer | Component and owner | Compute/runtime | Storage | Responsibility and evidence |
|---|---|---|---|---|
| Intake | Collector service, security engineering | Container job with egress allowlist | Object storage, compliance owner | Fetches permitted public pages; object version ID and SHA-256 prove the capture. |
| Interpretation | Claim normaliser, incident response | Stateless worker | PostgreSQL evidence database, incident response owner | Creates a claim only from a captured object; unique (capture_hash, claim_text) prevents duplicate assertions. |
| Review | Reviewer API, incident commander | API deployment behind SSO | Append-only decision table | Records confidence and rationale; audit event proves who changed a conclusion. |
| Search | Read model, security engineering | Stateless query API | No cache required for the small incident ledger | Returns immutable claim versions; database query plan and audit ID are evidence. |
The collector is not an investigator: it preserves bytes. The normaliser attaches a claim to those bytes, and the reviewer adds a clearly attributed judgement. The audit event makes a later correction visible rather than rewriting history.
Build the evidence ledger
An unstructured incident document cannot answer which claim justified a containment action. Define a narrow record schema first, then require every conclusion to point back to its source object.
CREATE TABLE incident_claim (
claim_id uuid PRIMARY KEY,
capture_sha256 text NOT NULL,
boundary text NOT NULL,
statement text NOT NULL,
confidence text NOT NULL CHECK (confidence IN ('reported','corroborated','inferred','unknown')),
reviewed_at timestamptz,
reviewer_id text
);
CREATE UNIQUE INDEX claim_source_once ON incident_claim (capture_sha256, statement);
Declared intent: retain a reviewable claim, not a mutable narrative. Interpreter: PostgreSQL enforces the key and confidence constraint. Software effect: the normaliser can insert one versioned assertion per source. Hardware effect: the database consumes durable disk and its backup policy; the collector’s source objects remain in separate versioned object storage. Evidence: SELECT confidence, count(*) FROM incident_claim GROUP BY 1; must show unknowns rather than hiding them.
Reconcile claims without inventing certainty
Conflicting reports are dangerous because a majority vote is not corroboration when sources repeat the same original assertion. Compare provenance, scope, and time, then leave the claim unresolved when those do not independently agree.
psql "$EVIDENCE_DATABASE_URL" -c \
"SELECT confidence, boundary, count(*) FROM incident_claim GROUP BY confidence, boundary;" # Inspect what the response is actually assuming.
Failure drill: insert a test claim with a missing capture_sha256; the database must reject it. Then query a known reported claim and confirm it remains available. This proves the rejection is scoped and does not erase valid evidence.
Before and after, side by side
A narrative-only response lets an untraceable sentence drive a broad control change. The ledger changes that decision by requiring an immutable source and an attributed confidence judgement.
Key takeaways
- Preserve source bytes separately from claims and reviewer judgement.
- An explicit unknown is a safer operational output than an invented explanation.
Checklist
- [ ] Capture hashes and object versions are retained.
- [ ] Every material claim has a confidence, boundary, and review trail.