Evidence & Auditor Walkthrough
The WORM chain is evidence. Making it admissible evidence — for a regulator, a court, or an internal audit — requires a chain of custody that documents who had access to it, when, how integrity was verified at each handover, and what proof accompanies the data when it leaves CITADEL. This page explains the chain-of-custody model, the export bundle format, and provides a step-by-step auditor walkthrough for verifying a chain, validating anchors, and exporting a custody manifest.
Overview
Every WORM entry is cryptographically linked to its predecessor and content-addressed via TripleHash (SHA-256 + SHA-512 + BLAKE3 in parallel). Every 100 entries, CITADEL writes a chain anchor — an Ed25519 signature over the current chain state — that elevates an internally consistent chain into an externally attestable one. When evidence leaves CITADEL, it travels with those anchors, the corresponding public keys, and a custody manifest that records who authorised the export, who received it, and a bundle-level hash the receiver can recompute to detect transit tampering.
What CITADEL records automatically
Every WORM entry carries the following fields. Each field serves a specific custody purpose that an auditor can verify independently:
| Field | Type | Custody purpose |
|---|---|---|
id | UUID | Unique reference that cannot be re-used; identifies the exact evidence item |
sequence_num | int64 | Position in the chain; gaps are detectable — a missing sequence number is a finding |
ts_utc | timestamp | CITADEL's server-authoritative time — not the caller's — establishing when the entry was committed |
source + event_type | string | Provenance — which subsystem produced this evidence and what class of event it records |
project_id | UUID | Scope — which investigation this evidence belongs to, enabling targeted export |
payload | BYTEA | The evidence itself, byte-for-byte, as submitted by the originating platform |
triple_hash | hex string | Content-addressable digest — 128-byte composite of SHA-256 + SHA-512 + BLAKE3; proof the bytes have not changed |
chain_hash | hex string | SHA-256 of the previous chain_hash concatenated with this entry's payload; proves the entry exists at this sequence position |
prev_hash | hex string | The chain_hash of the immediately preceding entry; proves this entry came after the prior one |
created_at | timestamp | DB-level insert timestamp; must match ts_utc — divergence is an integrity flag |
Chain anchors
Every 100 entries, CITADEL writes a chain anchor: an Ed25519 signature over (sequence_num, ts_utc, chain_hash) using the master CITADEL signing key. Anchors are what elevate an internally consistent chain into an externally attestable one — they bind the chain state to a private key only CITADEL holds, enabling external notarisation.
"cm"># Per-entry integrity (128-byte composite digest) triple_hash = SHA-256(payload) || SHA-512(payload) || BLAKE3(payload) "cm"># Chain linkage (forward hash) chain_hash[n] = SHA-256(chain_hash[n-1] || payload_bytes[n]) "cm"># Ed25519 anchor — written every 100 entries anchor_payload = sequence_num + "|" + ts_utc + "|" + chain_hash anchor_digest = SHA-512(anchor_payload) anchor_sig = Ed25519_Sign(master_key, anchor_digest)
All three hash algorithms in TripleHash must be broken simultaneously to forge an entry. The Ed25519 anchor every 100 entries binds the chain to the CITADEL signing key. After any disaster recovery — even from a cold backup — the restored log can be verified against the original anchors; tampered recovery is detectable.
Evidence export bundle
When evidence leaves CITADEL — to an auditor, a court, or a regulator — the export bundle must contain all five components for the evidence to be attestable:
- WORM entries (
worm_entries.jsonl) — the evidence itself, one JSON object per line, sorted bysequence_numASC. - Chain anchors (
chain_anchors.jsonl) — every anchor covering the exported range, one JSON object per line. - Public keys (
pubkeys.yaml) — all Ed25519 public keys that signed anchors covering this range, withid,pubkey_hex,issued,revoked, andreplaced_by. - Bundle hash (
bundle.sha256) — SHA-256 over the concatenated bytes ofworm_entries.jsonl,chain_anchors.jsonl, andpubkeys.yaml. The receiver recomputes this to detect transit tampering. - Custody manifest (
manifest.yaml) — who authorised the export, who received it, when, and a reference to the bundle hash. This is the operator's responsibility; CITADEL does not generate it automatically.
Custody manifest format
The custody manifest is a YAML document produced by the operator and included in every export bundle as manifest.yaml. It creates the meta-chain-of-custody: the manifest references the bundle hash, and the bundle contains the WORM entry that authorised the export. Any future dispute about whether an export was authorised can be resolved against the chain itself.
bundle_id: "citadel-export-2026-04-19-0001" produced_at: "2026-04-19T14:32:00Z" produced_by: user_id: 42 role: admin jwt_sub: "alice@example.com" authorised_by: user_id: 99 role: admin jwt_sub: "bob@example.com" received_by: name: "EU DG-CONNECT audit team" contact: "audit-lead@dg-connect.europa.eu" received_at: "2026-04-19T14:35:00Z" evidence: time_range: from: "2026-01-01T00:00:00Z" to: "2026-03-31T23:59:59Z" entries_count: 12847 first_sequence_num: 10201 last_sequence_num: 23047 anchors_count: 129 integrity: bundle_sha256: "<hex>" anchor_pubkeys: - id: "citadel-anchor-2026Q1" pubkey_hex: "<32-byte Ed25519 pubkey hex>" issued: "2026-01-01" revoked: null
Export authorisation
Exporting evidence is itself a governance-relevant action. It passes through the full MARSHAL pipeline:
action.typeisDATA_EXPORT— picked up by AUGUR rule_03, which requires a non-emptyincident_id(the reason for the export). Attempting aDATA_EXPORTwithout an associated incident results in an immediateHARD_STOP.- Gate 3 (NDS) enforces separation of duties — two distinct identities must sign off: the operator requesting the export and a verifier from a different role group.
- Gate 5 (WORM) records the export action itself, including operator, verifier, reason, and the range exported, in the immutable audit chain.
The WORM entry for the export is included in the bundle itself, so the bundle is self-referencing: the custody manifest references the bundle hash, and the bundle contains cryptographic proof that the export was properly authorised.
Retention policy
Per NIS2 Directive Article 21(2)(b), incident-related evidence must be retained long enough to support authority review. CITADEL's default policy is indefinite — WORM entries are never deleted.
| Retention window | When to apply |
|---|---|
| 7 years | Default for regulated entities (financial services, healthcare, critical infrastructure) |
| 10 years | Where national law requires it (defence contractors, classified handling) |
| 30 days | Development and staging environments only — must be a completely separate CITADEL instance |
There is no TTL on WORM entries in v1.0.0. Archival tiering (moving entries older than one year to cold storage) is planned for v2.0; today, storage cost scales linearly with time.
Key rotation and revocation
Never delete an anchor public key, even after an Ed25519 key is rotated. Every anchor signed with the old key remains valid for the duration of the evidence retention period. The key's custody record shows:
issued— date the key came into use.revoked— date a replacement was issued. The old key no longer signs new anchors, but all existing signatures still verify against it.replaced_by— the newpubkey_id.
Export bundles for a given time range automatically include all public keys that signed anchors touching that range, including rotated keys. An auditor receiving a bundle for a range that spans a key rotation will see both the old and new key in pubkeys.yaml.
Auditor walkthrough
The following steps are written for external or internal auditors receiving a CITADEL evidence bundle. Complete all six steps in order — any failure at any step means the bundle should be rejected, either for re-export (if transit corruption is suspected) or for escalation (if tampering is suspected). A legitimate CITADEL deployment never produces a failing bundle.
Step 1 — Bundle integrity
Recompute the bundle hash and compare it against bundle.sha256:
"cm"># Recompute the bundle hash (concatenated bytes of the three data files) cat worm_entries.jsonl chain_anchors.jsonl pubkeys.yaml | sha256sum "cm"># Compare the output against bundle.sha256 diff <(cat worm_entries.jsonl chain_anchors.jsonl pubkeys.yaml | sha256sum | awk '{print $1}') <(cat bundle.sha256 | tr -d '[:space:]') "cm"># Clean result: empty diff — no output, exit 0 "cm"># Mismatch: the bundle was modified in transit — reject and request re-export
Step 2 — Pubkey registry
For each public key in pubkeys.yaml, verify:
pubkey_hexis a valid Ed25519 public key — exactly 32 bytes (64 hexadecimal characters).issuedandrevokeddates are plausible: not in the future, no unexplained gaps between consecutive keys.- If
revokedis set,replaced_bymust point at another key in the bundle whoseissueddate is on or after the revocation date. - No anchor in
chain_anchors.jsonlreferences a key outside that key's[issued, revoked)validity window.
Step 3 — Anchor signatures
For each anchor in chain_anchors.jsonl, verify the Ed25519 signature:
"cm"># The verification script (provided with the bundle) runs this for all anchors: "cm"># "cm"># payload = sequence_num + "|" + ts_utc + "|" + chain_hash "cm"># digest = SHA-512(payload) "cm"># ed25519.Verify(pubkey_for(anchor.pubkey_id), digest, anchor.signature) == true "cm"># "cm"># Run the bundled script: ./citadel-verify anchors --bundle ./export-bundle/ "cm"># Expected output on success: "cm"># [OK] 129/129 anchors verified
Every anchor must verify. A single failing anchor voids the integrity claim for the range that anchor covers. Reject the bundle on any anchor failure.
Step 4 — Linear chain walk
Iterate entries in worm_entries.jsonl sorted by sequence_num ascending. For each entry:
- Recompute TripleHash from the raw payload bytes and confirm it matches the stored
triple_hash:TripleHash recomputationBash"cm"># For each entry: expected_triple = sha256(payload) || sha512(payload) || blake3(payload) assert expected_triple == entry.triple_hash - Recompute chain_hash from the previous entry's chain_hash and this entry's payload:chain_hash recomputationBash
"cm"># For each entry: expected_chain = sha256(hex_decode(entry.prev_hash) || entry.payload_bytes) assert expected_chain == entry.chain_hash - Check continuity — each entry's
prev_hashmust equal the preceding entry'schain_hash:Continuity checkBash"cm"># For each entry[i] where i > 0: assert entry[i].prev_hash == entry[i-1].chain_hash "cm"># Any mismatch indicates insertion, deletion, or re-ordering — reject the bundle
./citadel-verify chain-walk --bundle ./export-bundle/ --verbose "cm"># Expected output on success: "cm"># [OK] 12847 entries walked "cm"># [OK] TripleHash verified: 12847/12847 "cm"># [OK] chain_hash verified: 12847/12847 "cm"># [OK] Continuity: no gaps or sequence breaks
Step 5 — Anchor-over-chain-walk
Confirm that the first and last entries in the bundle are covered by anchors that verified in Step 3. An anchor covers entries [anchor.prev_sequence_num + 1, anchor.sequence_num]. The chain_hash at anchor.sequence_num must match the value derived from the chain walk in Step 4.
Any chain_hash from the walk that does not appear in the anchor set means anchor coverage is incomplete. The bundle is valid as far as the walk proves, but the uncovered range has weaker evidence than anchor-sealed range — flag this as a finding and ask the deployer to re-export with complete anchor coverage.
./citadel-verify anchor-coverage --bundle ./export-bundle/ "cm"># Expected output on success: "cm"># [OK] First entry sequence 10201 covered by anchor at sequence 10300 "cm"># [OK] Last entry sequence 23047 covered by anchor at sequence 23100 "cm"># [OK] No gaps in anchor coverage
Step 6 — Time-range sanity
Confirm the bundle's claimed range (from manifest.yaml's evidence.time_range) is consistent with the actual entries:
evidence.first_sequence_num≤ the first entry'ssequence_numinworm_entries.jsonl.evidence.last_sequence_num≥ the last entry'ssequence_num.evidence.time_range.from≤ the first entry'sts_utc.evidence.time_range.to≥ the last entry'sts_utc.
./citadel-verify time-range --bundle ./export-bundle/ "cm"># Expected output on success: "cm"># [OK] Manifest range 2026-01-01T00:00:00Z → 2026-03-31T23:59:59Z "cm"># [OK] First entry: seq=10201, ts=2026-01-01T00:01:14Z (within range) "cm"># [OK] Last entry: seq=23047, ts=2026-03-31T23:58:02Z (within range) "cm"># [OK] Entry count matches manifest: 12847
Claims a green bundle supports
A bundle that passes all six steps supports the following claims in an audit or legal context:
- These specific payloads existed at their timestamps. The
chain_hashat eachsequence_numencodes the full history up to that point; re-deriving it from the payloads and the genesis hash yields the same value. - The timestamps are CITADEL-authoritative.
ts_utcis set by CITADEL's server clock at append time — not by the calling platform. - The payloads have not been altered since committal. TripleHash is content-addressable; any mutation would fail the chain walk at Step 4.
- The chain was not re-ordered or spliced. Continuity checks (Step 4.3) detect insertion, deletion, and re-ordering at any position in the range.
- The range is cryptographically sealed. The Ed25519 anchor signatures (Step 3) bind the chain_hashes to a private key only CITADEL holds; an attacker would need that key to forge anchors over a modified chain.
Claims a bundle does not support
Be explicit with stakeholders about what the bundle cannot prove:
- The payloads are truthful. CITADEL proves a payload was submitted and committed, not that its business-semantic content is accurate. If a platform submitted incorrect data, the WORM entry faithfully records the inaccuracy.
- The actor was who they claimed to be. CITADEL uses whatever identity the calling platform's IdP asserted. A compromised credential produces genuine-looking WORM entries.
- Actions were carried out. The WORM records the MARSHAL decision, not the execution outcome. Evidence for the effect of a decision lives in the downstream system's own records — you need both layers for a complete picture.
Common findings
| Finding | Likely cause | Action |
|---|---|---|
Gaps in sequence_num (e.g. 1234 → 1236, 1235 missing) | Gaps should not occur — the WORM table serialises appends and assigns sequence numbers monotonically with LOCK TABLE | Raise as a finding; the export may have missed entries, or the chain has been tampered with |
Two anchors with identical sequence_num | Cannot happen legitimately — each anchor covers a unique range | Duplicate entries in the anchor table; this is itself an incident — escalate immediately |
| Payload is not UTF-8 / not JSON | payload is BYTEA — legally any bytes; in practice always JSON, but the format does not enforce this | Chain integrity is still valid; the payload is opaque to this analysis but the custody proof holds |
| Bundle SHA-256 mismatch on receipt | Transit corruption or tampering | Reject and request re-export; if the deployer cannot produce a matching bundle, treat as a security incident |
Do's and don'ts
- Do verify the bundle hash before beginning any other step — transit corruption is the most common non-adversarial failure mode.
- Do include the WORM entry for the export action itself in the bundle so the custody chain is self-referencing.
- Do retain rotated Ed25519 public keys indefinitely alongside the anchors they signed.
- Do not hand out raw database dumps. A dump has the bytes but not the cryptographic proof chain.
- Do not modify or redact a payload in place. Redaction is done via a compensating WORM entry that references the old entry and marks it superseded — the original entry remains in the chain unchanged.
- Do not attempt a
DATA_EXPORTwithout an associated incident ID. AUGUR rule_03 will force aHARD_STOPand create a P1 incident automatically.