Structural manifest

Stream Records, Never Emit Them

Frame physical records from byte chunks while preserving exact provenance and output silence.

U2 LIVE: PROVENSYNTHETIC EXAMPLE · /demo-claude/

Arbitrary byte chunks become complete physical records and sanitized metadata.

You will be able to

Frame LF-delimited records across arbitrary chunks.
Keep byte accounting even when parsing fails.
Describe output silence as an invariant, not a redaction step.

Core model

A stream may split anywhere—even in the middle of a UTF-8 sequence or line. U2 therefore treats bytes as the source of truth, finds LF record boundaries, counts a final unterminated record, and keeps exact offsets. Parsing can fail without losing byte coverage.

Mental model: A conveyor belt delivers puzzle pieces in arbitrary boxes. The framer rebuilds complete rows at the local station, records their positions, and discards the picture after measuring it.

The file digest consumes every raw byte. Physical records use half-open `[start,end)` spans. Valid top-level timestamp strings contribute observations; invalid UTF-8, empty records, invalid JSON, and non-object JSON become stable faults. The manifest stores none of the raw buffers.

Boundary table

Question	Observed boundary	Owner	Verdict
Chunk boundary	Arbitrary	Never a record boundary by itself	STREAM
LF	Physical delimiter	Ends one byte span	FRAME
Final bytes without LF	Unterminated	Still one physical record	FRAME
Invalid JSON	Parse fault	Keep coverage; emit code only	FAULT

Primary local source

packages/sessions/src/claude-manifest.ts

interface ClaudeManifestByteSource {
  identityBefore: FileIdentity;
  bytes: AsyncIterable<Uint8Array>;
  identityAfter(): Promise<FileIdentity>;
  close(): Promise<void>;
}

Transport unit chosen by the stream, not the file format.

One LF-delimited span or a non-empty final span.

`start` belongs to the range; `end` does not.

Raw buffers never enter the manifest or error text.

Synthetic lab

Feed a synthetic stream shape to the framer.

Retrieval and feedback

Predict before revealing

The final record has no LF. Is it ignored?

No. A non-empty final byte span is one physical record and remains covered by hashes and windows.

Retrieval cards

Retrieval cardsByte chunkActivate to flip

Byte chunkTransport unit chosen by the stream, not the file format.

Retrieval cardsPhysical recordActivate to flip

Physical recordOne LF-delimited span or a non-empty final span.

Retrieval cardsHalf-openActivate to flip

Half-open`start` belongs to the range; `end` does not.

Retrieval cardsOutput silenceActivate to flip

Output silenceRaw buffers never enter the manifest or error text.

Check your model

What is the source of truth for offsets and hashes?

Parsing may fail or normalize text; byte provenance must not.

Evidence ledger

Contract · PROVENSCOPE + source

Implementation · PROVENfocused code + tests

Compiled boundary · PROVENU2 receipt recorded

Independent probe · PROVENU2 files/hashes/bytes match

Authority: docs/fable-session-mining/LOOP-LOG.md. Tests never upgrade a missing live receipt.