CodeC

Protocol map

Codec runs on one client/gateway/engine triangle. The wire frame, the per-modality map, and the response headers shift per pathway — the triangle does not. The diagram below is the at-a-glance view; the normative spec is in PROTOCOL.md and pipeline math is in PIPELINES.md.

Codec protocol map showing three pathways: text-tokens (v0.2), MCP tool-calls with leaf-mode bypass, and v0.3 latents. Each pathway shows the client → gateway → engine flow with the wire-level headers and frame types annotated.
Source diagram lives in PROTOCOL.md as Mermaid blocks; the polished SVG above is a manual mirror that gets updated alongside the spec.

Text-tokens (v0.2)

The original wire shape. Engines emit CodecFrames of uint32 token IDs; the gateway forwards them; the client detokenizes locally. The Codec-Tokenizer-Map response header carries the sha256 of the active tokenizer map so the client can fail fast on mismatch.

See codec-sglang, codec-vllm, codec-llamacpp.

MCP tool-calls (leaf-mode bypass)

The fast path on a Codec-aware gateway: the tool itself tokenizes its result once, against a pinned tokenizer map, and ships the IDs in a sibling _codec_meta block. The gateway recognizes the pre-tokenized output and bypasses its back-compat shim — the [Codec][leaf] log fires; [Codec][shim] warnings disappear. Non-Codec-aware tools in the same namespace still work via the shim path.

See codec-metamcp, @codecai/mcp-leaf, codec-time-leaf reference image.

Latents (v0.3)

Image and video diffusion models stream VAE latents instead of decoded pixels. Wire shape: one LatentStreamHeader followed by one or N LatentFrames per the negotiated pipeline (one of seven: raw, int8, int4, int8-adaptive, int4-adaptive, delta+int8, delta+int4). The client runs vae_decode locally; pixels never touch the wire.

See codec-comfyui, codec-diffusers, PIPELINES.md.

Safety policies (v0.4)

A sixth HELLO/READY capability axis. Clients announce accept_safety_policies on HELLO; servers respond on READY with safety_policy_id + safety_policy_hash. The hash anchors to a sanitized descriptor served at /.well-known/codec/policies/<id>.json (and the content-addressed /sha256/<hex>.json sibling), which publishes the shape of enforcement (categories, action types, classifier family) but never operator-internal banned-token lists, regex patterns, or classifier thresholds. finish_reason: "policy_violation" surfaces when a server-side action fires. Non-breaking against v0.3 — older clients ignore the new optional fields and continue working.

See spec/versions/v0.4.md, @codecai/web-safety (browser-side prefilter + classifier registry), and codec-supervisor (server-side enforcement + admin app).