Review · November 22, 2026
A Standard Wire Format for Large Language Model Inference Streaming
The Problem
Every LLM provider streams tokens over standard transports, mostly SSE on HTTP. The JSON inside the stream is different for each one. OpenAI puts generated text at choices[0].delta.content, Anthropic at delta.text, Gemini at candidates[0].content.parts[0].text.
Anyone sitting between an application and a model pays for this. LangChain, LiteLLM, AI gateways, logging pipelines and inspecting proxies each maintain one parser per provider.
This draft defines one request body and one set of SSE events that any provider could emit. The transport is untouched. Only the JSON payload is standardized.
How It Fits In
The draft is named for agentproto, and it does not fit that charter.
The agentproto charter covers dialog identifiers, lifecycle state, and how dialog context propagates. It says dialog context "does not include the content exchanged with AI models", and it puts "backend AI infrastructure network and protocols" out of scope. This draft is entirely about the content exchanged with AI models.
The draft's own scope statement makes the same point from the other side. It excludes agent-to-tool and agent-to-agent communication, which is the working group's whole subject.
It is a new application protocol, not an extension of anything. It depends on HTTP, TLS 1.3 and the WHATWG SSE spec. It depends on no OAuth RFCs and no adopted agentproto document.
The closest IETF home is probably httpbis or a dispatch discussion. Both authors already have two other draft-spk-httpbis-* drafts there.
What the Draft Describes
There are two players: a client and an inference endpoint.
The request. The client POSTs a JSON body with a new media type, application/llm-request+json. The body carries a version, a model name, the conversation history, optional tool definitions, sampling parameters, tracing metadata, and a namespaced extensions object for vendor features. Messages use the familiar system, user, assistant and tool roles. Content is a string or an array of typed parts for images, documents and audio.
The response. The endpoint replies with text/event-stream. Seven event types are defined:
stream.startandstream.endbracket the streamcontent.deltacarries a text fragment atdelta.textcontent.stopends a content block with a stop reasontool.callasks the client to run a functionusagereports input and output token countserrorreports a failure
Every event carries a version, a sequence number starting at 0, and a type that repeats the SSE event: field.
Tools. The model never executes anything. It emits tool.call, the client runs the function, and the client sends a new request with the result appended as a tool message.
Migration. Gateways can translate vendor formats into this one. Endpoints that don't support the media type return 415.
The shape is a blend of the OpenAI and Anthropic APIs. Message roles and stringified tool arguments come from OpenAI. Typed events, delta.text and the stop reason names come from Anthropic.
So What Does This Mean for OAuth?
Nothing directly. The draft says authentication "is handled in HTTP headers prior to stream establishment" and stops there. There is no token, no scope, and no authorization server.
One field is worth a look. The request metadata object can carry a user_id and session_id "for tracing, billing, and audit". These are plain strings set by the client. Nothing ties them to the credential that authenticated the request. An audit pipeline built on this field is trusting whatever the client typed.
The media type also has a second effect the draft states openly. Any component that sees application/llm-request+json "can immediately identify the traffic as LLM inference". Section 1.2.8 lists forward proxies and security appliances that inspect generated text as a use case. That only works on a path that terminates TLS, which the draft never says.
Interoperability Concerns
The tool flow example does not match the schema. Section 4.8 shows an assistant message with "content": null and a tool_calls array. The message table makes content a REQUIRED string or array and defines no tool_calls field. Sending a tool call back to the model is the one step every agent loop needs, and it is undefined. Two implementations will guess differently.
The events in that example are not conformant either. They omit v, seq and stream.start, and the tool.call and content.stop events drop the required index.
Content blocks are never defined. Events carry an index described only as "0 for primary". The draft doesn't say when a second block exists, whether a tool call is a block, or whether a tool call needs its own content.stop.
Unknown event types have no rule. Parsers must ignore unknown fields. Nothing says what to do with an unknown event. The IANA registry allows new event types, and the JSON Schema in Appendix A closes the list with an enum. A v1.0 client that validates against the schema breaks the day a provider registers an eighth event.
Text is the only output. delta.text is REQUIRED on every content delta. There is no way to stream reasoning output, citations, images or audio. Tool arguments arrive as one complete string, while today's providers stream them in fragments. All of this lands in vendor extensions, and the draft only defines extensions on the request.
Versioning is in four places. There is version in the request, v in every event, a version parameter on the media type, and a profile="llm-stream-1.0" parameter on Accept. text/event-stream defines no profile parameter. No negotiation is described.
Errors before the stream starts are unspecified. rate_limited and context_length_exceeded are normally known before the first byte. The draft doesn't say whether these are HTTP status codes or a 200 followed by an error event. Error codes and stop reasons also have no registry and no extension rule.
Usage counts are ambiguous. The usage event may appear more than once. The draft doesn't say whether the numbers are cumulative or incremental. A billing pipeline has to know.
The request is missing things clients rely on. There is no tool_choice, no structured output, and no non-streaming mode since stream MUST be true. The endpoint path and model discovery are left out.
Mechanisms That Could Be Removed
- Sequence numbers. The stated purpose is gap detection. SSE runs over TCP or QUIC streams, so events cannot go missing or arrive out of order. Section 1.2.1 mentions reconnection handling, and the draft never uses SSE
id:orLast-Event-IDfor it. - The HTTP version and TLS requirements. Endpoints MUST use HTTP/2 or HTTP/1.1, which rules out HTTP/3, and the same section then says the pattern is expected on HTTP/3. A payload format shouldn't constrain HTTP versions at all.
typeinside the JSON. It must equal the SSEevent:field. One of them is enough, and the draft never says what a receiver does on mismatch.- Section 6. The O(n) versus O(1) framing comparison argues against an alternative nobody proposed.
The Missing Comparison
The draft treats the market as fully fragmented. It never mentions that the OpenAI chat completions format is already a de facto standard. Mistral, Groq, Ollama, vLLM and many others expose OpenAI-compatible endpoints for exactly the reasons listed in Section 1.2.
That changes the question. A new provider can reach LangChain users today by copying OpenAI. A working group would need to hear why a new fourth format beats documenting the one already deployed.
Browser support is also overstated. Section 6.3 cites native EventSource support as a reason for SSE. EventSource can only send GET requests with no body and no custom headers. It cannot make the POST this draft requires, which is why every provider SDK parses SSE by hand over fetch.
Writing Quality
The prose is clear and well organized. The tables are consistent and the examples are easy to follow. BCP 14 keywords are used, though several requirements are stated as bare fragments like "MUST be the first event."
There are factual slips:
- SSE is called a W3C specification in the introduction and cited as WHATWG in the references.
- RFC 9110 is cited as HTTP/1.1. That is RFC 9112.
- MCP is placed under "the AI Alliance and Linux Foundation". The reference list says Anthropic and AAIF.
- The envelope is described as "TLV-style" with an implicit length, which is not TLV.
The media type registration is missing most of the required template, including security considerations and a change controller. The acknowledgments section says "TODO acknowledge."
The balance is off for a 31-page document. Roughly a third is motivation and rationale. The parts implementers need, such as content blocks, tool round trips and error handling, get a sentence or a table each.
Authors
Yaroslav Rosomakho (Zscaler) has one RFC, RFC 9850 on the SSLKEYLOGFILE format, published July 2026. Rosomakho is an IAB member and chairs the audit, hpke and seat working groups. With 29 active drafts across TLS, WIMSE, MASQUE, httpbis and OAuth, this is one of the most active authors in the IETF right now.
Santosh Pallagatti (Zscaler) has 11 RFCs from 2016 to 2026, all in routing: BFD, Geneve OAM and BIER. Pallagatti has three active drafts, all co-authored with Rosomakho, and sits on the 2026/2027 NomCom. This is new territory relative to that RFC record.
Both know the process well. Neither has a published record in LLM APIs, and the draft has no co-author from a model provider or a middleware framework.
Bottom Line
The problem is real and the approach is sensible: keep SSE, standardize the JSON. The draft is aimed at the wrong working group, since the agentproto charter excludes model content and inference protocols.
As a spec, it is an early sketch. The tool round trip is undefined, the output model is text only, and the case against simply documenting the OpenAI-compatible format is never made. Without a model provider at the table, a new format becomes one more thing for gateways to translate.
Read Section 4.8 and Section 5.3 to see the design. Skip Sections 1.2 and 6.