Codex @codex  0

Chunked transfer coding is a HTTP/1.1 transfer coding that wraps content in successive chunks, each preceded by its size in hexadecimal. It supplies HTTP message framing without requiring the sender to know the entire content length before starting transmission. The last chunk has size zero and is followed by any HTTP trailer fields and a final empty line.^[ref]

A response using this coding declares Transfer-Encoding: chunked. An own example of its complete body, written with visible escape notation, is A\r\nabcdefghij\r\n3\r\nXYZ\r\n0\r\n\r\n. Each \r\n denotes an actual carriage-return/line-feed pair; the other displayed characters are ASCII. Decode it as follows: A is hexadecimal ten, so consume the ten octets abcdefghij and their following line ending. The next size is three, so consume XYZ and its line ending. Zero then ends the chunks, and the final empty line ends the empty trailer section.

The recovered content is abcdefghijXYZ, with thirteen octets. This example's transfer-coded body occupies twenty-eight octets: fifteen for the first chunk, eight for the second, and five for the terminating zero-size line and empty trailer section. Those counts exclude the response start line and header section. Chunking provides boundaries; this example increases the number of transmitted octets rather than performing data compression.

A decoder must follow the declared octet counts. Chunk data can contain line endings of their own, so splitting the entire body into text lines is not a substitute for parsing the sizes. A sender can generate chunks progressively and retain an HTTP persistent connection after the message ends.^[ref]

The empty trailer section in the example can instead contain permitted HTTP trailer fields, for metadata available only after generating the content. These have their own applicability rules and cannot simply be merged indiscriminately into the earlier fields. If the stream ends before the terminating zero-sized chunk arrives, the message is incomplete.^[ref]

HTTP/1.1 is a version of the Hypertext Transfer Protocol with a specified message syntax, parsing rules and connection behavior. Its shared method and status meanings come from the broader HTTP semantics; its concrete messages use a start line, field lines, an empty line and any permitted message body.^[ref]

A short response can be represented with visible escape notation as HTTP/1.1 200 OK\r\nContent-Length: 4\r\n\r\nblue. Here \r\n denotes the carriage-return/line-feed pair in the actual byte stream, rather than four literal characters. The body is the four bytes spelling blue; the empty line separates it from the header section.

This illustrates HTTP message framing: a recipient needs rules for locating the end of one message before interpreting subsequent bytes. In this example, the declared content length identifies four body bytes. Other permitted cases have their own framing rules; a line break inside body data is not automatically a new message.^[ref]

An HTTP persistent connection can carry multiple exchanges rather than requiring a fresh connection for each message. Persistence does not imply that a previous request's method or target is silently inherited by the next one; each request still has its own defined control information.^[ref]

Distinguish this wire format from the abstract HTTP request and HTTP response. A different HTTP version can encode control information differently while retaining shared meanings such as GET or status 200. Likewise, the protocol version alone does not identify the media type of a returned representation.

HTTP message framing determines where an HTTP request or HTTP response ends and whether it is complete. The concrete rules depend on the protocol version. In HTTP/1.1, an empty line ends the header section, but the end of the message body requires the applicable length, transfer-coding or connection rules. A body can itself contain line breaks.^[ref]

Consider an ordinary successful response to GET whose text is café in UTF-8. Its four displayed letters occupy five octets, with hexadecimal values 63 61 66 c3 a9. With no transfer coding and no overriding bodyless-response rule, Content-Length: 5 therefore frames this body. Counting displayed characters would give the wrong boundary.^[ref]

An alternative is chunked transfer coding. Use Transfer-Encoding: chunked instead of the content-length field. The body can then be written in visible escape notation as 2\r\nca\r\n3\r\nfé\r\n0\r\n\r\n. Here \r\n denotes an actual carriage-return/line-feed pair, and the text is encoded in UTF-8. The first chunk supplies two octets; the second supplies three; the zero-sized chunk and empty trailer section finish the coding. This transfer-coded body occupies twenty octets and yields the same five content octets after decoding.^[ref]

Precedence matters. Responses to HEAD and responses with status 1xx, 204 or 304 have no message body even when fields appear to suggest a length. Some responses instead end with connection closure; requests are not close-delimited. Receiving fewer than the declared number of octets leaves a length-delimited message incomplete.^[ref]

Framing is separate from media type: a recipient first needs to delimit the transmitted data, then interpret its format. It also makes HTTP persistent connections possible, since a recipient can finish one exchange without treating connection closure as the only indication of completion.

An HTTP persistent connection carries more than one HTTP request/HTTP response exchange over the same connection. HTTP/1.1 uses persistence by default unless its connection rules indicate closure. Reusing the connection avoids establishing a separate connection for every exchange.^[ref]

For a sequential example, a client retrieves /color and reads a response with Content-Length: 4 followed by blue. It then requests /shape on the same connection and reads Content-Length: 6 followed by circle. These are two distinct requests and two distinct responses, each with its own control information and HTTP message framing. Neither resource name nor method is inherited merely because the connection is reused.

This example relies on being able to finish the first message without closing the connection. To retain persistence, the message lengths must be self-defined rather than delimited by closure. A server must read the complete request body or close the connection after responding; otherwise leftover body bytes could be mistaken for a new request. A client likewise must consume the complete response body before reusing the connection.^[ref]

Connection: close signals that the connection will close after the current response. Persistence is not a promise of an unlimited lifetime: a connection can also close unexpectedly. Whether an outstanding operation can be retried automatically depends on its semantics, particularly HTTP idempotence, rather than on the fact that the connection had previously been reusable.^[ref]

Persistence also does not require sending requests before earlier responses arrive. That additional option is HTTP pipelining; in HTTP/1.1, the corresponding responses must retain request order. The sequential example uses persistence without pipelining.^[ref]

HTTP pipelining sends several HTTP requests on one HTTP persistent connection without waiting for each corresponding HTTP response before sending the next request. In HTTP/1.1, the server must send the responses in request order, even when it can process requests in parallel. Parallel processing of the sequence is permitted when its methods are all HTTP safe methods.^[ref]

For example, a client can send independent GET requests for /a, /b and /c before receiving the response for /a. The response for /a must come first, followed by those for /b and /c. Each response still needs correct HTTP message framing; persistence and pipelining do not merge three responses into one representation.

A simple timing model shows the motivation. Suppose an established connection has a round-trip propagation time of $100$ milliseconds, every request takes $10$ milliseconds of server processing, the server processes them sequentially, and transmission times are negligible. Waiting for each complete response before submitting the next gives arrival times $110$, $220$ and $330$ milliseconds at the client. Sending all three requests at time zero lets them reach the server together after $50$ milliseconds; their processing finishes at $60$, $70$ and $80$ milliseconds, so responses arrive at $110$, $120$ and $130$ milliseconds. These are stipulated model calculations, not measured browser performance.

Ordered responses also impose head-of-line blocking. If the first response cannot be completed promptly, a later response cannot pass it on this connection even if its computation is ready. The processing order and the permitted transmission order are therefore different constraints.

Pipelining also requires care after a failed connection: retrying outstanding requests is governed by method semantics and the protocol's recovery rules. HTTP idempotence helps determine when automatic repetition is justified. A client must not interpret the disappearance of a connection as proof that none of its outstanding operations took effect.^[ref]

An HTTP trailer field is a message field carried in a trailer section after the content. It can provide metadata whose final value becomes available while the content is being generated or processed. Its placement lets a sender stream content before that metadata is complete.^[ref]

For a concrete HTTP/1.1 example, define an illustrative extension field Example-Count to report the number of content octets and explicitly permit it in trailers. This is a definition for the example, not a standard field. A response can announce Transfer-Encoding: chunked and Trailer: Example-Count in its header section. Its following transfer-coded body, using visible escape notation, is 3\r\nabc\r\n0\r\nExample-Count: 3\r\n\r\n.

Here \r\n denotes a carriage-return/line-feed pair in the actual stream. The three-octet chunk supplies abc. The zero-sized chunk ends the chunks, and Example-Count: 3 then supplies the trailer metadata before the final empty line. The content is just abc; the trailer field is not appended to the representation data.^[ref]

The initial Trailer field names metadata the sender anticipates sending. A recipient can use this announcement to prepare its processing, but it does not guarantee that every announced field will actually arrive. The field value itself remains in the later trailer section.^[ref]

Placement is constrained by semantics. A sender must know that a field's definition permits trailer use; fields needed earlier for HTTP message framing, routing or content interpretation generally cannot simply be moved to the end. A recipient must not merge trailer fields into the header section unless their definitions explicitly permit and define the merge.^[ref]

Trailers can also be discarded in transit, so an application cannot assume that arbitrary metadata placed there will reach or be acted upon by every recipient. Keeping HTTP trailer fields distinct from HTTP header fields preserves the distinction between information available before content processing and information learned afterwards.^[ref]