fzdwx

fzdwx

Hello , https://github.com/fzdwx
HomeArchives

HTTP Protocol

#network#http#interview85
AI Translation
This post is translated from Chinese into English through AI.View Original
AI-generated summary
This article provides an overview of the HTTP/1.1 and HTTP/2 protocols. It explains the structure of HTTP/1.1 requests and responses, including the method, URI, headers, and body. It also discusses the main features of HTTP/1.1, such as persistent connections, caching, and chunked responses. The article then introduces the concepts of HTTP/2, including connections, streams, messages, and frames. It compares the composition of HTTP/1.1 and HTTP/2 messages and explains the different types of frames used in HTTP/2. The article also discusses why HTTP/2 is typically used with HTTPS and highlights the benefits of HTTP/2, such as multiplexing and header compression. It provides some useful tools for generating test certificates and debugging HTTPS connections.

We won't discuss the implementations before HTTP 1.1 because they have been outdated for too long and I haven't encountered them while browsing the internet, so let's mainly talk about HTTP/1.1 and HTTP/2.

HTTP/1.1#

Introduction to HTTP/1.1 Protocol Messages#

CRLF: \r\n

METHOD: HTTP request, GET, POST, PUT, DELETE, etc.

URI: Uniform Resource Identifier, for example, /, /index.html, etc.

HTTPVersion: Version number of the HTTP protocol, for example, HTTP/1.1, HTTP/2

HEADERS: Request headers, for example, Host:localhost, Accept: */*.

BODY: Request body, for example, a JSON data {"name":"fzdwx"}

HTTPStatus: HTTP response status, commonly used ones are 200, 404, etc.

HTTPStatusDesc: Description of the HTTP response status, OK corresponds to 200.

Request#

METHOD<SPACE>URI<SPACE>HTTPVersion
HEADERS
<CRLF>
BODY

Example:

GET /hello HTTP/1.1
Host: 192.168.1.107:8889
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.82 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9

Response#

HTTPVersion HTTPStatus HTTPStatusDesc
HEADERS
<CRLF>
BODY

Example:

If transfer-encoding: chunked is used in the response instead of Content-Length, it indicates a response with an undefined size, and the end is usually delimited by 0\r\n.

HTTP/1.1 200 OK
transfer-encoding: chunked
content-type: application/json; charset=utf-8

0/r/n

Main Features of HTTP/1.1#

  1. Default to keep-alive connections (Connection: Keep-alive), supporting multiple requests on a single TCP connection.
  2. Cache strategies, using Cache-Control, Expires, Last-Modified, ETag, etc. in the request headers for control.
  3. Allow response chunking, as mentioned above with transfer-encoding: chunked, allowing the server to send the response body in multiple parts.

However, there are still some issues, such as if a TCP connection is blocked, a new TCP connection will still be opened to handle the request.

H2#

Key concepts in HTTP2:

  1. Connection: A TCP connection contains one or more Streams, and all communication is done on one TCP connection.
  2. Stream: A bidirectional data flow that contains one or more Messages, each data flow has a unique identifier and optional priority information.
  3. Message: Corresponds to a request or response in HTTP/1.1, containing one or more Frames.
  4. Frame: The smallest unit of transmission, encoded in binary.

HTTP Communication Diagram

In HTTP/1.1, it consists of Start Line + header + body, while in H2, it consists of a HEADER Frame and multiple DATA Frames.

Frame#

There are usually some common fields, such as Length, Type, Flags, and Stream Id, as well as fields unique to each type.

The classification is as follows:

  • DATA: Used for transmitting the HTTP message body.
  • HEADERS: Used for transmitting header fields.
  • PRIORITY: Used to specify or re-specify the priority of referenced resources.
  • RST_STREAM: Used to notify the abnormal termination of a stream.
  • SETTINGS: Used to negotiate configuration data between the client and server, such as setting the initial bidirectional flow control window size.
  • PUSH_PROMISE: Server push permission.
  • PING: Used for calculating round-trip time and performing "liveness" check.
  • GOAWAY: Used to notify the peer to stop creating streams in the current connection.
  • WINDOW_UPDATE: Used to adjust the flow control of individual streams or the entire connection.
  • CONTINUATION: Specifically used to transmit large HTTP headers in multiple frames.

Why does H2 have to use HTTPS?#

This is not specified in the H2 standard. It is mainly for the convenience of upgrading/negotiating the HTTP protocol. There are usually two ways to confirm whether a web server supports H2:

  1. Set Upgrade: HTTP/2.0 and Connection: Upgrade,HTTP2-Settings, etc. in the request headers, similar to upgrading to Websocket.
  2. Use the ALPN (Application Layer Protocol Negotiation) field in TLS (Transport Layer Security), which can be determined during the Client Hello and Server Hello stages.

Most browsers now implement the second method, which is bound with HTTPS. However, if we don't use a browser for access, HTTPS can be omitted.

For more details, please refer to Protocol Negotiation in HTTP2.

Why can H2 achieve parallel response to requests?#

In HTTP/1.1, requests and responses are one-to-one. In the same connection, the client sends two requests one after another and receives a response from the server after a period of time. This response must correspond to the first request sent because there is no indicator to indicate which response corresponds to which request.

In H2, based on the design of Stream and Frame: Each Frame carries a Stream Id to identify whether it is data within the same Stream, and each Stream is independent of each other. This allows multiple requests/responses to be transmitted within a single TCP connection.

New Features of H2#

The core optimization of H2 over HTTP/1.1 is to use as few connections as possible.

  1. Multiplexing: Handling multiple requests/responses on a single TCP connection without opening additional TCP connections, achieved through Stream and Frame.
  2. Binary framing: Using Frame as the smallest unit of communication and using binary encoding.
  3. Header compression: Optimized using the HPACK algorithm.
    • Maintain a common static dictionary that contains common combinations of request headers as key-value pairs.
    • A dynamic dictionary that can be dynamically expanded (maintained separately for each connection).
    • Support Huffman coding (static Huffman code table).

    In HTTP/1, the message body can be compressed using gzip, but the request headers usually have no compression. Sometimes the request headers may even be larger than the request body.

  4. Request prioritization: Usually carried in HEADERS frames and PRIORITY frames, often depends on the server's level of support.

Tools#

Generate Test Certificates#

go run $GOROOT/src/crypto/tls/generate_cert.go --host localhost

Debug HTTPS with curl#

curl https://zcygov.cn -vv

Links#

Loading...
Ownership of this post data is guaranteed by blockchain and smart contracts to the creator alone.