应用层协议原理



针对不同的 HostName 建立了两个并行的 TCP 连接 (Q: 怎么通过工具观察图 2 和图 3 的两个连接是否是持续的)


GET / HTTP/1.1
Host: cnhktrz3k5nc.hack-challenge.lug.ustc.edu.cn:13202
Cookie: session=eyJ0b2tlbiI6IjU2ODpNRVVDSVFEOUtzZWdBT2c5ZE9jdGkzUmJPSUVTSTdSTWQ5cDRQaW8vdWNxTXY3V0dhUUlnR0dIMHVROHhISWI4SzBNT2lidlFjVk0xaGoxZEdQZGNOYkpGbXJyZE45az0ifQ.ZUGxTw.PrwGL6P9rTzYpTfdu21r7irxBjQ
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/118.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Te: trailers
Connection: close
----------------------------------------------------------
🌟 加上不显示的回车和换行符后,报文中的空格也是真实存在的 (感谢ChatGPT的帮助)
GET / HTTP/1.1\r\n
Host: cnhktrz3k5nc.hack-challenge.lug.ustc.edu.cn:13202\r\n
Cookie: session=eyJ0b2tlbiI6IjU2ODpNRVVDSVFEOUtzZWdBT2c5ZE9jdGkzUmJPSUVTSTdSTWQ5cDRQaW8vdWNxTXY3V0dhUUlnR0dIMHVROHhISWI4SzBNT2lidlFjVk0xaGoxZEdQZGNOYkpGbXJyZE45az0ifQ.ZUGxTw.PrwGL6P9rTzYpTfdu21r7irxBjQ\r\n
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/118.0\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8\r\n
Accept-Language: en-US,en;q=0.5\r\n
Accept-Encoding: gzip, deflate\r\n
Upgrade-Insecure-Requests: 1\r\n
Sec-Fetch-Dest: document\r\n
Sec-Fetch-Mode: navigate\r\n
Sec-Fetch-Site: none\r\n
Sec-Fetch-User: ?1\r\n
Te: trailers\r\n
Connection: close\r\n\r\n 🌟 在表面上看来末尾有两个空行
🌟 使用ASCII编码转换为16进制,数据被注入套接字后穿过传输层、网络层、数据链路层、物理层(中间存在分组(?)等情况)以比特流的形式经过网络基础设施
G E T SP / SP H T T P / 1 . 1 CR LF ...
47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a 48 6f 73 74 3a 20 63 6e 68 6b 74 72 7a 33 6b 35 6e 63 2e 68 61 63 6b 2d 63 68 61 6c 6c 65 6e 67 65 2e 6c 75 67 2e 75 73 74 63 2e 65 64 75 2e 63 6e 3a 31 33 32 30 32 0d 0a 43 6f 6f 6b 69 65 3a 20 73 65 73 73 69 6f 6e 3d 65 79 4a 30 62 32 74 6c 62 69 49 36 49 6a 55 32 4f 44 70 4e 52 56 56 44 53 56 46 45 4f 55 74 7a 5a 57 64 42 54 32 63 35 5a 45 39 6a 64 47 6b 7a 55 6d 4a 50 53 55 56 54 53 54 64 53 54 57 51 35 63 44 52 51 61 57 38 76 64 57 4e 78 54 58 59 33 56 30 64 68 55 55 6c 6e 52 30 64 49 4d 48 56 52 4f 48 68 49 53 57 49 34 53 7a 42 4e 54 32 6c 69 64 6c 46 6a 56 6b 30 78 61 47 6f 78 5a 45 64 51 5a 47 4e 4f 59 6b 70 47 62 58 4a 79 5a 45 34 35 61 7a 30 69 66 51 2e 5a 55 47 78 54 77 2e 50 72 77 47 4c 36 50 39 72 54 7a 59 70 54 66 64 75 32 31 72 37 69 72 78 42 6a 51 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 4d 6f 7a 69 6c 6c 61 2f 35 2e 30 20 28 4d 61 63 69 6e 74 6f 73 68 3b 20 49 6e 74 65 6c 20 4d 61 63 20 4f 53 20 58 20 31 30 2e 31 35 3b 20 72 76 3a 31 30 39 2e 30 29 20 47 65 63 6b 6f 2f 32 30 31 30 30 31 30 31 20 46 69 72 65 66 6f 78 2f 31 31 38 2e 30 0d 0a 41 63 63 65 70 74 3a 20 74 65 78 74 2f 68 74 6d 6c 2c 61 70 70 6c 69 63 61 74 69 6f 6e 2f 78 68 74 6d 6c 2b 78 6d 6c 2c 61 70 70 6c 69 63 61 74 69 6f 6e 2f 78 6d 6c 3b 71 3d 30 2e 39 2c 69 6d 61 67 65 2f 61 76 69 66 2c 69 6d 61 67 65 2f 77 65 62 70 2c 2a 2f 2a 3b 71 3d 30 2e 38 0d 0a 41 63 63 65 70 74 2d 4c 61 6e 67 75 61 67 65 3a 20 65 6e 2d 55 53 2c 65 6e 3b 71 3d 30 2e 35 0d 0a 41 63 63 65 70 74 2d 45 6e 63 6f 64 69 6e 67 3a 20 67 7a 69 70 2c 20 64 65 66 6c 61 74 65 0d 0a 55 70 67 72 61 64 65 2d 49 6e 73 65 63 75 72 65 2d 52 65 71 75 65 73 74 73 3a 20 31 0d 0a 53 65 63 2d 46 65 74 63 68 2d 44 65 73 74 3a 20 64 6f 63 75 6d 65 6e 74 0d 0a 53 65 63 2d 46 65 74 63 68 2d 4d 6f 64 65 3a 20 6e 61 76 69 67 61 74 65 0d 0a 53 65 63 2d 46 65 74 63 68 2d 53 69 74 65 3a 20 6e 6f 6e 65 0d 0a 53 65 63 2d 46 65 74 63 68 2d 55 73 65 72 3a 20 3f 31 0d 0a 54 65 3a 20 74 72 61 69 6c 65 72 73 0d 0a 43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 63 6c 6f 73 65 0d 0a 0d 0a
GET / HTTP/1.1
Host: cnhktrz3k5nc.hack-challenge.lug.ustc.edu.cn:13202 🌟 Web代理高速缓存要求
Cookie: session=eyJ0b2tlbiI6IjU2ODpNRVVDSVFEOUtzZWdBT2c5ZE9jdGkzUmJPSUVTSTdSTWQ5cDRQaW8vdWNxTXY3V0dhUUlnR0dIMHVROHhISWI4SzBNT2lidlFjVk0xaGoxZEdQZGNOYkpGbXJyZE45az0ifQ.ZUGxTw.PrwGL6P9rTzYpTfdu21r7irxBjQ
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/118.0 🌟 用户代理/向服务器发送请求的浏览器类型
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5 🌟 表示用户想获得网页的语言版本
Accept-Encoding: gzip, deflate
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Te: trailers
Connection: close 🌟 不需要使用持续连接,发送完请求的对象后就关闭这条连接

POST 报文POST / HTTP/1.1
Host: 202.38.93.111:12345
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/118.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest 🌟 通常被用于识别AJAX(Asynchronous JavaScript and XML)请求
Content-Length: 48 🌟 说明发送给接收方的消息主体的大小,即用十进制数字表示的八位元组的数目
Origin: http://202.38.93.111:12345
Connection: close
Referer: http://202.38.93.111:12345/
Cookie: session=eyJ0b2tlbiI6IjU2ODpNRVVDSVFEOUtzZWdBT2c5ZE9jdGkzUmJPSUVTSTdSTWQ5cDRQaW8vdWNxTXY3V0dhUUlnR0dIMHVROHhISWI4SzBNT2lidlFjVk0xaGoxZEdQZGNOYkpGbXJyZE45az0ifQ.ZUHm-g.kVPjkYgY94gYkzV03PmLKCKWuhI
QW5zd2VyMT0yMDIzLTA4LTEwJkFuc3dlcjI9SUNSUg==.txt 🌟 实体体
HEAD 方法,用一个 HTTP 报文进行相应但不返回请求对象HEAD / HTTP/2
Host: cnhktrz3k5nc.hack-challenge.lug.ustc.edu.cn:13202
Cookie: session=eyJ0b2tlbiI6IjU2ODpNRVVDSVFEOUtzZWdBT2c5ZE9jdGkzUmJPSUVTSTdSTWQ5cDRQaW8vdWNxTXY3V0dhUUlnR0dIMHVROHhISWI4SzBNT2lidlFjVk0xaGoxZEdQZGNOYkpGbXJyZE45az0ifQ.ZUGxTw.PrwGL6P9rTzYpTfdu21r7irxBjQ
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/118.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Te: trailers
HTTP/2 200 OK
Server: nginx
Date: Wed, 01 Nov 2023 05:39:21 GMT
Content-Type: text/html; charset=utf-8
Vary: Accept-Encoding
Vary: Cookie
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Referrer-Policy: no-referrer-when-downgrade
Permissions-Policy: interest-cohort=()
PUT 方法用于上传对象到 Web 服务器上的指定路径PUT /echo/put/json HTTP/1.1
Host: reqbin.com
Content-Type: application/json
Content-Length: 80
{
"Id": 12345,
"Customer": "John Smith",
"Quantity": 1,
"Price": 10.00
}
DELETE 方法删除 Web 服务器上的对象DELETE /echo/delete/json HTTP/1.1
Host: reqbin.com
Accept: */*
Authorization: Bearer mt0dgHmLJMVQhvjpNXDyA83vA_PxH23Y
HTTP/1.1 200 OK 🌟 协议版本 状态码 状态信息
Server: nginx
Content-Length: 90
Via: 1.1 google
Date: Tue, 31 Oct 2023 19:59:05 GMT 🌟 响应生成的时间
Age: 36693
Content-Type: text/html 🌟 实体体中的对象类型
Cache-Control: public,must-revalidate,max-age=0,s-maxage=3600
Connection: close 🌟 发送完报文后就关闭TCP连接
<meta http-equiv="refresh" content="0;url=https://support.mozilla.org/kb/captive-portal"/>

Last-Modified 首部标识的日期比在该首部中列出的值要更晚,表示条件匹配成功。
If-Modified-Since 和 If-Match 首部字段的参数304 Not Modified 的响应
200 OK 响应码,连同新版本的资源,再次回到步骤 2 

import smtpd
import asyncore
class CustomSMTPServer(smtpd.SMTPServer):
def process_message(self, peer, mailfrom, rcpttos, data):
print('Received message from:', peer)
print('Message addressed from:', mailfrom)
print('Message addressed to :', rcpttos)
print('Message length :', len(data))
print('Data:')
print(data)
server = CustomSMTPServer(('0.0.0.0', 25), None)
try:
asyncore.loop()
except KeyboardInterrupt:
server.close()
> telnet localhost 25
S: 220 smtp.example.com ESMTP Postfix
C: HELO relay.example.org
S: 250 Hello relay.example.org, I am glad to meet you
C: MAIL FROM:<[email protected]>
S: 250 Ok
C: RCPT TO:<[email protected]>
S: 250 Ok
C: RCPT TO:<[email protected]>
S: 250 Ok
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: `From: "Bob Example" <[email protected]>`
C: `To: "Alice Example" <[email protected]>`
C: `Cc: [email protected]`
C: `Date: Tue, 15 Jan 2008 16:02:43 -0500`
C: `Subject: Test message`
C:
C: Hello Alice.
C: This is a test message with 5 header fields and 4 lines in the message body.
C: Your friend,
C: Bob
C: .
S: 250 Ok: queued as 12345
C: QUIT
S: 221 Bye
{The server closes the connection}
HELO/EHLO:这些命令发出 “Hello” 并在客户端和服务器之间启动 SMTP 连接。“HELO” 是该命令的基本版本;“EHLO” 用于一种特别类型的 SMTP。MAIL FROM:这个命令告诉服务器谁在发送该电子邮件。如果 Alice 试图给她的朋友 Bob 发电子邮件,客户端可能会发送 “MAIL FROM:[email protected]”。RCPT TO:这个命令用于列出电子邮件的收件人。如果有**多个收件人,客户端可多次发送该命令。如上例子中,Alice 的电子邮件客户端将发送: “RCPT TO: [email protected]"。DATA:这个命令放在电子邮件的内容前,DATA 中的内容 (邮件报文格式)由 RFC 5322 - Internet Message Format 规定,并以 <CR><LF>.<CR><LF> 结尾RSET:该命令重置连接,删除所有以前传输的信息,但不关闭 SMTP 连接。RSET 在客户端发送了错误信息的情况下使用。QUIT:结束连接。HTTP 主要是一个拉协议 (pull protocol),SMTP 主要是一个推协议 (push protocol),可以想象一下木板和绳子的不同之处。一块木板可将某物向前推,但不能将它拉回来。绳子可以拉动某物,但不能推动它。SMTP 将电子邮件“推”到邮件服务器,而 HTTP 将 Web 对象“拉”到用户的应用程序中
| IMAP | POP 3 |
|---|---|
| 用户可以从任何设备上访问他们的电子邮件。 | 默认情况下,只能从下载邮件的设备上访问电子邮件。 |
| 服务器存储电子邮件;IMAP 充当服务器和客户端之间的中介。 | 若非另有配置,下载后,电子邮件就会从服务器上删除。 |
| 电子邮件不能离线访问。 | 电子邮件可以离线访问,但只能在其下载的设备上访问。 |
| 在用户点击之前,不会下载电子邮件的正文,但主题行和发件人姓名会迅速在电子邮件客户端弹出。 | 默认情况下,电子邮件会被下载到设备上,所以可能需要更长的时间来加载邮件内容。 |
| IMAP 需要更多的服务器空间,因为电子邮件不会从服务器上自动删除。 | POP 3 节省了电子邮件服务器的存储空间,因为电子邮件会自动从服务器中删除。 |
S: <wait for connection on TCP port 110>
C: <open connection>
S: +OK POP3 server ready <[email protected]>
C: APOP mrose c4c9334bac560ecc979e58001b3e22fb
S: +OK mrose's maildrop has 2 messages (320 octets)
C: STAT
S: +OK 2 320
C: LIST
S: +OK 2 messages (320 octets)
S: 1 120
S: 2 200
S: .
C: RETR 1
S: +OK 120 octets
S: <the POP3 server sends message 1>
S: .
C: DELE 1
S: +OK message 1 deleted
C: RETR 2
S: +OK 200 octets
S: <the POP3 server sends message 2>
S: .
C: DELE 2
S: +OK message 2 deleted
C: QUIT
S: +OK dewey POP3 server signing off (maildrop empty)
C: <close connection>
S: <wait for next connection>
APOP :APOP 命令使用了一个加密的哈希值(MD 5 散列值)来验证用户的身份。当客户端连接到邮件服务器时,服务器会发送一个带有时间戳的随机字符串。然后客户端将该字符串和用户的密码进行组合,并使用 MD5 算法进行哈希运算。最后,客户端发送用户名和计算得到的哈希值给服务器,以完成身份验证。STAT:获取收件箱中邮件的统计信息LIST:请求邮件服务器列出所有存储的邮件及其长度RETR:发送对应的邮件DELE:给邮件做删除标记QUIT:结束该 POP3 会话并删除标记为删除的邮件C: <open connection>
S: * OK IMAP4rev1 Service Ready
C: a001 login mrc secret
S: a001 OK LOGIN completed
C: a002 select inbox
S: * 18 EXISTS
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * 2 RECENT
S: * OK [UNSEEN 17] Message 17 is the first unseen message
S: * OK [UIDVALIDITY 3857529045] UIDs valid
S: a002 OK [READ-WRITE] SELECT completed
C: a003 fetch 12 full
S: * 12 FETCH (FLAGS (\Seen) INTERNALDATE "17-Jul-1996 02:44:25 -0700"
RFC822.SIZE 4286 ENVELOPE ("Wed, 17 Jul 1996 02:23:25 -0700 (PDT)"
"IMAP4rev1 WG mtg summary and minutes"
(("Terry Gray" NIL "gray" "cac.washington.edu"))
(("Terry Gray" NIL "gray" "cac.washington.edu"))
(("Terry Gray" NIL "gray" "cac.washington.edu"))
((NIL NIL "imap" "cac.washington.edu"))
((NIL NIL "minutes" "CNRI.Reston.VA.US")
("John Klensin" NIL "KLENSIN" "MIT.EDU")) NIL NIL
"<[email protected]>")
BODY ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 3028
92))
S: a003 OK FETCH completed
C: a004 fetch 12 body[header]
S: * 12 FETCH (BODY[HEADER] {342}
S: `Date: Wed, 17 Jul 1996 02:23:25 -0700 (PDT)`
S: `From: Terry Gray <[email protected]>`
S: `Subject: IMAP4rev1 WG mtg summary and minutes`
S: `To: [email protected]`
S: `Cc: [email protected], John Klensin <[email protected]>`
S: `Message-Id: <[email protected]>`
S: `MIME-Version: 1.0`
S: `Content-Type: TEXT/PLAIN; CHARSET=US-ASCII`
S:
S: )
S: a004 OK FETCH completed
C a005 store 12 +flags \deleted
S: * 12 FETCH (FLAGS (\Seen \Deleted))
S: a005 OK +FLAGS completed
C: a006 logout
S: * BYE IMAP4rev1 server terminating connection
S: a006 OK LOGOUT completed
主机名到 IP 地址转换 的目录服务


标志字段格式
Field Description Length (bits) QR Indicates if the message is a query (0) or a reply (1) 1 OPCODE The type can be QUERY (standard query, 0), IQUERY (inverse query, 1), or STATUS (server status request, 2) 4 AA Authoritative Answer, in a response, indicates if the DNS server is authoritative for the queried hostname 1 TC TrunCation, indicates that this message was truncated due to excessive length 1 RD Recursion Desired, indicates if the client means a recursive query 1 RA Recursion Available, in a response, indicates if the replying DNS server supports recursion 1 Z Zero, reserved for future use 3 RCODE Response code, can be NOERROR (0), FORMERR (1, Format error), SERVFAIL (2), NXDOMAIN (3, Nonexistent domain), etc. 4
问题 RR 格式
Field Description Length (octets) NAME Name of the requested resource Variable TYPE Type of RR (A, AAAA, MX, TXT, etc.) 2 CLASS Class code 2
回答 RR 格式
Field Description Length (octets) NAME Name of the node to which this record pertains Variable TYPE Type of RR in numeric form (e.g., 15 for MX RRs) 2 CLASS Class code 2 TTL Count of seconds that the RR stays valid (The maximum is 231−1, which is about 68 years) 4 RDLENGTH Length of RDATA field (specified in octets) 2 RDATA Additional RR-specific data Variable, as per RDLENGTH

