curl / httpstat 分析網頁下載各階段的時間

2020060416:02
利用 curl 來測試網頁下載時
查詢各個連接階段所花費的時間,例如 DNS查詢時間、下載時間、SSL連接花費的時間...

把這段存為 curltime.sh
curl -w @- -o /dev/null -s "$@" <<'EOF'
    time_namelookup:  %{time_namelookup}\n
       time_connect:  %{time_connect}\n
    time_appconnect:  %{time_appconnect}\n
   time_pretransfer:  %{time_pretransfer}\n
 time_starttransfer:  %{time_starttransfer}\n
                    ----------\n
         time_total:  %{time_total}\n
     speed_download:  %{speed_download} bytes/s\n

EOF
 
用法:
$ ./curltime.sh https://www.pchome.com.tw/
    time_namelookup:  0.001854  DNS 查詢花費的時間
       time_connect:  0.013127  TCP Connection,也就是 TCP 三向交握的时间(Three Way Handshake)
    time_appconnect:  0.067827  SSL/TSL handshake 的時間 (若是 http 這兒就是 0)
   time_pretransfer:  0.068630  
 time_starttransfer:  0.086331  發出 request 後直到收到網頁的第一字元花費的時間(FTTB)
                    ----------
         time_total:  0.188357
     speed_download:  1042845.000 bytes/s
參考:
Timing Details With cURL

time_pretransfer:
The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.


其它 curl 變數:
http_code The numerical response code that was found in the last retrieved HTTP(S) or FTP(s) transfer. In 7.18.2 the alias response_code was added to show the same info.
http_connect The numerical code that was found in the last response (from a proxy) to a curl CONNECT request. (Added in 7.12.4)
http_version The http version that was effectively used. (Added in 7.50.0)
local_ip The IP address of the local end of the most recently done connection - can be either IPv4 or IPv6 (Added in 7.29.0)
local_port The local port number of the most recently done connection (Added in 7.29.0)
num_connects Number of new connects made in the recent transfer. (Added in 7.12.3)
num_redirects Number of redirects that were followed in the request. (Added in 7.12.3)
response_code The numerical response code that was found in the last transfer (formerly known as "http_code"). (Added in 7.18.2)
scheme The URL scheme (sometimes called protocol) that was effectively used (Added in 7.52.0)
size_download The total amount of bytes that were downloaded.
size_header The total amount of bytes of the downloaded headers.
size_request The total amount of bytes that were sent in the HTTP request.
size_upload The total amount of bytes that were uploaded.
speed_download The average download speed that curl measured for the complete download. Bytes per second.
speed_upload The average upload speed that curl measured for the complete upload. Bytes per second.
ssl_verify_result The result of the SSL peer certificate verification that was requested. 0 means the verification was successful. (Added in 7.19.0)
time_appconnect The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed. (Added in 7.19.0)
time_connect The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
time_namelookup The time, in seconds, it took from the start until the name resolving was completed.
time_pretransfer The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
time_redirect The time, in seconds, it took for all redirection steps including name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections. (Added in 7.12.3)
time_starttransfer The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
time_total The total time, in seconds, that the full operation lasted.
url_effective The URL that was fetched last. This is most meaningful if you've told curl to follow location: headers.  
完整資料:   https://curl.haxx.se/docs/manpage.html

 

httpstat

另外有個  httpstat 工具
把上面 curl 的結果畫成更易理解的圖表
https://github.com/davecheney/httpstat
It's like curl -v, with colours.
 
$ ./httpstat.py https://www.yahoo.com.tw/
Connected to 124.108.115.101:443 from 192.168.0.4:55853

HTTP/2 301
date: Sun, 14 Jun 2020 03:14:51 GMT
p3p: policyref="https://policies.yahoo.com/w3c/p3p.xml"
cache-control: max-age=3600, public
location: https://tw.yahoo.com/
content-length: 0
content-type: text/html; charset=UTF-8
:::略過header:::

  DNS Lookup   TCP Connection   TLS Handshake   Server Processing   Content Transfer
[     5ms    |      15ms      |     45ms      |       16ms        |        0ms       ]
             |                |               |                   |                  |
    namelookup:5ms            |               |                   |                  |
                        connect:20ms          |                   |                  |
                                    pretransfer:65ms              |                  |
                                                      starttransfer:81ms             |
                                                                                 total:81ms  

備註
Server Processing = time_starttransfer - time_pretransfer
TLS Handshake    = time_pretransfer - time_connect