Categories
Posts

Timing Details With cURL

Jon’s recent Find the Time to First Byte Using Curl post reminded me about the additional timing details that cURL can provide.

cURL supports formatted output for the details of the request ( see the cURL manpage for details, under “-w, –write-out <format>” ). For our purposes we’ll focus just on the timing details that are provided.

Step one: create a new file, curl-format.txt, and paste in:

time_namelookup: %{time_namelookup}
time_connect: %{time_connect}
time_appconnect: %{time_appconnect}
time_pretransfer: %{time_pretransfer}
time_redirect: %{time_redirect}
time_starttransfer: %{time_starttransfer}
———
time_total: %{time_total}

Step two, make a request:

curl -w "@curl-format.txt" -o /dev/null -s http://wordpress.com/

What this does:

  • -w "@curl-format.txt" tells cURL to use our format file
  • -o /dev/null redirects the output of the request to /dev/null
  • -s tells cURL not to show a progress meter
  • http://wordpress.com/ is the URL we are requesting

And here is what you get back:

time_namelookup: 0.001
time_connect: 0.037
time_appconnect: 0.000
time_pretransfer: 0.037
time_redirect: 0.000
time_starttransfer: 0.092
———
time_total: 0.164

Jon was looking specifically at time to first byte, which is the time_starttransfer line. The other timing details include DNS lookup, TCP connect, pre-transfer negotiations, redirects (in this case there were none), and of course the total time.

The format file for this output provides a reasonable level of flexibility, for instance you could make it CSV formatted for easy parsing. You might want to do that if you were running this as a cron job to track timing details of a specific URL.

For details on the other information that cURL can provide using -w check out the cURL manpage.

42 replies on “Timing Details With cURL”

Nice. How do I also include the current date+time of when the curl initiated the HTTP request?

@Joseph,

That’s what I did. But I was wondering if there was better way using just the curl command.

Thanks! 🙂

Can anybody tell me when ever I am sending a curl request (curl http://x.x.x.x/file_name) if I want to parse the size of the file then how to do?
I already trace the size from header.(i dump the header bu “-D” and redirect it).but it is taking so much time.

If the server includes the ‘Content-Length’ header in the response then that will give you the size of the file. If it is a large file and you don’t want to download the whole thing just to get the size, try making a HEAD request. Here is an example:


curl -v --compressed -I https://s1.wp.com/home.logged-out/css/homepage-one-field-long.css > /dev/null

The response includes “Content-Length: 35326”.

thank you sir,
but the response still contain 6-7 line of information including “200 OK”,”last modification”,”accept-range” ,”content-length” etc i just want 1 line information i.e “content-length : 23456 “.can it be possible?

regards
swa

If in curl I want to extract an exact size data let curl http://x.x.x.x/ 1.doc 1234
where 1.doc is of size 30000 bytes and i want 1234 bytes from that file 1.doc. Whenever I will send the request the curl request will extract for 1234 byte data only. Is there any way to do this.

OK still thank you. Then is there any way to send multiple curl request in parallel . Multiple means not 10 or 100 or 1000 its like 100,000 request. I tried the method
curl http://x.x.x.x/{1.doc,2.doc,3.doc………} but it limits upto 10500.

What about setting timeout option?
From manpage:
–connect-timeout
-m, –max-time

Hello,

Need help!

Why is total_time not equal to (time_namelookup + time_connect + time_appconnect + time_pretransfer + time_redirect + time_starttransfer)?

Does total_time really count in milliseconds the request time from the beggining to the end {page loaded} ?

BR
Christopher

The cURL man page – http://curl.haxx.se/docs/manpage.html – describes total_time as:

time_total The total time, in seconds, that the full operation lasted. The time will be displayed with millisecond resolution.

While the time breakdown from cURL can be useful, it doesn’t breakdown all of the various times for a request. For instance WebPageTest will report times for:

– DNS lookup
– Initial connection
– TLS negotiation
– Time to first byte
– Content download

Nice post.
Can you please help by specifying how to get these timing details while using libcurl in C/C++?

Thank you so much for this tip, this is awesome!
20 years using curl and never figured out about using a file to format the output.

Could someone explain the meaning of each parameter?

time_namelookup:
time_connect:
time_appconnect:
time_pretransfer:
time_redirect:
time_starttransfer:
time_total:

Almost – in later versions, you have to put “n” at the end of each line in the format file.

Leave a Reply

Your email address will not be published. Required fields are marked *