Seeing the exact text of an HTTP request for Troubleshooting

Sometimes it is helpful to see the *EXACT* response from a server for an HTTP request. There are numerous methods of achieving this result.

To see the full HTTP header of a request you can use a site like Rex Swain's HTTP Viewer.

Using the HTTP viewer you can easily see the header information from a request for http://www.google.com which I pasted below after running it.

HTTP/1.1·200·OK(CR)(LF)
Cache-Control:·private(CR)(LF)
Content-Type:·text/html;·charset=UTF-8(CR)(LF)
Set-Cookie:·PREF=ID=074c8cb0423601b2:TM=1193758394:
LM=1193758394:S=lvyqhAAv4fz-RLO9;·expires=Thu,
·29-Oct-2009·15:33:14·GMT;·path=/;·domain=.google.com
(CR)(LF)
Server:·gws(CR)(LF)
Transfer-Encoding:·chunked(CR)(LF)
Date:·Tue,·30·Oct·2007·15:33:14·GMT(CR)(LF)
Connection:·Close(CR)(LF)
(CR)(LF)

This is of course only a portion of the total results. With the header request utility you can specify an alternate user agent to see how your script responds. Visit Rex's site for more.

Alternatively you can use an application called cURL to request the contents of a site using a variety of protocols. You are effectively interacting with a site as a command line would. From the cURL site:

curl is a command line tool for transferring files with URL syntax, .... HTTP form based upload, proxies, cookies, user+password authentication ... and a busload of other useful tricks.

They have a download wizard to get the correct flavor or cURL for your needs here. I use a number of different computers, but at the moment I am on a Win XP box and the wizard took me to this download page.

I downloaded the ZIP package and extracted it to the following location. Note it does not have an installer, just extract the folder to a location you can get to.

c:\ program files \ cURL \ <paste here>

To run cURL on an XP box open the Command Prompt (cmd.exe) and navigate to that folder (cd program files, etc)

Once in the folder you can type a command like this:

curl http://www.schipul.com >c:\temp schipulDownloadTest.txt

which creates a text file of the contents of that page. Of course this is a rudimentary example that you could achieve much easier by simply doing a view source and save on a site. So explore more with cURL to learn more about what you can do!