Watching the HTTP traffic

Recently I've been impressed by the variety of web site issues that can be diagnosed by watching the http traffic from your web browser to/from the web server(s) serving a page. The simplest way to watch this traffic is by tools like IEWatch (you have to pay for IEWatch but the other tools mentioned are free to use) and IBM's Page Detailer for Internet Explorer or Firebug for Firefox. You could also use lower-level network protocol analyzers like Ethereal (multiple platforms) or iptraf (for Linux and perhaps other command line interfaces) but those are less straightforward to use. Here's an example of what Firebug for Firefox will show you:

If you haven't used any of these tools before, I highly recommend trying it (probably the easiest to use and get to know quickly is Firebug). If nothing else you'll get an appreciation for what happens when a page is loaded.

For troubleshooting, this also helps in a variety of cases:

• When using a technology that's new to you, you can better see how things work. For instance, you may want to discover if resources are being loaded into the browser immediately or only upon action by the user -- that's a piece of cake with Firebug and the other tools above. Moreover, you can figure out what's going wrong with new tools/platforms you're working with (and, if you see requests that you're not expecting, you have something specific to search for online).
• In some cases, there are strange differences between browsers which can be better diagnosed with http analysis. For example, recently we were trying to look more closely at some javascript that was running successfully in Firefox but not in IE when dealing with multibyte characters. By using Firebug and IEWatch, it was easy to determine that it had to do with the number of bytes we were trying to deal with, and it was a hard-edged success/fail between it working and not (a hypothesis before was that it "sometimes" worked, which in a sense was true but this allowed us to better understand *why*).

Some other ways that reviewing the HTTP traffic can be helpful: confidently confirming standard browser behavior before designing solutions (for instance, looking at exactly how/when different browsers set the referrer in the HTTP request) and also reviewing your pages for performance improvements.