Tales From The Command Line: Where Has My Bandwidth Gone? (iftop & SurplusMeter)

I believe I can safely say that the primary objective for users running OS X is to connect to the Internet to read mail, check out web sites, chat with friends, download new apps or grab/share multi-media content, etc. With bandwidth caps staring to become all the rage by the mega-providers, knowing how much you are consuming may be critical knowledge to hold back the costs/fees on your monthly bill.

This post covers two of three key programs for managing bandwidth: iftop (a command-line utility) and SurplusMeter (a GUI tool). The third utility – lsof (another command-line utility) – will require a dedicated article in-and-of itself.

How Much Am I Consuming?

SurplusMeter is a small tool from the fine folks over at SkoobySoft with one mission: to show you how much bandwidth you are consuming with the option to enter any known caps to ensure you are not over your limit. You can download SurpluMeter directly from their site. It is a PPC binary compatible down to OS X 10.3.9. For those who want to live life on the wild side, you can grab the source code and compile it yourself. I have built an OS X 10.5 compatible Universal Binary version which you can download via this post. No matter which way you decide to go, it is important to copy the application to your local volume as it runs an agent program – which runs in the background collecting bandwidth data – that will make it difficult to remove mounted volumes if you keep it running.

The main view of SurplusMeter is fairly straightforward. You can set which day to start the monthly tracking period on and specify your known bandwidth cap – which can also include upstream usage. Monitoring can be paused if you know you will be moving between networks, the collected data can be reset and you can even choose which interface to monitor via their “English” names vs OS X short device names (e.g. “Ethernet port” vs “en0“). For my example, I did a short sample of bandwidth on my AT&T 3G ExpressCard, hence the PPP modem selection. If there were hiccups during program execution or you know of other bandwidth usage on your connection not emanating from your Mac, you can add bytes to the current data collector.

Similarly, you can also remove bytes if you were measuring data on an interface that moved between networks.

SurplusMeter is kind enough to store its data in ~/Library/Application Support/SurplusMeter/surplusmeter_data.plist in a very human- and machine-readable format (a well-annoted Apple plist) so you can do what you like with it:

In general, it is a great, special purpose utility to have around.

What’s Going On Here?

SurplusMeter tells you that you are using bandwidth, but does not provide any further details. This is where tool number two – iftop – comes into play. You can grab a pre-built package of iftop (“interface top”) from AFP548.com or grab the source and try your hand at building it (you may need to download some support libraries). Users of various “ports” tools should be able to find iftop in one of the repositories.

Where the command-line tool top provides a mechanism for determining what processes are consuming precious system resources (in a very similar fashion to the Activity Monitor application), iftop does something similar for network usage on a particular interface.

You will need some more technical information to run iftop successfully. First, you will need to know which network interface you want to monitor. To find out which interfaces you have on your system, open Terminal.app and run the command:

ifconfig -l

My output from that shows:

lo0 gif0 stf0 en0 fw0 en1 vmnet8 vmnet1 ppp0

Interface en0 generally is equivalent to “Ethernet” and en1 is usually equivalent to your AirPort card. ppp0 would refer to most modems, including 3G cards. Additional interfaces may be related to a VMware or Parallels install, your local firewall interface or other local types of network devices.

With Terminal.app open, run iftop -h to see what the command line options are for the tool:

Synopsis: iftop -h | [-npbBP] [-i interface] [-f filter code] [-N net/mask]

   -h                  display this message
   -n                  don't do hostname lookups
   -N                  don't convert port numbers to services
   -p                  run in promiscuous mode (show traffic between other
                       hosts on the same network segment)
   -b                  don't display a bar graph of traffic
   -B                  Display bandwidth in bytes
   -i interface        listen on named interface
   -f filter code      use filter code to select packets to count
                       (default: none, but only IP packets are counted)
   -F net/mask         show traffic flows in/out of network
   -P                  show ports as well as hosts
   -m limit            sets the upper limit for the bandwidth scale
   -c config file      specifies an alternative configuration file

For this example, the most useful options are “-i” to let us choose which interface to monitor and “-P” to show which ports are in use. The tool requires elevated privileges to work so you have to run the following to start your view: sudo iftop -P -i ppp0 (again, replace “ppp0” with “en0” or “en1” or whatever interface you need to monitor).

You should see something similar to the following screen upon successful execution (minus the annotation):

The main part of the display lists, for each pair of hosts, the rate at which data has been sent and received over the preceding 2, 10 and 40 second intervals. The direction of data flow is indicated by arrows, <= and =>. So in this example, where I started iTunes just after kicking off iftop, we can see that:

  • (my local machine) made a series of http (web) requests to Apple servers
  • some of Apple’s servers do not resolve from IP addresses to host names
  • the average transfer rate over 40 seconds is between 0.2 kilobytes and 9 kilobytes per second

After quitting iTunes and running for a while, then letting it sit “idle” (not actively doing network activity), you can see that the pattern of usage can change dramatically.

While iftop can let you see more of what is going on, it cannot tell you which applications or processes are causing the usage. You can infer quite a bit (i.e. http traffic is most likely coming from your browser – but this is not necessarily the case as shown by the last screen), but finding out core details is where lsof can be of real value and will be covered in our next installment.

While I have presented a free way to monitor bandwidth usage, Guy Meyer has a set of tools – Net Monitor & Net Monitor Sidekick which do something similar but are not free (the Sidekick program is in beta which is expired so I was not able to test it).

If you are using any of these or similar tools to monitor bandwidth utilization, drop a note in the comments to share your insights with TAB readers and keep an eye out for our post on lsof!