Cheap Process Monitoring: Echo, Watch, and Netcat

I frequently work with memcached, gearman, and a variety of other services that allow you to retrieve useful stats by connecting via telnet and issuing a status command. In a lot of cases, retrieving these stats in realtime is beneficial for debugging and diagnosing data pipeline issues.

Fortunately, a few basic tools found on almost any Linux system can make this easy.

The watch utility runs whatever command you provide it at the interval you specify. Here's the world's cheapest realtime clock.

$ watch -n 1 'date'

Netcat is too versatile to cover in depth in this post, but one of its most basic capabilities is accepting data via standard input, sending that data to a host / port combination of your choosing, and then dumping the result to standard output.

$ printf "HEAD / HTTP/1.0\r\n\r\n" | nc google.com 80 | grep Server
Server: gws

The above command takes the HTTP HEAD request from printf, sends it to netcat, which then passes it on to google.com on port 80. The results are printed to standard output, and the grep command limits the result output to the server returned (in this case gws).

Combining these two commands provides a ton of utility. Here are two useful examples that I use regularly.

1) Monitoring Gearman queues in realtime

watch -n 1 '(echo status; sleep 0.1) | nc localhost 4730'

publishMessage  4       0       1
consumeMessage  7       0       8
archiveMessage  28      0       12
.

2) Monitoring memcached get/set commands in realtime

watch -n 1 '(echo stats; sleep 0.1) | nc localhost 11211 | grep cmd'

STAT cmd_get 506
STAT cmd_set 8
STAT cmd_flush 0
STAT cmd_touch 0
STAT auth_cmds 0

Sort: Human Numeric

The sort command can accept input data formatted in human readable form. The du -h command is a longtime favorite for finding file sizes in a format that's easy to read with the naked eye. Coupled with sort, you can easily find and display the largest files in a directory tree.

$ du -h | sort -r --human-numeric | head

Update: I should have mentioned that --human-numeric is a recent addition to GNU coreutils, so this won't work on OS X or older distros. Here's a more generic solution (borrowed from here).

for i in G M K; do du -ah | grep [0-9]$i | sort -nr -k 1; done | head -n 11