Cheap Process Monitoring: Echo, Watch, and Netcat

I frequently work with memcached, gearman, and a variety of other services that allow you to retrieve useful stats by connecting via telnet and issuing a status command. In a lot of cases, retrieving these stats in realtime is beneficial for debugging and diagnosing data pipeline issues.

Fortunately, a few basic tools found on almost any Linux system can make this easy.

The watch utility runs whatever command you provide it at the interval you specify. Here's the world's cheapest realtime clock.

$ watch -n 1 'date'

Netcat is too versatile to cover in depth in this post, but one of its most basic capabilities is accepting data via standard input, sending that data to a host / port combination of your choosing, and then dumping the result to standard output.

$ printf "HEAD / HTTP/1.0\r\n\r\n" | nc google.com 80 | grep Server
Server: gws

The above command takes the HTTP HEAD request from printf, sends it to netcat, which then passes it on to google.com on port 80. The results are printed to standard output, and the grep command limits the result output to the server returned (in this case gws).

Combining these two commands provides a ton of utility. Here are two useful examples that I use regularly.

1) Monitoring Gearman queues in realtime

watch -n 1 '(echo status; sleep 0.1) | nc localhost 4730'

publishMessage  4       0       1
consumeMessage  7       0       8
archiveMessage  28      0       12
.

2) Monitoring memcached get/set commands in realtime

watch -n 1 '(echo stats; sleep 0.1) | nc localhost 11211 | grep cmd'

STAT cmd_get 506
STAT cmd_set 8
STAT cmd_flush 0
STAT cmd_touch 0
STAT auth_cmds 0

Sensible Defaults with Vim-Sensible

If you have a fresh Vim install, and you'd like a sensible set of defaults, consider installing the vim-sensible plugin. If you've already installed Pathogen, installing the plugin can be accomplished as follows.

cd ~/.vim/bundle
git clone git://github.com/tpope/vim-sensible.git

Sort: Human Numeric

The sort command can accept input data formatted in human readable form. The du -h command is a longtime favorite for finding file sizes in a format that's easy to read with the naked eye. Coupled with sort, you can easily find and display the largest files in a directory tree.

$ du -h | sort -r --human-numeric | head

Update: I should have mentioned that --human-numeric is a recent addition to GNU coreutils, so this won't work on OS X or older distros. Here's a more generic solution (borrowed from here).

for i in G M K; do du -ah | grep [0-9]$i | sort -nr -k 1; done | head -n 11

Pathogen Makes Plugin Management Easy

I have been using Pathogen for installing new plugins whenever possible for quite a while now. In my opinion, Pathogen is the simplest path to extending your existing Vim runtime with additional functionality.

You can install Pathogen with the following command (assuming you're using a Linux operating system).

mkdir -p ~/.vim/autoload ~/.vim/bundle; \
curl -Sso ~/.vim/autoload/pathogen.vim \
https://raw.github.com/tpope/vim-pathogen/master/autoload/pathogen.vim

Once Pathogen is installed, enable it by adding the following to your .vimrc.

execute pathogen#infect()

Once Pathogen is enabled, you can install a wealth of plugins by simply cloning them to your ~/.vim/bundle directory. As an example, you can install the vim-sensible plugin as follows.

cd ~/.vim/bundle
git clone git://github.com/tpope/vim-sensible.git

The Wonderful "F" Key

If you press the "f" key in normal mode, Vim will move the cursor forward to whatever character you input after "f" is pressed. As an example, consider the following line:

a quick brown fox

If the cursor was at the absolute beginning of the line, and you pressed "fb" in normal mode, Vim would move the cursor so that it was positioned over the "b" in "brown".

If you press "F", Vim will move the cursor backwards instead of forward. Given the previous sentence, if pressed "Fq", and the cursor was at the end of the line, it would move to the "q" in "quick".

Back From The Dead

After an almost four year hiatus, I've decided to bring this blog back from the dead. Looking over the last decade, my career as a programmer has been an interesting and challenging endeavor, and over that time period, Vim has been a constant in my professional toolbox. There are countless work days where Vim has saved me hours of pushing bytes around the screen, and a lesser editor would have fallen short of the task.

With this in mind, I feel that I still have a good deal of useful information to share. With the advent of Github and other powerful open source tools, the landscape of the hacker community has evolved. The tools are better, and with reasonable motivation, it feels like any problem is solvable.

In the midst of all of this change, I still believe that Vim is the best tool for the admittedly rote task of munging source code into the best state one can manage given the time constraints and project demands that many of us face.

In lieu of my longstanding love for this text editor (and Linux tools in general), I will resume posting tips to this blog (as they come to mind) in hopes that they can on occasionally help all of you make your hardest problems a little easier and your easy problems a little more fun. Happy Vimming! -Travis