Character Above, Below

If you're in insert mode, and you want to insert the character above the cursor, hit ctrl-y. To insert a character below the cursor, hit ctrl-e. Alternatively, if you want to grab entire words above and below, you can use the following mappings (which use text objects).

inoremap <C-Y> <Esc>klyiWjpa
inoremap <C-E> <Esc>jlyiWkPa

Rebuild a Debian Package

If you use Debian, Ubuntu, or some other Debian derived Linux distribution, there may come a time when a packages default configuration options don't meet your needs. This generally leaves you two options. You can either build from source and no longer have the luxury of the package manager maintaining your package, or you can rebuild the package with the custom options included in the build. This is easier than it sounds and really only involves a few steps. In this example, I'll rebuild netcat.

1) make sure you have the packages required to re-build a .deb

apt-get install devscripts build-essential fakeroot

2) fetch the source for the package to re-build

apt-get source netcat

3) fetch dependent packages for the build

sudo apt-get build-dep netcat

4) unpack vendor specific source archives

# determine appropriate dsc file
ls *dsc
netcat_1.10-33.dsc
# run dpkg-source on dsc file
dpkg-source -x netcat_1.10-33.dsc

5) cd into source directory

cd netcat-1.10

6) modify source if necessary, apply patches, etc...

7) set custom build options if desired (optional)

# fake DEB_BUILD_OPTIONS here, just for the sake of example...
DEB_BUILD_OPTIONS="--enable-sockets" CC=gcc-3.4 fakeroot debian/rules binary

8) build the package

dpkg-buildpackage -rfakeroot -b

9) install the package ;-)

sudo dpkg -i ../netcat_1.10-33_i386.deb

Quick Paste

You may have noticed that pasting outside text into Vim from insert mode can lead to awkwardly stair-stepped text. You may also know that this is easily avoidable via :set paste from normal mode. I paste from outside often enough, that I've added the following to my vimrc making it that much easier.

set pastetoggle=<F5>

Repeat Last Macro

There are times I want to repeat a macro just a few times. I know you can do count@{0-9a-z":*} but sometimes I don't feel like counting. You can use @@ to repeat the last macro, and it's an easy command to type very quickly, so it fits the bill.

Bash For Loops

I use bash and zsh quite a bit in system admin tasks. One thing I find myself needing to do repeatedly is run a command in a loop a given number of times. I'm very well acquainted with cron, but there are often instances where I prefer to run a bash loop inside of a screen session while logging to a file. This gives a lot more control over bleeding edge scripts, and it's easier to "check in" and see what's going on. Here are some simple ways to loop with bash:

# C style for loop

for ((i=0; i<=100; i+=1)); do
  # some command
  sleep 1
done

# quick loop

for I in 1 2 3; do
  echo $I
  sleep 1
done

# infinite loop

while [ 1 ]; do
  # some repeated command
  sleep 60
done

# file loop

for I in `ls *.txt`; do
  echo "file $I"
done

Vim Visual Reference

For any visual learners in the crowd, here's a very nice visual Vim reference. There are a lot of cheat sheets out there, and I may add a few more to this post down the road.

Vim Visual Reference.

Finding a File's Owner

I generally gravitate towards Debian derived Linux distributions; although, I spend quite a lot of time on the Red Hat side of the tracks as well. My primary reason for liking Debian and Ubuntu is the package management. I plan on doing a whole series on apt and dpkg tricks in the future, but for the moment, here's a really easy way to find out what package a file belongs to.

travis@travis-desktop:/home/travis% dpkg -S /bin/bash
bash: /bin/bash

Only install from tarballs as a last resort, and you should be able to track down over 90% of the files on your system.

Stripping Non-Printable Characters

Here's a quick way to strip non-printable characters in PHP. This is pretty handy for cleaning data before putting it in a DB.

$val = preg_replace('/[^\r\n\t\x20-\x7E\xA0-\xFF]/', ' ', $val)


Being a Perl compatible regexp, you can use it in your language of choice so long as it supports PCRE.

UPDATE:

Steve Laniel reminded me that you can use a Posix regexp to do roughly the same thing:

$val = preg_replace( '/[^[:print:]]/', '', $val )

This is a lot simpler for most cases; although, the patterns are slightly different (Posix is \x20-\x7E).

Easier Search + Sub

I can't say how many times I've tried to work out a complex substitute command in Vim only to have the pattern fail and not match what I wanted. I've recently come across an easier way to accomplish a substitution on a pattern that's already guaranteed to match. Say you have the following data in your editor.

"this is a line 12345"

Execute a search (instead of a sub):
" match digits
/\d\+

Now execute a sub with no pattern:

%:s//7890/g

The result would be:

"this is a line 7890"

Obviously this is a simple example, but I find this technique handy whenever I'm working out complex patterns.

Be a Better Programmer

Years ago, I became interested in Lisp and embarked on a path from which I would never return. I learned a good chunk of the language and spent a lot of late nights sitting inside the interactive shell provided by the CMUCL Lisp interpreter. While I don't do much Lisp programming these days, I've since incorporated elements of functional programming into all the code I write. Two basic tenants I've adopted as widely as possible are, "rely on return values whenever possible" and "avoid side effects". You can apply those tenants to OOP, procedural programming or really any style. Basically, avoid passing a pointer (or reference) to a function (or method) unless performance dictates otherwise. Rely on a single return value whenever possible. Returning an array is fine if it makes sense in the context of the method (array_unique() for example), but if your method is returning two distinctly different values, it might be a good idea to refactor. Avoiding side effects means that every method you write should do one thing. If your method prints and returns a data structure, consider refactoring into two methods (one that returns, one that prints). Return values should be as consistent as possible across your program.

The Ruby programming language works on the "principle of least surprise". You don't have to be a Ruby programmer to follow this very simple but effective rule. No matter what language you're writing in, try to think of what the "average programmer" would expect your method to accept and return. There's a time and a place for condensing twenty lines of code into two or three insanely terse pieces of syntactic sugar, but simplicity generally rules the day.

Python, Perl, Ruby, and PHP provide a lot of the facilities that made Lisp so popular to begin with. One such facility is an interactive shell. When I'm writing code, I don't waste time guessing exactly how a method or piece of code is going to behave. I have an interactive shell running alongside my editor window most of the time. This allows me to quickly evaluate a methods behavior if I'm not sure exactly what it will return. Here are instructions on firing up interactive shells in those languages.

If you don't currently program interactively, I highly suggest trying it. You'll get your work done faster and there will be less bugs at runtime.

Python:

Being a language with batteries included, all you have to do is invoke the python interpreter, and it drops you directly into an interactive shell.

travis@travis-desktop:/home/travis% python
Python 2.5.1 (r251:54863, Mar 7 2008, 04:10:12)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 1 + 1
2

Perl:

Perl's debugger provides an interactive shell. It doesn't have readline support by default, which is a minus, but it still gets the job done effectively.

travis@travis-desktop:/home/travis% perl -d # press ctrl-d after you hit enter (sends EOF)

Loading DB routines from perl5db.pl version 1.28
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

Debugged program terminated. Use q to quit or R to restart,
use o inhibit_exit to avoid stopping after program termination,
h q, h R or h o to get additional info.

DB<1> x 1 + 1
0 1

Ruby:

Ruby uses a program called irb to provide an interactive shell. Depending on your Ruby distribution you may have to install it separately using your package manager of choice. Once it's installed, irb provides readline support and is on par with the python's interactive shell.

travis@travis-desktop:/home/travis% irb
irb(main):001:0> 1 + 1
=> 2

PHP:

I have pretty mixed feelings about PHP, but that's another rant entirely. PHP includes an interactive mode out of the box, but a fatal error will drop you back to the command-line. Either way, it's better than nothing.

travis@travis-desktop:/home/travis/Apps% php -a
Interactive mode enabled

<?
echo 1 + 1, "\n";
2
?>

A better solution is to use this third-party app:

http://jan.kneschke.de/projects/php-shell/

Once it's installed, you have a full-featured interactive shell with readline support. It also handles exceptions gracefully.

travis@travis-desktop:/home/travis% php-shell.sh
PHP-Shell - Version 0.3.1, with readline() support
(c) 2006, Jan Kneschke <jan@kneschke.de>

>> use '?' to open the inline help

>> 1 + 1
2

Slammed

Sorry, no tip in the last day or so. I've been completely slammed at work launching our new product, Grooveshark Lite.

History

Most people know you can use the Unix history command to see what's been done on the command-line recently.

travis@travis-desktop:/home/travis% history
9985 ls
9986 grep -ri pidgin
9987 grep -ri pidgin *
9988 cd ..
9989 ls
9990 cd .gnome2
9991 grep -ri pidgin *
9992 cd ..
9993 cd .purple
9994 ls
9995 vim prefs.xml

What some people don't know, is you can execute anything in the history list very easily. Just use !number where number is the item number on the left-hand side of the history output.

!9995 would execute `vim prefs.xml'.

Using Awk to Grab a Column

A lot of times when hacking on the command-line, I need to filter a particular column coming in from a pipe. A good example would be grabbing a pid from ps output or cat'ing a file with mysql process status and grabbing mysql thread id's for a subsequent kill. I generally use awk to do this because it's installed everywhere, and it makes doing this very simple. Take the following example:

user@host.org [~]# ps aux
root 2 0.0 0.0 0 0 ? S Mar09 1:04 [migration/0]
root 3 0.0 0.0 0 0 ? SN Mar09 0:10 [ksoftirqd/0]
root 4 0.0 0.0 0 0 ? S Mar09 1:14 [migration/1]
root 5 0.0 0.0 0 0 ? SN Mar09 0:10 [ksoftirqd/1]

Now to grab the pid's (2,3,4,5):

[~]# ps aux | awk '{print $2}'

2
3
4
5

One step further to kill those pid's (you wouldn't want to kill these, but...)

[~]# ps aux | awk '{print $2}' | xargs kill -15

Last Jump Location

This technique was used in the "quick uncomment" tip I posted recently, but I figured I would mention it explicitly. A lot of times when I'm editing a file, I'll set a mark at my current location because I need to do something like jump to the bottom of the file and change something, and I anticipate needing to return to the current line. If you know you're going to quickly return to the current position, there's no need to set a mark. Just hit `` and it will return you to the position you were at before the last jump.

Quickly Truncate a File

Quick shell tip:

If you want to truncate a file to zero bytes from the command-line, here's an easy way.

user@host [~]$ > filename

I used to use:

user@host [~]$ cp /dev/null filename

But the first approach is less typing.

Ways to Avoid the Esc Key

I've been running this into the ground, but reaching for the Esc key is inefficient on modern keyboard layouts. I've spent some time recently investigating ways to avoid reaching way the hell over to the top-left hand corner of the keyboard in an effort to maintain focus on the home row. I struck gold with a few of the ideas presented on this Vim tip. Here's my take on what works best.

1) Use Ctrl-[

This is what I've been doing until recently; although it's a bit awkward as far as key chords are concerned. It still beats the hell out of going all the way to Esc, and it's built right into Vi(m), so there aren't any fancy tricks required to make it work.

2) Use Ctrl-c

Another builtin key chord that will take you from insert to normal mode. Depending on your keyboard and personal preference, it may feel better than Ctrl-[.

3) Map a combo inside Vim

I've seen people suggest two combos that were particularly attractive from an efficiency standpoint.

map! ii <Esc> " map ii to Esc
map! ;; <Esc> " map ;; to Esc

With these, you're typing in insert mode, hit `;;' and you're back in normal mode. From a touch-typist point of view `ii' is particularly attractive but has the obvious downside of not allowing you to type a literal "ii" in your document without typing it twice.

4) Remap Caps lock

Caps lock is placed in a very convenient location and is very rarely used by most programmers. Reclaim this valuable real estate by mapping it to Esc as it was on the original HP 9000 ITF HIL keyboards. This is the approach I'm using right now, and I LOVE IT. Remapping the key on X Windows is done as follows.

a) Create a file in your home directory called .Xmodmap

# begin -------------
remove Lock = Caps_Lock
keysym Caps_Lock = Escape
# end ----------------

b) Create a reference to .Xmodmap in your .xinitrc

# begin ---------------
if [ -f ~/.Xmodmap ]; then
xmodmap ~/.Xmodmap
fi
# end -----------------

If you don't want to restart X, you can manually run xmodmap ~/.Xmodmap after you create the first file.

5) Avoid shift-space Mapping

Quite a few people recommend mapping the shift-space key chord to Esc; however, many terminals can't differentiate between shift-space and regular ol' space. Gnome terminal is one of them, so if you run Linux, there's a good chance this mapping won't work for you.

Restore a Deleted SVN Directory

Occasionally when working with a group of developers, someone (and I'm not going to name names...) will accidentally delete an entire directory of the subversion trunk and commit that change. Luckily, it's not the end of the world if this happens. Change directories into the root of your subversion source tree and issue a merge command as follows:

svn merge -r [bad revision]:[good revision] .


example:

svn merge -r 13831:13830 .

After that, checkin your changes, and you're back in business. You can determine revision numbers by doing something like:

svn log -v . | less

More Global Fun

Today I had a friend ask me, "is there a way to load every line matching a pattern into a single register?" This lead me to look into a way to append to a register. I've come to find out that yanking something into a register using a capital rather than lowercase letter appends instead of overwriting the register. From there, it was just a matter of using the global command to invoke a yank on all the matches. Here's the result.

:%g/.*foo.*/normal "Ayy/

Note that `A and `a share the same register. So you can simply put using "ap (as usual). Another neat trick is a quick way to clear a register. Registers often persist among sessions (depending on your Vim setup). You can clear any register a to z as follows.

:let @a=@_

I knew there was a use for the blackhole register ;-).

Quick Uncomment

This tip was contributed by my friend (and expert Vim user), Christopher Suter. BTW, if anybody else has a tip they would like to contribute, just send me an email via tinymountain at gmail dot com.

To uncomment a /* */ - style block comment, put the cursor on the beginning or ending tag - whichever is closest - and in normal mode, do %dd``dd

this sequence jumps to the matching 'brace' (/* or */) deletes that one, then jumps to the previous jump point, which is of course the first 'brace.' you can replace dd with xx if the braces are not on their own lines for example:

/* this.commentedInvocation(); */

A really handy idiom that is great for large block-commented code.

Thanks Chris!

Using Vim Effectively

This is going to be sort of a common sense post aimed at newer Vim users. One thing I notice is that a lot of people start using Vim in hopes of improving their efficiency and speeding up their editing tasks. This makes sense because Vim can easily allow you to edit text an order of magnitude faster than an ordinary text editor; however, getting these kinds of gains out of Vim requires you to adopt the philosophy that it was built on. Probably the most basic and fundamental reason for using a modal editor (such as Vim) is that your fingers rarely have to leave the home-row of keys. A lot of new Vim users learn the basics of how to save and open a file and then navigate around Vim using the arrow keys to get where they want to go. This is probably the single worst habit new Vim users get into, and it destroys productivity. If you're one of those Vim users, please read the following tips as they will help you get into good habits that will ensure you get the most out of using Vim.

1) Start with vimtutor:

vimtutor starts of with the most fundamental and basic skills. To use Vim effectively, you need to have these skills committed to memory and "automatic".

2) Never use the arrow keys:

You simply NEVER need to use the arrow keys in Vim. If you are using them, go back to vimtutor and learn to navigate using hjlk. You'll thank yourself for taking the time to learn the "right way" later.

3) Avoid the Esc key:

The Esc was a mis-step in Vi's initial design. It requires you to lose focus on the home row. Use Ctrl-[ instead, or better yet, remap CAPS Lock to either Esc or Ctrl. I'll add a tip for how to do this in the future.

*update*:

My friend Skyler tells me I'm wrong here. Apparently, moving the Esc key to the top-left corner of keyboards was a mis-step in keyboard evolution.

4) Use word motions:

When scrolling left and right on a line, don't go a character at a time. At minimum, you should jump a word at a time using "w" (forward one word) or "b" (backward one word). Vim has the ability to jump to the next word, sentence, paragraph, etc. Learn how to get around using the word motions that make the most sense for your workflow (see :help word-motions). If you see a word way down the screen, and you want to get to it quickly, consider using a search followed by the enter key (/someword).

5) Insert smarter:

Pressing "i" to go into insert mode will get you there, but is it what you really want? Use the following insert and delete-insert commands when you need them.

I = insert at the beginning of the line
A = insert at the end of the line
o = insert on the line below current
O = insert on the line above current
C = delete all text to end of line and enter insert mode
D = delete all text to end of line

6) Join lines:

Too many times when someone wants to put a subsequent line on the current line, I've seen them go into normal mode, jump down to the next line, go into insert mode and press backspace. You NEVER need to do this. Just press "J" in normal mode.

7) Use H, M, and L:

Don't bother scrolling to the top, middle or bottom of the screen. Just use H, M, and L. I use the following mnemonics:

H = high (top of screen)
M = mid (medium of screen)
L = low (bottom of screen)

8) Use gg and G:

You never need to scroll to the top or bottom of the file. Simply hit "gg" to go to the top or "G" to go to the bottom. A number followed by "G" takes you to that line number.

9) Avoid the mouse:

I see a lot of people copy and paste with the mouse. While this works, it forces you to leave the home row. Anything that can be done in Vim with the mouse is faster with the keyboard. If you want to copy and paste, just yank (from visual mode if necessary) and put. The same goes for navigating text on the screen. Find the normal mode navigation sequences that work for you.

10) Find what works:

The list of commands available in Vim is so expansive that it's questionable if anybody could really "learn" them all. The best bet for using Vim effectively is finding the subset of commands that work for you and making them "automatic". Incorporate them into your daily programming activities and if something seems inefficient, find a better way. The help system in Vim is amazingly in-depth and provides a good avenue for finding answers quickly (see :help to get started).

Visual Block Change

In Vim, you can replace a selected portion of text across multiple lines with a new piece of text using a Visual-block change. As an example, highlight a portion of text across multiple lines using ctrl-v (blockwise visual mode), now press the "c" key to delete the selected text and enter insert mode. Type in some new text and press the Esc key. The text should be inserted for each row of the selection.

SSHFS via Fuse

This is relatively new to me and pretty damn cool. FUSE (Filesysten in Userspace) is a software package which allows you to mount various types of filesystems from inside userspace. It also includes an API for integrating such functionality into your own software and all sorts of other goodies, but that's beyond the scope of this post. I'm constantly hopping between the Grooveshark dev server and my local machine. Thinking it would be really convenient to have the remote filesystem available from inside my home directory, I decided to give Fuse a try. On my local Ubuntu workstation the sequence went something like this:

desktop:~$ apt-get install fuse-utils sshfs
desktop:~# usermod -a -G fuse travis # add my user to the fuse group
... logout ... log back in ...
desktop:~$ mkdir devmnt
desktop:~$ sshfs -p someport -oIdentityFile=/path/to/id/file user@host:/ devmnt
desktop:~$ cd devmnt
desktop:~/devmnt$ ls
... files !...

If you don't use an identity file or alternative port to ssh, you can omit those options from the sshfs command-line. Also, the filesystem is mounted with the same privileges as if you were logged onto the box as the user specified... so you can read and write files at will so long as the filesystem permissions allow.

Oh, one important little piece of information that was surprisingly absent from the sshfs manpage, you can unmount the filesystem as follows:

fusermount -u mountpoint