Monday, June 30, 2008

Command-Line Vim

Before anybody says anything, I know this can often be better accomplished using sed, perl, <your favorite tool>. That said, you can use any command in the ex toolbox directly from the command-line without actually opening Vim. Any argument preceeding the filename to be edited that begins with a plus symbol (+) is taken as an ex-command. Alternatively, you can use the -c command line flag to indicate an ex-command is being specified. This has two handy uses. For one, you can pre-process a file with an ex-command before it's loaded into Vim. Secondly, you can use Vim in a script non-interactively. For example:

travis@travis-ubuntu:/home/travis% echo "hello world" > fun.txt
travis@travis-ubuntu:/home/travis% vim "+s/hello/goodbye cruel/" "+wq" fun.txt
travis@travis-ubuntu:/home/travis% cat fun.txt
goodbye cruel world

You could get an identical result with:

vim -c "s/hello/goodbye cruel/" -c "wq" fun.txt

As an aside, this is the 100th post to Daily Vim. I want to thank everyone who's contributed to the blog so far and made this a great learning experience. You guys have made this a really fun project, so keep the comments coming!

Tuesday, June 24, 2008

MySQL Pager = Vim

When using MySQL, you can set any pager you wish using the pager command.

mysq> pager less
PAGER set to 'less'

Using Vim as your pager allows you to quickly munge query output into whatever format you want.

mysql> pager vim -
PAGER set to 'vim -'

Thanks to Jay for the tip!

Thursday, June 19, 2008

Grep For a Column

Sometimes when using MySQL, you need to know every table that contains a given column name. In the past I've accomplished this with a short script, but I've come to find out that you can simply use a SQL statement instead.

SELECT table_name FROM information_schema.COLUMNS WHERE table_schema='my_db' AND column_name='MyColumn';

Apache MaxClients

This isn't an exact science, but assuming you're using pre-fork and not MPM inside Apache 2, you should be able to estimate your MaxClients setting using the following strategy.

1) calculate average RSS usage for apache

Do a ps -ylC httpd and average the RSS column using a spreadsheet or command-line trickery. This gives you the average memory size of your Apache children.

2) apply the following formula

(server RAM * MEM percent dedicated) / average RSS usage

Given a server with 10 Gigs of RAM with 80% dedicated to Apache and an average RSS size of 20 megs, and using Google as a calculator:

(10 gigabytes * .80) / (20 megabytes) = 409.6 connections

From there, you could set MaxClients to roughly 400 connections and monitor resource usage. As always, peak load performance must be taken into account as well. Like I said, it's not an exact science; rather, a simple strategy for establishing a baseline value.

Wednesday, June 18, 2008

Quick Redraw

You can use z. in Vim to quickly redraw the current cursor line at the center of the window. zt redraws at the top and zb redraws at the bottom. Thanks to Nate for the tip!

Tuesday, June 17, 2008

Pushd + Popd

Although pushd and popd are intended mainly for programming, they fill a nice niche when navigating directory structures. If you're current working directory is some deeply nested location like /home/travis/src/myprojects/vim-patch/includes and you need to cd into /tmp for a bit but know you'll be coming back to the original directory, you should use pushd rather than cd. pushd takes the current directory and pushes it into a stack. popd pops the top directory off the stack and does an automatic cd into it. For example:

travis@travis-ubuntu:/usr/share/vim/vim71/colors% pushd /tmp
/tmp /usr/share/vim/vim71/colors
travis@travis-ubuntu:/tmp% popd

Monday, June 16, 2008

Text Formatting Features

A friend of mine contributed the following tip. I was going to rewrite it, but he explained it very clearly, so here it is verbatim:

I've been editing a plain text latex input file and am using vim's text formatting features for the first time. First useful item is :set textwidth=80 (or your preferred column width) which auto-wraps lines as you type. This is great, unless you edit part of the file before the end and mess up the text width. I discovered through the glorious vim help system the normal command gq which auto-formats a range of text. Its default behavior is a bit weird if you use it on the whole file (like gg=G for auto-indenting code), but you can do nifty things like:

format the current paragraph: {V}gq ( { = go to previous white-space-only line V=visual line select }=go to next white-space-only line )

here's how to apply formatting to all "paragraphs" using the :global command:

:%g/^\s*$\n\s*[A-Za-z]/normal V}gq

This is really specific to LaTeX, since "paragraphs" are divided by white space lines. This pattern will match any whitespace-only line, followed by a line starting with (possibly) whitespace, then an [A-Za-z].

Thanks Chris for the great tip!

Thursday, June 12, 2008

Comment Out Line Range

This is one of my favorite ex hacks. It's pretty old school and should work all the way back to the original Vi. This assumes your comment character is # (octothorpe), but would work just as well with // comments or any other line-wise comment character. Go to column zero of the first line you want to start your comment block on. Hit ma to set a mark. Now scroll down to column zero of the last line you want to comment out. Issue the following ex command:

:'a, . s/^/# /

That's it. It runs a substitute on the range from the mark stored in register 'a to the current line. Take note that you can specify ranges for any ex command using the same technique.

Wednesday, June 11, 2008

A Few MySQL Tricks

Here are a few tricks I use quite a lot with MySQL.

1) copy an existing table's schema to a new table

This is handy for making a quick copy of an existing table to test indexes, populate data, etc...

CREATE TABLE NewTableName LIKE OldTableName;

2) copy an existing table to a new table

If you want more than just the structure, you can copy a table's raw data to a new table really easily. The downside is that keys and other constraints aren't preserved.


3) copy a table's exact structure and data

If you want to preserve everything (constraints, keys, etc...) and copy the data, it requires two steps.

CREATE TABLE NewTableName LIKE OldTableName;
INSERT INTO NewTableName SELECT * FROM OldTableName;

4) quickly load data while specifying columns

After reading the MySQL forums, I've realized a lot of people online think this is impossible. It's actually right there in the documentation; although, somewhat obscurely tucked away. MySQL DOES allow you to specify what columns to operate on when using LOAD DATA INFILE. Just use the following syntax.

LOAD DATA INFILE '/some/file/name.csv' INTO TABLE SomeTableName FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (field1, field2, field3);

The field list goes at the end. Adjust the statement according to your needs, and enjoy the 20X increase in data load times.

5) atomic rename

There are a number of situations where you want to replace an entire table worth of data with a new table instantly. This is easily accomplished with an atomic rename and avoids a brief downtime while the tables are being switched.

RENAME TABLE TableName TO TableNamePrevious, TableNameNew TO TableName;

6) monitor a long running insert

If you've ever dealt with long-running inserts, you probably know how annoying it can be to just sit and wonder when the task will complete. For inserts performed in a single transaction, a simple select count(*) won't do because they're all or nothing. Take the following example:


Fortunately, there's a way to peak behind the scenes. Do a count on the target table beforehand and make sure it has a populated primary key column. From there, you can do:


Look at the last line of the table definition, and you'll see an AUTO_INCREMENT value. That's the last increment of the primary key. If you started with 25,000 rows, and AUTO_INCREMENT is 50,000 then you know 25,000 new rows have been inserted.

7) monitor a LOAD DATA INFILE

If you're using InnoDB, you can do SHOW INNODB and look at the transactions list. From there, "undo entries" displays the number of rows inserted via the current LOAD DATA INFILE operation thus far. This is convenient because I don't believe the AUTO_INCREMENT trick works for LOAD DATA INFILE.

Tuesday, June 10, 2008

Quickly Find Subdir

I used to use a long-winded invocation of find with the prune option to non-recursively find all the directories in the current dir:

find . -name . -o -type d -print -prune

From there, I moved to using ls and grep:

ls -l | grep '^d'

I recently learned that the following shell pattern works just as well:

ls -ld */.

Monday, June 9, 2008

Source A File

If you make changes to your ~/.bashrc or ~/.bash_profile or ~/.zshrc, etc... you can load the changes without logging in and out again by issuing a source command:

source ~/.bashrc # load changes, execute commands, etc...

alternatively, you can use the shortcut dot syntax:

. ~/.bashrc

You can do the same in Vim to read a file of ex-commands such as your ~/.vimrc.

:source ~/.vimrc

Friday, June 6, 2008

Syntax Check

When working on anything important, it makes sense to check the syntax of your file before setting it live. This is usually accomplished with a simple command-line flag. Consider the following:

travis@travis-ubuntu:/home/travis% ruby -c test.rb
Syntax OK

travis@travis-ubuntu:/home/travis% perl -c syntax OK

travis@travis-ubuntu:/home/travis% php -l test.php
No syntax errors detected in test.php

Python doesn't make this as easy. I'm not sure why because it seems to have a pretty well defined separation between it's compiler and runtime, but the general consensus is to break your code into classes and just have a single "main" file for your runtime code. That way you can just run python filename, which will compile the file to bytecode as well as check syntax; however, this is frustrating if you just want to check the syntax and not actually run the program (common with DB applications). Consider the following hack an alternative, which could easily be wrapped up in a helper shell script.

python -c "import py_compile; py_compile.compile('')"

This produces no output on success and an error on failure. It has the possibly unwanted side effect of compiling to bytecode, but I don't think there's any getting around that.

Thursday, June 5, 2008

Leaning Toothpick Syndrome

One thing that's always bothered me about PHP is it's lack of a qq style operator. For example, in Perl, I could do something like this:

print qq/a $fun "string" with "double quotes"/;

In PHP it'd be something gross like:

print "a $fun \"string\" with \"double quotes\"";

Anyway, I found a nice solution that's been right under my nose all this time:

printf('a %s "string" with "double quotes"', $fun);

This is quite a bit cleaner for things like hyperlinks, so I thought I'd pass it along.

Ruby borrows quite a few ideas from Perl, and has a very nice %Q operator:

print %Q/a #{fun} "string" with "double quotes"/

going even further, you can use %r for regexp:

mystring =~ %r{/usr/src/linux} # no leaning toothpick required

or in Perl:

$mystring =~ m#/usr/src/linux#

Wednesday, June 4, 2008

Google Shell

I haven't played around with this very extensively, but I'm a fan of just about any sort of shell, so I thought this was sort of fun.

A non-google sponsored Google Shell.

Tuesday, June 3, 2008

Star Power

This is a very simple tip, but some people may not know it. If your cursor is positioned on a given word in Vim and you press the `*' key, it will jump to the next word of the same name.

Monday, June 2, 2008

Hashes Under The Hood

If you've ever worked with hashes before and wondered how they work behind the scenes, I've written a very straightforward Ruby implementation of a hashing class. The "lookup" method in particular is pretty easy to follow and should give a decent understanding of how hash retrieval works in general.

Simple Hashing Implementation

Edit: Looks like Ruby Garden is temporarily down. Hopefully it will be back up soon.