Friday, December 12, 2008

Value Under Cursor #2

Expanding on a previous tip. Here are some methods for figuring out the value of a given character.

1) Using gf - place the character under a character and hit ga in normal mode to obtain it's ascii value.

2) Using g8 - same as above but applies to UTF-8 characters.

3) :set list - toggles displaying unprintable characters.

4) pipe into the shell command od -c

5) type the :ascii ex command to display a characters ascii value

6) use cat -vET filename

Setting the Filetype

Vim normally sets the current filetype automatically and very accurately at that. In some cases though, such as when you're starting a file from scratch, it doesn't know the filetype. In these cases, you can set the filetype manually.

:set filetype=perl
:set filetype=php
:set filetype=python

etc...

Thursday, December 11, 2008

Ubuntu: Downgrading X.org

First things first, if you use multiple monitors and you're thinking about upgrading to Ubuntu 8.10 (Intrepid), I would recommend holding off. A good number of users, myself included, have experienced a very annoying bug where the mouse spontaneously loses it's ability to click. The only way to regain mouse support is to kill X with ctrl-alt-backspace (ugh). I've been able to faithfully reproduce the bug by doing the following:

* Enable Xinerama in xorg.conf
* Open firefox fullscreen on both monitors
* Rapidly move the mouse between monitors

It's worth noting that this bug has nothing to do with what window manager you're using. I've experienced it under Gnome, AwesomeWM, and Xmonad. The bad news is that even though a lot of people are complaining about it, the bug has not been fixed. As of right now, the only reliable method to regain workstation stability seems to be downgrading X.org back to the Hardy packages. To assist anyone who might be Google'ing this issue, here are the instructions for doing so.

Log into a text console as root and shut down X with '/etc/init.d/gdm stop'.

Add the following to your /etc/apt/sources.list:

deb http://us.archive.ubuntu.com/ubuntu hardy main
deb http://us.archive.ubuntu.com/ubuntu hardy-updates main
deb http://us.archive.ubuntu.com/ubuntu hardy-security main

Run apt-get update.

Purge all the packages you can by doing the following:

cd /var/lib/dpkg/info/
dpkg --purge `ls *xorg*.list | sed s/.list// `

Some of these will fail to purge due to dependency issues. That's OK. Continue with the instructions.

Reinstall the packages from the Hardy repo:

apt-get --reinstall install x11-common/hardy xorg/hardy xserver-xorg/hardy xserver-xorg-core/hardy xserver-xorg-input-kbd/hardy xserver-xorg-input-mouse/hardy

Note that if you're using a stock Ubuntu video driver such as the Matrox driver you will need to include it in the list as well (xserver-xorg-video-mga/hardy). I use the third-party Nvidia driver, so it wasn't necessary to include it in the list.

After this, you can start X with "/etc/init.d/gdm start" and hopefully login to a working desktop. The final step is to go into synaptic and lock the reinstalled packages to avoid the upgrade manager from nagging.

Thanks to Nelson for providing these instructions, which I've modified slightly for presentation here.

Friday, December 5, 2008

PHP: Invoking a method with Map

Ever since I learned the basics of functional programming, I've been a fan of the map function. In the right context, it can be an elegant solution for array transformation. Of course, like anything, it can be misused. Using PHP's map, it's non-obvious how to make it invoke an instance method rather than a function. The following example illustrates how this can be accomplished.

class M
{
function double($a) {
return $a * 2;
}
}

$m = new M;
$list = array(1, 2, 3);
$list = array_map(array($m, 'double'), $list);
print_r($list);

Array
(
[0] => 2
[1] => 4
[2] => 6
)

Tuesday, December 2, 2008

Find Dot Files

Sometimes I forget the name of a particular config file but know it's stored as a dot file in my home directory. The following one-liner will non-recursively display any regular file begining with a dot.

find . -maxdepth 1 -type f -name '.*'

Find does it again. ;-)

Right = Left

Here's a tip that's mostly for fun. Open up Vim and type :set rightleft.

This changes display orientation to right-to-left. This can be useful for editing Hebrew and Arabic or just for freaking people out.

Wednesday, November 26, 2008

AwesomeWM

I've used a lot of window managers and desktop environments over the years (Gnome, KDE, Window Maker, Blackbox, Fluxbox, Enlightenment, XFCE, etc...). The last few years, I've lost a lot of motivation to experiment and stuck with the OS default (Gnome/Metacity in my case). Recently though, I've been increasingly trying to avoid ever touching the mouse. This is partially due to wrist issues and partially just to try and be more efficient. The quest for a mouse free lifestyle lead me to find the aptly named Awesome Window Manager.

If you're like me and have used many window manager, you're probably groaning right now and saying, "what makes Awesome any different from my current window manager?" Well, quite a few things actually. For one, Awesome is a tiling window manager. This means that if you open three terminal windows on the same screen, the WM will neatly organize them on the screen in an intelligent fashion. If you don't like the tile layout, you can switch layouts with a keystroke. If you want a certain application to fall outside of the default tiled layout, you can set it to float, and it will behave like it would in any other window manager.

While tiling is nice, it's not the only thing that makes Awesome unique. Awesome also avoids the traditional concept of virtual desktops or workspaces in favor of a tagging mechanism. Tags allow you to assign applications to logical groups in which they will appear together. An application can be bound to any number of tags. Even better, tags allow you to assign rules to newly opened applications, so I can say, "Firefox will open on the `Firefox' tab in floating mode on my 2nd monitor". You can also assign a default layout and number of columns to a given tab for tiling. Last but not least, each monitor can have it's own tagset giving you the equivalent of an independent session per monitor.

This is all good and well, but what does it have to do with getting around with the keyboard? Well, quite a lot actually. Awesome allows you to setup keybindings to navigate between tabs, send applications to a given tab, switch between applications on a given tab, resize windows, rotate windows within the tileset, jump between monitors, etc. All this is done inside of your awesomerc file, and the syntax is very straightforward. In my custom configuration, I've setup Awesome to respect keybindings very similar to Vim, so I can jump between monitors and windows using a combination of my meta key (windows key), a modifier (alt), and H, J, K, and L. As a bonus, it also happens to be extremely fast with performance in the same class as fluxbox and it's cousins.

A final feature that I've come to absolutely love is it's widget support. There's a user configurable statusbar that by default contains your current tagset listing and a small layout diagram. The rest of the statusbar is essentially open for whatever you want to add. All an application has to do to update the statusbar is write to standard output, so I've constructed a series of shell scripts to show the current time, CPU usage, memory usage, active users, and active streams at Grooveshark. All of these are basically one-liners. Here's my clock for the sake of example:


#!/bin/sh
while true
do
echo "0 widget_tell mystatusbar clock text " \
"`date +"%A %B %d, %Y %l:%M %p"`" | awesome-client
sleep 1
done


In case I missed anything, here's the laundry list of features from the Awesome website.

* Very stable, fast, small and simple;

* Only window manager using asynchronous XCB library instead of the old synchronous Xlib: make awesome faster than any other window manager;

* Very well documented source code and API;

* No mouse needed: everything can be performed with keyboard;

* Real multihead support (XRandR, Xinerama or Zaphod mode);

* Implement many Freedesktop standards: EWMH, XDG Base Directory, XEmbed, System Tray;

* Some real transparency support (using Composite extension and xcompmgr);

* Doesn't distinguish between layers: there is no floating or tiled layer;

* Whether or not the clients of currently selected tag(s) are in tiled layout, you can rearrange them on the fly. Popup and fixed-size windows are automatically floating.

* Layout handling: automatically manage your windows placement according to the chosen policy for each tag;

* Use tags instead of workspaces: allow to place clients on several tags, and display several tags at the same time;

* A lot of Lua extensions to add features: dynamic tagging, widget feeding, tabs, …;

* D-Bus support;

* And more.

And of course a screenshot is worth a thousand words.

awesome screenshot

And Finally, The AwesomeWM Website.

[edit]
As requested, here's a copy of my awesomerc. Note that I'm using version 2.3.4.

Tuesday, November 25, 2008

Reload VIMRC

You can quickly and easily reload your vimrc with the following command.

:source $MYVIMRC

If you want to know what your current vimrc is, just do the following.

:echo $MYVIMRC

Tabs

It took tabs a while to grow on me when using Vim. This is mainly due to the flexibility that buffers and split windows provide; however, once I realized tabs could act as a convenient way of grouping split windows, I was sold. There's nothing complicated or confusing about using tabs, but a few mappings will make your life easier. Add the following to your vimrc.

let mapleader = ","
map <leader>tt :tabnew<cr>
map <leader>tc :tabclose<cr>
map <leader>tm :tabmove
map <leader>tn :tabnext<cr>
map <leader>tp :tabprevious<cr>

I've already been called a n00b once on this blog for not seeing the value of the comma motion (repeat latest f, t, F or T in opposite direction). If you use it regularly, you'll obviously want to choose a different map leader.

Now that your mappings are setup, using tabs should be quick and intuitive.

* Press ,tt to open a new tab
* Press ,tc to close the current tab
* Press ,tm [number] to move to tab [number]
* Press ,tn to move to the next tab
* Press ,tp to move to the previous tab

In standard Vim, the tab list will appear at the top of your editing window. Gvim provides the industry standard GUI tabs. Obviously there are more tab commands available, but these basics should get you pretty far.

Friday, November 21, 2008

Using a Mapleader

As you have probably realized, keyboard real estate in Vim is pretty heavily occupied in normal mode. Specifically, just about every key is already bound to a command. Luckily, Vim provides a facility to define additional mappings on these keys. It's called a mapleader, and it's easy to setup.

First you need to define what key you want your mapleader to be. I prefer comma because it's (in my daily usage) unused, and it's easy to reach. Add the following to your vimrc.

let mapleader = ","

Now that mapleader is defined, you can use it for custom mappings at will.

:map <leader>n :new<cr>

Press `,' then `n' in normal mode, and it splits to a new window.

:map <leader>i I

Press `,' then `i' in normal mode, and it puts you into insert mode on the first character of the current line.

These usage examples are slightly obtuse, so I'll be showing how to incorporate using a mapleader into tab management in an upcoming post.

Thursday, November 20, 2008

Share a Screen Session

It's possible for two users to share a screen session. This is particularly useful if multiple people want to monitor a long running task, and it's easy to do.

user #1:

screen -R longbuild

user #2:

screen -x -R longbuild

The -x flag tells screen to allow you to attach to an already attached session. This is also good for crazy stuff like a collaborative Vim session (if you're into that sort of thing).

Monday, October 20, 2008

Vim to Postscript

Thanks to Karl for the following tip:

Here's a little Vim trick that might be of interest. If you're on a non-Windows machine, you can make the Vim hardcopy command print to a postscript file instead of a real printer. Try issue the following:

:hardcopy > file.ps

You'll get a postscript output of the file you're currently editing.
From there you can view the postscript file with gv or convert it to a pdf with ps2pdf.

Thursday, October 16, 2008

MySQL DBD in Activestate

I'm currently doing some consulting work to bridge the gap between Oracle and MySQL. Being in a Windows environment, I'm using the Activestate Perl distribution. Fortunately, it includes a nice ODBC module for connecting to Oracle out of the box. The bad news is, it doesn't contain a MySQL module. Luckily, I found a third party host that does provide the module. The following ppm (Perl Package Manager) one-liner takes care of the issue.

C:\>ppm install http://cpan.uwinnipeg.ca/PPMPackages/10xx/DBD-mysql.ppd

The Golden Rules of Recursion

1) never iterate
2) always follow the first rule ;-)

Where I've Been

I've come to realize that my post per day ratio here at Daily Vim is a good measure of how busy I am in my daily life. To be more specific, my workload is inversely proportional to my blogging habits, but please don't think I've forgotten about this blog. I haven't, and the posting will continue. Lately though besides being buried under a mountain of side-work, I've been working diligently alongside the rest of the Grooveshark team to facilitate one of our largest launches in recent memory.

Today, all that hard work came to fruition, and I'm happy to announce that our streaming service, Grooveshark Lite, has been expanded with the addition of the following features.


  • Shared Playlists

  • Blog Widget

  • Improved Social Features (follow users, find similar users, etc...)

  • Music Upload Client

  • Autoplay Improvements

  • More Reliable Streaming


If you scroll down this page, you can see an example of the blog widget in action. The widget builder page allows you to embed a playlist or song of your choosing into any website, size it appropriately, and drop it in. I'll be updating my blog playlist frequently as a means to passively share music that I enjoy.

Back on the topic of Daily Vim, now that things are settling down a bit, I'm hoping to post more often, as I have some cool tips I'm looking forward to sharing.

Thursday, October 9, 2008

Why Ruby is My Favorite Language

I started programming seriously roughly ten years ago. I was working in the call center of an ISP, and there was a need for a reseller tracking application. I was attending the local community college at the time, and had been indoctrinated into the world of computer programming trhough what amounted to vocational training in Visual Basic and Microsoft Access. Fortunately, the ISP was a Linux shop, and a few employees were nice enough to introduce me to the basic philosophy of Linux and assist in learning some rudimentary shell commands.

One of these employees was a Perl hacker, and he assisted in convincing the company to form a small team of three programmers to tackle the programming job. I was lucky enough to be chosen for the position, and to be frank, after having spent a few months in the call center, I would have considered janitorial duties a promotion. The only problem was that I didn't know Perl. I quickly ran to the nearest bookstore, grabbed a copy of Learning Perl, and set to work learning the language. A week later, I was neck deep in the project and thankfully had an experienced programmer to lead me through my first big job. The project was a success, and the team became a permanent fixture at the company.

What impressed me so much about Perl was the succintness, mneumonic friendliness, and consistency of the language. After a few weeks of intense study, I could perform the majority of my daily programming tasks from memory and rarely needed to consult a reference.

A year or so later, PHP hit the scene. With the huge buzz surrounding the language, I figured there had to be something really groundbreaking going on. The team I was working on chose to build a large project in PHP, and once again, I set out in earnest to learn the language. This time, things weren't so easy. While Perl had "sunk in" right away, PHP lacked the natural abstraction I had come to love about Perl. As an example, I didn't understand why PHP needed sixteen sort methods (sort, arsort, asort, krsort, ksort, natsort, natcasesort, rsort, usort, array_multisort, uasort, uksort, dbx_sort, imap_sort, ldap_sort, yaz_sort) while Perl only needed one. The database methods lacked the wonderful platform independence of DBI, and a lot of other things felt redundant. Mostly though, I disliked the way the language encouraged mixing interface with your implementation. At the time, the PHP website frequently featured examples interpsersing functions and HTML and even touted this ability as a "language feature".

Progressing through my courses, I ended up taking a class on object oriented design. I learned about the model view controller architectural pattern and immediately saw it's advantages as a design strategy. I learned the basics of object oriented programming with Perl, but I was dissatisfied with the way it was implemented. I spent a considerable amount of effort learning Java and later Python. I found both languages to be "good enough" in the way they possessed a healthy amount of easily accessible OOP goodness, but I missed the fluidity of programming in Perl and neither ever became a "favorite".

At this point I was working at an environmental laboratory, and I was given complete freedom to use whatever technology I wanted to solve the problems at hand. I was tasked with writing a data analysis and computation software for an ICAP (Inductively Coupled Argon Plasma Spectrometer), and I was again at a crossroads. An article on Slashdot had drawn my attention to the Ruby programming language, and I found an excellent book that was freely available online. Reading the introductory walkthrough section, I was immediately impressed by the fact that literally everything in the language was an object. The core language was incredibly consistent, concise, and well documented with absolutely no cruft. All of the wonderful Perl idioms I'd come to know and love were right there at my fingertips; albiet, in a more readable fashion. I was sold.

I learned the language and developed the project simultaneously. This was long before Rails was even a whisper in the programming world, so I rolled my own MVC style framework containing a presentation layer that used a combination of XML, eruby templates, and LaTeX. This made it easy for the app to generate both HTML and PDF documents on a whim. The development cycle was fast, the project was finished ahead of time, and everybody loved the final product. Later I wrote a C extension for Ruby to easily access the Win32 serial port allowing an archaic lab instrument to dump data straight to an OpenOffice spreadsheet. I'd never considered writing a C extension to any other language, but again, Ruby made it easy.

Since then, Ruby on Rails has rolled onto the scene and has allowed me to develop projects in which I would have needed a team in the past. MVC is the core paradigm and being able to use my favorite language alongside my favorite architectural pattern is nothing short of a godsend as a web developer. I've used Rails to develop web applications on the fast track and have brought in a nice supplementary income doing so. Best of all, I feel good about the final product when writing a Rails app knowing I'm developing in a standardized way where another programmer will be able to come along later and easily maintain the work I've done. There are MVC frameworks for PHP, but there's still no de facto standard way to roll a PHP app. This makes every PHP app a "special surprise" when you look under the hood. Sadly, the surprise isn't always pleasant.

I'm not saying you can't write good code in PHP. Actually, working at Grooveshark has shown me that it's possible to write extremely clean code in PHP, but I still run to the PHP manual several times a day to lookup one of it's 3000+ core method signatures.

I'm acutely aware that languages are like religions among developers and that everyone has their favorite for one reason or another. I'm also certain there are people out there that would disagree with some of the arguments I've made here. Arguments aside, if you're looking for a new language to learn, and you haven't tried Ruby, take a look, and I think you'll be pleasantly surprised at all that it has to offer.

Wednesday, October 1, 2008

Vimperator

There's a plugin for firefox which allows firefox to behave more like Vim. Some of the features include:

* Vim-like keybindings (h,j,k,l,gg,G,0,$,ZZ,,etc.)
* Ex commands (:quit, :open www.foo.com, ...)
* Hit-a-hint like navigation of links (start with 'f' to follow a link)
* Vim-like statusline with a wget-like progress bar
* Macros to replay key strokes

etc...

The author really threw in the kitchen sink. I've seen my co-worker using this, and it's SUPER cool.

Vimperator Plugin

Tuesday, September 30, 2008

SSH Remote Connections via Config

Thorsten has contributed the following ssh tip. Thanks Thorsten!

You wrote about ssh connects via gnome terminal:

http://dailyvim.blogspot.com/2008/09/gnome-terminal-tips.html

I'm using the config file of ssh to achieve the same goal:

just put the file "config" in your .ssh directory (chmod 600) with the
following lines:


host mysshhost
user root
identityfile /root/.ssh/my_ssh_key.ssh
port 34021
hostname 127.0.0.1
localforward 10100 localhost:10100
ForwardX11 yes


then back at the command line just try:

myhost:/ # ssh mysshhost

and voila, welcome to your ssh connected system (even the
bash-completion works with the config file, just do a ssh )

Monday, September 29, 2008

Question: Browsing Remote Filesystems

A Daily Vim reader asks the following question:

Travis, the feature that has me married to Kate (KDE text editor) is the file browser with bookmarks built in. Kate allows users to browse remote filesystems from within the app just like they could browse the local filesystem in a file manager. Furthermore, there is a bookmarks feature where users can bookmark specific directories / files on remote servers. Anythink like those two features in VIM?

Thanks!

Anybody have any ideas on this? I have a few ideas, but I'll wait and see if anybody else has a suggestions first.

Thursday, September 25, 2008

Whatprovides

If you're using a RedHat derivative that utilizes yum as it's package manager, you can use the `whatprovides' command to see what package provides a specific file.

Example:

yum whatprovides /usr/bin/vim

... grab a coffee ...

vim-enhanced.x86_64 1:6.3.046-0.40E.7.cent base
Matched from:
/usr/bin/vim

Debian derivatives provide a similar functionality via the apt-file package.

# sudo apt-get install apt-file
# sudo apt-file update
$ apt-file search /usr/bin/diff
diff: /usr/bin/diff

Friday, September 19, 2008

Unhighlight Current Search

If you want to un-highlight the currently highlighted search terms, issue the following command.

:noh

Friday, September 12, 2008

Search Register

You can manually push a value into the search register using the following syntax.

let @/='Some String'

This is particularly nice for script hacking when you want to highlight a group of strings in the current buffer.

Vimgrep Without Object Names

My friend Chris (Vim user extraordinaire) has contributed the following tip. It looks like it could be quite useful to any Java hackers in the crowd.

I constantly want to use :vimgrep to find usages/references to certain object types in my java code. However, when I :vimgrep for the object name, it (naturally) matches all of the 'import' statements, too. I've known for awhile that there was an idiom in the regex syntax to negate a previous atom (in this case, the 'import' at the beginning of the line) but I've never found a way to phrase it that works. Here it is:

.*ObjectName\&^\(import\)\@!

This matches any line meeting the following two constraints:
1) contains anything (.*) followed by ObjectName
2) starts (^) with something which isn't 'import' (\(import\)\@!)

the \@! says, match if the preceding atom does not match here -- since we put ^\(import\) this means match when the first thing in the line isn't import.

There may be better ways to do this, but it's the first I've found and I'm using it like crazy now... Enjoy!

Thanks Chris!

Question: Highlight From Insert?

Iyo writes:

"I offten do some coding in Vim and I miss an feature that will highlight
the word I`m writing in that moment and other same words. (It is mainly
for variables - sometimes I switched letters and afterwards I`m looking
for mistake for a quite long time. The NetBeans IDE has a feature like this.

I know about * but it`s for search in the text. I`m looking for something that work automaticly in insert mode."



Answer:

You can use ctrl-o from inside of insert mode to enter a command, but to me typing ctrl-o * while I'm hammering out some code feels awfully cumbersome. There's also the negative side-effect that * jumps forward, so you'd lose cursor position, which would assuredly interrupt workflow. Instead, I'd just define a simple mapping along the lines of the following.

imap <F2> <Esc>#*A

So, you're inside insert mode, you just typed a word, and you want to match all occurrences. You would simply hit F2, and it would backward match the word nearest the cursor. This has the negative side effect of jumping backwards; thereby, losing cursor position. To compensate, we add a * to jump forward followed by "A" to return to insert mode and insure you're at the end of the line. Obviously, you can substitute any keybinding you want for F2.

This seems to work well on my machine; although, there might be more optimal solutions. If anybody has one, feel free to share.

Thursday, September 11, 2008

Got a Question?

I had a reader of Daily Vim contact me yesterday about the possibility of periodically submitting questions regarding Vim best practices, etc. I told him that I'm very happy to assist in answering questions any Daily Vim readers might have but if I don't know the answer, I'll post the question here anyway, so someone smarter than me can ;-).

So, if you have a question regarding Vim, Linux, scripting or something along those lines, feel free to post them under this thread or email me directly via tinymountain at gmail dot com. I'll post the question and possibly an answer to the blog, and as usual, feedback is welcome.

To kick things off, I have a question of my own. Does anybody know of a Vim plugin or some method to spontaneously highlight every other line of a file? I frequently use Vim as a pager inside of MySQL, and this would be very handy for eyeballing all the values on a specific row.

UPDATE:

I solved this problem thanks in part to an initial suggestion to use syntax highlighting to perform the highlighting. Here's my finished solution. It seems to have issues with language files with pre-existing syntax highlighting, but for it's intended purpose (use within the mysql pager), it works beautifully. Just do "pager /usr/bin/vim -R -" inside the mysql client, add the code to your vimrc, run a query and hit F2.


hi AlternateLine ctermfg=grey ctermbg=darkblue

function! HiLine(lineno)
let tmpline = escape(getline(a:lineno), '/\[]')
exec 'syn match AlternateLine /.*' . tmpline . '.*/'
endfunction

function! DoHighlight()
global /^/if line('.')%2|call HiLine(line('.'))
normal 1G
endfunction

map <F2> :call DoHighlight()<cr>


Jump to Percent of File

Sometimes you don't know the exact line you want to go to in a file, but you can infer an approximate percentage. I was browsing a 200 meg logfile with Vim yesterday and guestimated the area of interest was about 10% back from where I was at (85%). You can easily jump to a desired percentage in a file by typing in the percentage followed by a percent symbol.

Jump to 50% of the file:

50%

Easy to remember to say the least ;-)

Wednesday, September 10, 2008

Scrollbind

You can :set scrollbind in any windows in Vim that you would like to scroll together.

Thanks to Chris for the tip.

Monday, September 8, 2008

Determining Linux Version

Linux distributions don't tend to standardize where they store information about what release is installed. Going further, there's not even a standard way to detect what distribution is installed. If a system is running a popular distro, this information can usually be found in one of the following files:


Debian /etc/debian_release, /etc/debian_version,
Fedora /etc/fedora-release
Gentoo /etc/gentoo-release
Mandrake /etc/mandrake-release
Novell SUSE /etc/SUSE-release
Red Hat /etc/redhat-release, /etc/redhat_version
Slackware /etc/slackware-release, /etc/slackware-version
Solaris/Sparc /etc/release
Sun JDS /etc/sun-release
Ubuntu /etc/lsb-release
UnitedLinux /etc/UnitedLinux-release
Yellow dog /etc/yellowdog-release


You can find more information on this problem as well as a shell script to assist with OS detection here. An even larger master list can be found here.

Ubuntu Audio Sharing

For a good while, audio sharing issues plauged my Ubuntu workstation. Whenever I'd hit a website that used Flash 9, none of my other audio applications would be able to access the audio device until my Gnome session was restarted. Googl'ing around for Ubuntu and Flash, I stumbled onto this package, which I'm happy to say solved the issue.

Ubuntu libflashsupport.

Obviously, you can install it quite easily with apt.

sudo apt-get install libflashsupport

Hope this helps somebody.

Friday, September 5, 2008

Repeat Last Command Line

To repeat the last command-line given in ex mode, you can simply enter a count followed by "@:".

Swap Characters

To swap two adjacent characters in Vim you can issue an "xp" command in normal mode.

Gnome Terminal Tips

I often use the profiles feature in gnome terminal to organize all my remote shell accounts. Inside of gnome terminal, you can go to Edit->Profiles to manage your profiles. From there, you can either edit an existing profile or create a new one. Either action will take you to the "Editing Profile" window. Once inside, click on "Title and Command" tab and check the box that says "Run a custom command instead of my shell". You can put your ssh login info inside of the "Custom command" box. An example would be:

ssh -i /home/username/.ssh/id_rsa.1 -p 7822 myuser@myhost.com

That would invoke an ssh session using port 7822 and the identity file stored in your .ssh directory. It's particularly useful to setup sessions with shared keys so you can just go File->Open Terminal and point to the desired session. If you have certain ssh session you use a lot, it might be worth adding a launcher to your panel and specifying the desired profile like so:

gnome-terminal --window-with-profile SomeProfile

It's also worth noting that gnome terminal allows you to color code your saved profiles. I find that this helps prevent stupid mistakes caused by thinking your logged into a different session than you actually are.

Since I use gnome terminal to store many remote login sessions, I sometimes need to retrieve this information whilst remotely logged into my workstation. An example would be when I ssh to my work PC from home over a VPN connection. Gnome terminal stores all of it's saved profile information in the following location.

/home/$username/.gconf/apps/gnome-terminal/profiles/$profilename/%gconf.xml

In this case, $username and $profilename would represent your username and desired gnome profile name respectively. Inside of the %gconf.xml file, there's a section for custom command that looks like this.

<entry name="custom_command" mtime="1193759925" type="string">
    <stringvalue>ssh -p 7822 username@somehost.org</stringvalue>
</entry>

From there, you should be able to extract whatever info is desired.

Bonus tip: you can open a new terminal window whilst inside an active gnome terminal session by doing ctrl-shift-N.

Friday, August 29, 2008

Scrolling Horizontally

If you would like to scroll horizontally with the cursor following the screen, you can use the following commands to scroll left and right.

zl - move the view [count] characters right
zh - move the view [count] characters left

Count Words

If you would like to count the number of words in the current Vim buffer, you can simply issue the following command: g CTRL-G (no space between g and CTRL-G).

Thursday, August 28, 2008

Nautilus MP3 Column Display

There have been a few instances when I've felt it would be useful to have Nautilus, Gnome's file browser, display MP3 metadata info. It's particularly nice when organizing large volumes of files by directory. KDE has had this functionality for a while, but it's been suspiciously absent from Gnome. I found a quick hack, which allows you to add metadata columns at will.

Nautilus MP3 Column Display

Daily Vim Song of the Week

I can't get enough of this song.

The Reigning Sound - Find Me Now

If anybody else has a song of the week, post it here ;-).

Uppercase, Lowercse, Swapcase

In Vim, you can use the following motions to modify the case of an entity.


gum - lowercase text operated on by motion
gUm - uppercase text operated on by motion
g~m - switch case operated on by motion
~ - if notildeop is set (default), switch case under cursor
~ - if tildeop is set, switch case of {motion} text


Examples:


guw - lowercase current word
gU$ - uppercase to end of line
g~w - swapcase current word
v$~ - swapcase to end of line (if notildeop is set)
~w - swapcase of word (if tiildeop is set)

mk-table-checksum and Columns

UPDATE: I submitted a patch to the Maatkit developers, and the column bug mentioned below if fixed in SVN.

At work, we've been struggling with MySQL replication losing synchronization for the last few months. It's usually something like a few timestamps being off here and there, but we've also seen primary keys with artificially high values on the slave, certain replication events just not happening, and other weirdness. One toolset that has been very helpful in figuring out when problems occur is Maatkit. It a suite of tools written in Perl, which allow you to do things like run checksums on the master and slave for a given table, compare, and see if they're out of sync.

Unfortunately, some of the documentation is very sparse, which leads me to this blog entry. Google had nothing useful regarding how to use the columns feature, which allows the checksum to be calculated only on specific columns. I post this here in hopes it will help somebody.

We had been trying to use mk-table-checksum in the following way:

mk-table-checksum 127.0.0.1 -usomeuser -p'password' -tmydb.TableOne --replicate=maint.checksum --columns=UserID,TSAdded

Unforunately, running it this way would give a warning:

Use of uninitialized value in concatenation (.) or string at /usr/bin/mk-table-checksum line 431.

and produce no results. I finally got annoyed enough that I fired up the Perl debugger and stepped through what was going on. As it turns out, the data stored internally in $table->{cols} is all lowercase. If you pass in your column list camel-capped, or uppercase, it simply won't work.

The following invocation would produce useful results:

mk-table-checksum 127.0.0.1 -usomeuser -p'password' -tmydb.TableOne --replicate=maint.checksum --columns=userid,tsadded

Stupidly simple, but not necessarily obvious.

Wednesday, August 27, 2008

Bash, Re-writing a Line

There are instances when writing a shell script that you want to write to the same line over and over. This is mainly used for progress meters that update the same screen position over and over. Most languages represent a carriage return without newline as the \r escape sequence. In shell, it's no different, but there is one small caveat. The echo command requires two arguments.


#!/bin/bash

# simplest progress meter
items=100000
for ((i=0; i<$items; i+=1)); do
echo -n -e "Processed $i/$items\r"
done


The -n argument to echo tells it to suppress the implicit newline that it usually includes. The -e argument tells it to respect escape sequences such as \r.

Tuesday, August 26, 2008

Perl Broken on CentOS 5.2 and Fedora 9

I'm not really a fan of re-blogging stuff, but I found this so staggeringly stupefying that I felt the need to post it here.

Redhat Perl, What A Tragedy

We run a variety of Perl scripts on our production systems here. A quick test using the script from the article shows that a basic loop calling bless 50,000 times takes 0.16 seconds on my Ubuntu workstation. Meanwhile, our uber-powerful production server, which runs a Redhat derivative, takes over four seconds to run the script. Our development server, running the same OS on slightly less powerful hardware took 24 seconds.

To summarize, if you run CentOS or Fedora, and you use Perl in production, you may have a problem.

Monday, August 25, 2008

Sharing a Session

You can use script and mkfifo to allow a second session to supervise the first using this very simple hack.

# on 1st terminal
mkfifo /tmp/session
script -f /tmp/session

# on 2nd terminal
cat /tmp/session

Now the 2nd terminal can interactively watch whatever is going on with the 1st terminal. Occasionally some control characters will get a little garbled when a lot of curses activity is going on, but in general, it works pretty well.

Thursday, August 21, 2008

Installing Ruby DBI, MySQL on Centos 5.2

There's no good documentation on how to do this online, so here's a quick tutorial on how to get Ruby's DBI and MySQL packages installed on CentOS 5.2. There aren't any pre-packaged binaries for this, so building from scratch is pretty much your only option.

# download
wget http://tmtm.org/downloads/mysql/ruby/mysql-ruby-2.7.tar.gz

# extract
tar zxvf mysql-ruby-2.7.tar.gz

# change dir
cd mysql-ruby-2.7

# configure (requires mysql development package AND gcc)
ruby extconf.rb --with-mysql-config
checking for mysql_ssl_set()... yes
checking for mysql.h... yes
creating Makefile

# finish (as root)
make install
(ignore warnings)

Now DBI:

# download
wget http://rubyforge.org/frs/download.php/41303/dbi-0.2.2.tar.gz

# extract
tar zxvf dbi-0.2.2.tar.gz

# change dir
cd dbi-0.2.2

# setup
ruby setup.rb config --with=dbi,dbd_mysql
ruby setup.rb setup

# finish (as root)
ruby setup.rb install

Ubuntu + Bootlogd = ?

Apparently the last three releases of Ubuntu have been plagued by issues with bootlogd. In fact, it's been such an issue that boot logging is now disabled by default. I like to see what's going on under the hood during bootup, so I Google'd around and compiled the following tips to get it working again.

1) This thread contained a custom tarball of an older version of bootlogd. Grab the tarball from here. Then run the following commands (assuming it's saved in /tmp).

# unpack the source
tar xzvf bootlogd_2.86.02.tar.gz
# build the package
cd bootlogd-2.86.02
debuild -us -uc -b
cd ..
# install the package
sudo dpkg -i bootlogd_2.86.02_*.deb

2) After that, make sure it's setup at the appropriate priority in the init system.

update-rc.d -f bootlogd remove
update-rc.d bootlogd defaults 08

3) Finally, make sure it's enabled.

echo "BOOTLOGD_ENABLE=Yes" > /etc/default/bootlogd

The custom version of bootlogd writes it's log to /var/log/bootmsg instead of /var/log/boot. One final gotcha: initially, I tried running bootlogd as root from inside of X, and I got the following message:

bootlogd: cannot find console device 136:1 in /dev

This is because it's not being run from the console and does not mean that it's broken. Do a ctrl-alt-F1, login as root, and you can test running it from there.

Wednesday, August 20, 2008

What's Stealing That Port?

If something is bound to a port, and you want to know what it is, you can use the fuser command to determine what's attached. This is particularly handy if some rogue process has stolen port 80, and you want to know what it is.

root@travis-ubuntu:/home/travis# fuser -v http/tcp
USER PID ACCESS COMMAND
http/tcp: root 5950 F.... apache2
www-data 19140 F.... apache2
www-data 21501 F.... apache2
www-data 21508 F.... apache2
www-data 21521 F.... apache2
www-data 23207 F.... apache2
www-data 23209 F.... apache2
www-data 23245 F.... apache2
www-data 23353 F.... apache2
www-data 23460 F.... apache2
www-data 23555 F.... apache2

Tuesday, August 19, 2008

Reverse A File

Sometimes it's useful to cat a file in reverse order. An example would be doing things like parsing log files and such. I had a need to do this today and stumbled onto the cleverly named tac program. Usage is simple:

tac somefile.txt > somefile_reversed.txt

Preserving History

If you use more than one terminal during your workday, you've probably been annoyed that your shell history always seems incomplete. This is due to most shells overwriting existing history data in lieu of their own history when a session is closed. Fortunately, both bash and zsh support variables to avoid this behavior.

For bash, you should be able to just add the following to your bashrc:

shopt -s histappend
PROMPT_COMMAND='history -a'

For zsh, I use the following history related options:

setopt appendhistory histignoredups
setopt histignorespace extended_history
setopt inc_append_history share_history

Interactive Linux Kernel Map

Thanks to Nate for sending in this awesome map of the Linux kernel. To make it even more fun, you can scroll your mouse wheel to zoom in and out of the different components of the system.

Interactive Kernel Map

Monday, August 18, 2008

Autoplay

After a lot of hardwork and research, Grooveshark, the company I work for has released the next iteration of Grooveshark Lite, our web-based music discovery service. The newest release of the service supports an Autoplay feature, which generates recommendations based upon your listening habits. At the simplest level, you can just drop a few songs in your queue, click Autoplay, and the system will provide a continuous stream of songs based on feedback you provide.

The system uses a combination of statistically generated relational data and expert driven recommendations to provide what we hope is a very enjoyable play experience. Of course, this is a beta, so forgive me if it's not always 100% to your liking. Myself and a few other employees have had a large role in the design of this system, and I have to say that it's been one of the more interesting and challenging projects I've worked on.

Relevant Links:

Blorge

Grooveshark Lite, Featuring Autoplay

Mashable Review

Download Squad Review

Profy Review

Fayer Wayer

Seans Rant

Some Interesting Artists I've found using Autoplay:

Kathleen Edwards

The Bird & The Bee

The Shylights

The National

Gogol Bordello

Friday, August 15, 2008

Making K Useful

The default behavior of the K command is to lookup the keyword under the cursor using man. This is handy at times, but it's not very useful when I spend 90% of my day writing PHP code. Vim's online help mentions that you can set keywordprg to an alternative program, so I did the following.

set keywordprg=/home/travis/pdoc

Where pdoc is the following shell script:

 #!/bin/sh
firefox http://us3.php.net/manual/en/function.$1.php

Now I can move the cursor over a builtin function, hit K, and firefox will open to the appropriate documentation. To keep things strictly textual, you could use lynx (or links) in lieu of firefox.

For ruby code, I like to do the following:

set keywordprg=ri

For Perl:

set keywordprg=perldoc\ -tf

Thursday, August 14, 2008

Recovering a Deleted Ext3 File

A co-worker accidentally deleted a .java file containing several hours worth of work. The file resided in his home directory, which conveniently had it's own dedicated partition. We managed to recover the file using the following (hackish) technique.

1) Take the machine to single user mode.

telinit 1

2) Unmount the partition containing the data.

umount /dev/sda2

3) Identify a semi-unique pattern located in the file.

His code contained a string that matched "Dictionary()".

4) Determine how much context around the string you need.

We chose 250 lines before and after the match.

5) Run grep on the partition.

grep --binary-files=text -250 "Dictionary()" < /dev/sda2 > /tmp/dump.bin

If you're lucky, the data will be somewhere in the file. We extracted the Java code using Vim, wrote a new file to /root, and went back to runlevel 2. One warning, remember not to write the recovered file to /tmp as it's often cleared when changing runlevels and rebooting.

GOTO Column

You can quickly jump to a specified column in Vim by simply typing the column number followed by a pipe.

80| # go to the 80th column

Tar Gzip On-The-Fly

I typically create tarballs by doing "tar cvf file.tar somedir" and then gzipping the resultant tarball. Sometimes there's not enough space on disk to have an intermediate uncompressed file sitting around. As an example, I just made a backup of 48 gigs of MySQL data yesterday. One handy way to avoid storing the uncompressed file on disk is to gzip on the fly.

tar cvf - somedir | gzip -c > somedir.tar.gz

In writing this post, I just noticed that you can also just give tar the -z flag to accomplish the same thing; however, explicitly piping to gzip allows you to specify options to gzip such as compression level.

Wednesday, August 13, 2008

Change Permissions on All Directories

Sometimes I need to recursively set permissions on all directories in a file hierarchy. Other times, I just want to set file permissions while ignoring directories. Fortunately, the find command makes this easy.

# make all directories read, write, execute for owner. read, execute for group and other
find . -type d -exec chmod 755 {} \;

# make all files read, write for owner. read for group and other
find . -type f -exec chmod 644 {} \;

# make all shell scripts read, write, execute for owner. read, execute for group and other
find . -type f -name "*.sh" 755 {} \;

Tuesday, August 12, 2008

Vim Can Spell

Thanks to Chris Suter for contributing the following tip:

I always knew this was in there somewhere, but usually my compiler tells me when I'm an idiot, so I've never needed it. Latex, however, does not.

:setlocal spell spelllang=en_us

turns on spell-check and highlighting for incorrectly spelled words.

navigation:
]s - next misspelled word
[s - prev "

interaction:
zg - add misspelled word to system dictionary
zG - add misspelled word to buffer-local set of "good" words
zw - add word to 'bad' list
zW - add word to buffer-local 'bad' list
z= - view a numbered list of suggested spellings from which to choose for auto-replacement

side note: Maybe there's a word list for php fn's, since it seems to assume that when you've mistyped something, you obviously meant 0, "zero", false and NULL simultaneously and chooses arbitrarily between them. This might prevent some headaches/suicides.

Monday, August 11, 2008

SVN: Add All New Files

I've been doing a lot of Ruby on Rails dev work lately, and the framework tends to generate a lot of files. Until recently, I was using a combination of "svn status" piped into grep, awk, and xargs to accomplish adding new files recursively. A friend recently told me to check out the following solution instead.

svn --force add *

The only negative side effect I can see is that it seems to add ignored files back into the tree. This is mostly a non-issue for me, but take note.

Sunday, August 10, 2008

S3Virge Woes

I decided to salvage an old junk computer over the weekend. I burned a copy of Xubuntu, and proceeded to install. When the installer tried to initialize X, it couldn't, so I had to do the install in safe graphics mode. The video card, a low-end S3Virge 3d/2x, uses the Xorg s3virge driver. After getting everything working in crappy Vesa mode, I stumbled onto a bug report indicating that there was a bug in s3virge resolution detection that was fixed in Git. I recompiled the appropriate driver from Git, and everything works now. SO, for future reference, here are the steps to re-build the s3virge driver from Git in Xubuntu.

# install packages required for build
apt-get install xorg-dev xserver-xorg-dev libtool autoconf automake

# install git
apt-get install git-core

# check out the repo
git clone git://anongit.freedesktop.org/git/xorg/driver/xf86-video-s3virge

# cd into the repo
cd xf86-video-s3virge

# build and install
./autogen.sh --prefix=/usr && make && sudo make install

Not Vim related I know, but I figure this might help someone.

Tuesday, August 5, 2008

Window Manipulation

If you have two horizontally split windows and would like to increase the size of the current window, you can do ctrl-w +. A count can be issued beforehand, so to increase the height by 10 lines 10 ctrl-w +. Ctrl-w - will decrease the current window height. To set the current window to an absolute height do N ctrl-w _ where N is the number of lines desired.

In a vertically split scheme, you can use ctrl-w < and ctrl-w > to adjust the width. To set the current window to an absolute width, do N ctrl-w | where N is the number of lines desired.

If you have two windows split horizontally and would like to shift to a vertical scheme, you can do ctrl-w H or ctrl-w L depending on which direction you want the current window to go. Ctrl-w J and ctrl-w K will take a vertically split scheme to a horizontal orientation.

Wednesday, July 30, 2008

Quickly Populate a Table

There are a lot of times in MySQL when I want to populate a table with dummy data. I've used scripts to do this on many occasions; however, this stored procedure works just as well. I'm posting it here mainly for my own reference. Obviously the insert and while sentinel need to be modified for your particular uses.


delimiter //
create procedure populate_table()
begin declare i int default 0;
while i < 10000
do
insert into test values (i, 'test', 'test2');
set i = i+1;
end while;
end//
delimiter ;

Window Management

If you've got a bunch of split windows in Vim and want to close them, here are two handy window management shortcuts.

Ctrl-w q (unsplit one window at a time)
Ctrl-w Ctrl-o (close all windows except current)

Thanks to Bill Pitre for the tips!

Tuesday, July 29, 2008

MySQL High Availability, Sandbox + Proxy

I've been doing a lot of research and testing lately in regard to getting a high availability MySQL solution in place. Namely, I've been looking into moving from a master/slave topology to a multi-master replication ring. This article is not intended to be a comprehensive howto; rather, a cursory overview of some of the available technologies and how they fit together.

Multi-master replication is one area where MySQL leads the pack of open-source DB solutions. Until 5.x, there wasn't really a satisfying solution in regard to avoiding primary-key collissions across masters. Fortunately, this key obstacle has since been resolved with the addition of a few simple commands:

server 1:
replicate-same-server-id = 0
auto-increment-increment = 2
auto-increment-offset = 1


server 2:
server-id = 2
replicate-same-server-id = 0
auto-increment-increment = 2
auto-increment-offset = 2


By defining a unique increment and offest value per server, primary key collisions on inserts are avoided when the binlog is processed. Telling MySQL not to replicate the same server ID means that in a circular replication topology, once an event has made it's way around the ring, it stops at the server from which it originated.

Setting up a test environment with multiple MySQL servers used to be a pain. Luckily, a relatively new project on the scene allows one to easily setup an arbitrary number of MySQL install instances on a single machine. Enter MySQL sandbox. Setting up a replication ring consisting of two masters is a matter of downlading the tarball of the version you want to sandbox and running the following command:

./make_replication_sandbox --master_master /path/to/mysql/tarball.gz

Passing the appropriate command-line arguments allows you to specify the number of servers you want sandboxed, concurrent versions you want installed, the topology, etc...

The last piece of software I want to mention is MySQL proxy. Although still in pre-beta, proxy is one of the more impressive pieces of software I've seen lately. As most proxies are, it's a piece of middleware, in this case, delegating wire requests between a client and MySQL server. An embedded lua interpreter allows fine-grained control over all aspects of communication. Creative uses include things like re-writing query syntax and providing a SQL interface to the Unix shell; however, the real attractiveness of proxy in my opinion is that it provides two very valuable facilities not baked into MySQL by default. The first is load balancing, and the second is failover. Consider an invocation such as the following:

#!/bin/sh
LUA_PATH="/usr/share/mysql-proxy/?.lua" /usr/sbin/mysql-proxy \
--proxy-address=:3306 \
--proxy-lua-script=/usr/share/mysql-proxy/rw-splitting.lua \
--proxy-backend-addresses=127.0.0.1:19101 \
--proxy-backend-addresses=127.0.0.1:19102 \
--proxy-read-only-backend-addresses=127.0.0.1:19103

This would tell proxy to listen on port 3306 in place of your standard MySQL server. Two read-write backends are given to proxy (ports 19101 and 19102) and one read-only backend (port 19103) is provided. Proxy is smart enough to delegate read-write requests to the masters and read-only requests to the slave. From my tests, requests seem to be sent to the first master by default until load picks up. From there, they seem to be load balanced in a round-robin type fashion. In the event that a server fails, proxy will gracefully remove a backend from the server pool until it comes back online. In regard to concurrency, it can scale well into 1000 connections from my own tests; although, real-world usage obviously depends on your hardware configuration among other things. Used in concert with MySQL sandbox, the possibilities are left to your imagination.

Obviously, I've only scratched the surface of each of these technologies. Really digging into their capabilities and learning their inherent limitations is left as an exercise to the reader. I leave you with the following links.

Dual-Master MySQL Replication Done Right


Advanced Replication Techniques

MySQL Sandbox

MySQL Proxy

Tuesday, July 22, 2008

Less? More? Vim

When viewing large files, people typically tend to use a pager program such as more or preferably less. Neither of these buffers the entire file into memory, so it's quick to open a large file and scroll around. If you're using more, you should try less. It lets you scroll backwards in the file as well as forwards.

Out of the box, Vim doesn't do so great for really big files. Features like syntax highlighting and undo history can really slow things down on huge buffers; however, there is a plugin that can ease the pain. The LargeFile plugin is available here, and it really does seem to speed up opening and moving around in files larger than a few megs. A word of caution though, if you're dealing with really huge files, a pager program is still the way to go due to the sheer amount of memory required to manipulate those files.

Monday, July 21, 2008

Untar a Single File

From time to time, I've had the need to extract a single file from a tar archive. How to do so is pretty poorly documented in the man page, but it's actually easy to do. If you have a tarball named "my.tar" with a group of files in it, and you want to extract "hello.txt", you would do the following:

tar -x hello.txt -vf my.tar

Before I extract anything from a tarball, I always preview it's contents to avoid "tarbombs" (tarballs that extract directly into the CWD).

tar -tvf my.tar

Or if it's gzipped:

tar tzvf my.tar.gz

Lastly, newer versions of Vim can browse a tarball out of the box and open any file contained within.

vim my.tar (displays a file browser)

Friday, July 18, 2008

Sorting Files By Size

In case anybody was wondering, I'm still alive. I've been inordinately busy lately and have been spending the majority of my free time online re-learning Ruby on Rails. I've also been digging into multi-master and circular replication schemes for MySQL and MySQL proxy. I plan on sharing some of the things I've learned sometime in the near future. Anyway, here's a handy tip that I just used this morning.

If you want to sort all the files in a directory by size, you can do the following:

ls -lsR | sort -nr

The sort command will order by the number of blocks in each file. The biggest files are at the top.

Monday, June 30, 2008

Command-Line Vim

Before anybody says anything, I know this can often be better accomplished using sed, perl, <your favorite tool>. That said, you can use any command in the ex toolbox directly from the command-line without actually opening Vim. Any argument preceeding the filename to be edited that begins with a plus symbol (+) is taken as an ex-command. Alternatively, you can use the -c command line flag to indicate an ex-command is being specified. This has two handy uses. For one, you can pre-process a file with an ex-command before it's loaded into Vim. Secondly, you can use Vim in a script non-interactively. For example:

travis@travis-ubuntu:/home/travis% echo "hello world" > fun.txt
travis@travis-ubuntu:/home/travis% vim "+s/hello/goodbye cruel/" "+wq" fun.txt
travis@travis-ubuntu:/home/travis% cat fun.txt
goodbye cruel world

You could get an identical result with:

vim -c "s/hello/goodbye cruel/" -c "wq" fun.txt

As an aside, this is the 100th post to Daily Vim. I want to thank everyone who's contributed to the blog so far and made this a great learning experience. You guys have made this a really fun project, so keep the comments coming!

Tuesday, June 24, 2008

MySQL Pager = Vim

When using MySQL, you can set any pager you wish using the pager command.

mysq> pager less
PAGER set to 'less'

Using Vim as your pager allows you to quickly munge query output into whatever format you want.

mysql> pager vim -
PAGER set to 'vim -'

Thanks to Jay for the tip!

Thursday, June 19, 2008

Grep For a Column

Sometimes when using MySQL, you need to know every table that contains a given column name. In the past I've accomplished this with a short script, but I've come to find out that you can simply use a SQL statement instead.

SELECT table_name FROM information_schema.COLUMNS WHERE table_schema='my_db' AND column_name='MyColumn';

Apache MaxClients

This isn't an exact science, but assuming you're using pre-fork and not MPM inside Apache 2, you should be able to estimate your MaxClients setting using the following strategy.

1) calculate average RSS usage for apache

Do a ps -ylC httpd and average the RSS column using a spreadsheet or command-line trickery. This gives you the average memory size of your Apache children.

2) apply the following formula

(server RAM * MEM percent dedicated) / average RSS usage

Given a server with 10 Gigs of RAM with 80% dedicated to Apache and an average RSS size of 20 megs, and using Google as a calculator:

(10 gigabytes * .80) / (20 megabytes) = 409.6 connections

From there, you could set MaxClients to roughly 400 connections and monitor resource usage. As always, peak load performance must be taken into account as well. Like I said, it's not an exact science; rather, a simple strategy for establishing a baseline value.

Wednesday, June 18, 2008

Quick Redraw

You can use z. in Vim to quickly redraw the current cursor line at the center of the window. zt redraws at the top and zb redraws at the bottom. Thanks to Nate for the tip!

Tuesday, June 17, 2008

Pushd + Popd

Although pushd and popd are intended mainly for programming, they fill a nice niche when navigating directory structures. If you're current working directory is some deeply nested location like /home/travis/src/myprojects/vim-patch/includes and you need to cd into /tmp for a bit but know you'll be coming back to the original directory, you should use pushd rather than cd. pushd takes the current directory and pushes it into a stack. popd pops the top directory off the stack and does an automatic cd into it. For example:

travis@travis-ubuntu:/usr/share/vim/vim71/colors% pushd /tmp
/tmp /usr/share/vim/vim71/colors
travis@travis-ubuntu:/tmp% popd
/usr/share/vim/vim71/colors
travis@travis-ubuntu:/usr/share/vim/vim71/colors%

Monday, June 16, 2008

Text Formatting Features

A friend of mine contributed the following tip. I was going to rewrite it, but he explained it very clearly, so here it is verbatim:

I've been editing a plain text latex input file and am using vim's text formatting features for the first time. First useful item is :set textwidth=80 (or your preferred column width) which auto-wraps lines as you type. This is great, unless you edit part of the file before the end and mess up the text width. I discovered through the glorious vim help system the normal command gq which auto-formats a range of text. Its default behavior is a bit weird if you use it on the whole file (like gg=G for auto-indenting code), but you can do nifty things like:

format the current paragraph: {V}gq ( { = go to previous white-space-only line V=visual line select }=go to next white-space-only line )

here's how to apply formatting to all "paragraphs" using the :global command:

:%g/^\s*$\n\s*[A-Za-z]/normal V}gq

This is really specific to LaTeX, since "paragraphs" are divided by white space lines. This pattern will match any whitespace-only line, followed by a line starting with (possibly) whitespace, then an [A-Za-z].

Thanks Chris for the great tip!

Thursday, June 12, 2008

Comment Out Line Range

This is one of my favorite ex hacks. It's pretty old school and should work all the way back to the original Vi. This assumes your comment character is # (octothorpe), but would work just as well with // comments or any other line-wise comment character. Go to column zero of the first line you want to start your comment block on. Hit ma to set a mark. Now scroll down to column zero of the last line you want to comment out. Issue the following ex command:

:'a, . s/^/# /

That's it. It runs a substitute on the range from the mark stored in register 'a to the current line. Take note that you can specify ranges for any ex command using the same technique.

Wednesday, June 11, 2008

A Few MySQL Tricks

Here are a few tricks I use quite a lot with MySQL.

1) copy an existing table's schema to a new table

This is handy for making a quick copy of an existing table to test indexes, populate data, etc...

CREATE TABLE NewTableName LIKE OldTableName;

2) copy an existing table to a new table

If you want more than just the structure, you can copy a table's raw data to a new table really easily. The downside is that keys and other constraints aren't preserved.

CREATE TABLE NewTableName SELECT * FROM OldTableName;

3) copy a table's exact structure and data

If you want to preserve everything (constraints, keys, etc...) and copy the data, it requires two steps.

CREATE TABLE NewTableName LIKE OldTableName;
INSERT INTO NewTableName SELECT * FROM OldTableName;

4) quickly load data while specifying columns

After reading the MySQL forums, I've realized a lot of people online think this is impossible. It's actually right there in the documentation; although, somewhat obscurely tucked away. MySQL DOES allow you to specify what columns to operate on when using LOAD DATA INFILE. Just use the following syntax.

LOAD DATA INFILE '/some/file/name.csv' INTO TABLE SomeTableName FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (field1, field2, field3);

The field list goes at the end. Adjust the statement according to your needs, and enjoy the 20X increase in data load times.

5) atomic rename

There are a number of situations where you want to replace an entire table worth of data with a new table instantly. This is easily accomplished with an atomic rename and avoids a brief downtime while the tables are being switched.

RENAME TABLE TableName TO TableNamePrevious, TableNameNew TO TableName;

6) monitor a long running insert

If you've ever dealt with long-running inserts, you probably know how annoying it can be to just sit and wonder when the task will complete. For inserts performed in a single transaction, a simple select count(*) won't do because they're all or nothing. Take the following example:

INSERT INTO TableA SELECT * FROM TableB;

Fortunately, there's a way to peak behind the scenes. Do a count on the target table beforehand and make sure it has a populated primary key column. From there, you can do:

SHOW CREATE TABLE TableName;

Look at the last line of the table definition, and you'll see an AUTO_INCREMENT value. That's the last increment of the primary key. If you started with 25,000 rows, and AUTO_INCREMENT is 50,000 then you know 25,000 new rows have been inserted.

7) monitor a LOAD DATA INFILE

If you're using InnoDB, you can do SHOW INNODB and look at the transactions list. From there, "undo entries" displays the number of rows inserted via the current LOAD DATA INFILE operation thus far. This is convenient because I don't believe the AUTO_INCREMENT trick works for LOAD DATA INFILE.

Tuesday, June 10, 2008

Quickly Find Subdir

I used to use a long-winded invocation of find with the prune option to non-recursively find all the directories in the current dir:

find . -name . -o -type d -print -prune

From there, I moved to using ls and grep:

ls -l | grep '^d'

I recently learned that the following shell pattern works just as well:

ls -ld */.

Monday, June 9, 2008

Source A File

If you make changes to your ~/.bashrc or ~/.bash_profile or ~/.zshrc, etc... you can load the changes without logging in and out again by issuing a source command:

source ~/.bashrc # load changes, execute commands, etc...

alternatively, you can use the shortcut dot syntax:

. ~/.bashrc

You can do the same in Vim to read a file of ex-commands such as your ~/.vimrc.

:source ~/.vimrc

Friday, June 6, 2008

Syntax Check

When working on anything important, it makes sense to check the syntax of your file before setting it live. This is usually accomplished with a simple command-line flag. Consider the following:

travis@travis-ubuntu:/home/travis% ruby -c test.rb
Syntax OK

travis@travis-ubuntu:/home/travis% perl -c test.pl
test.pl syntax OK

travis@travis-ubuntu:/home/travis% php -l test.php
No syntax errors detected in test.php

Python doesn't make this as easy. I'm not sure why because it seems to have a pretty well defined separation between it's compiler and runtime, but the general consensus is to break your code into classes and just have a single "main" file for your runtime code. That way you can just run python filename, which will compile the file to bytecode as well as check syntax; however, this is frustrating if you just want to check the syntax and not actually run the program (common with DB applications). Consider the following hack an alternative, which could easily be wrapped up in a helper shell script.

python -c "import py_compile; py_compile.compile('test.py')"

This produces no output on success and an error on failure. It has the possibly unwanted side effect of compiling to bytecode, but I don't think there's any getting around that.

Thursday, June 5, 2008

Leaning Toothpick Syndrome

One thing that's always bothered me about PHP is it's lack of a qq style operator. For example, in Perl, I could do something like this:

print qq/a $fun "string" with "double quotes"/;

In PHP it'd be something gross like:

print "a $fun \"string\" with \"double quotes\"";

Anyway, I found a nice solution that's been right under my nose all this time:

printf('a %s "string" with "double quotes"', $fun);

This is quite a bit cleaner for things like hyperlinks, so I thought I'd pass it along.

Ruby borrows quite a few ideas from Perl, and has a very nice %Q operator:

print %Q/a #{fun} "string" with "double quotes"/

going even further, you can use %r for regexp:

mystring =~ %r{/usr/src/linux} # no leaning toothpick required

or in Perl:

$mystring =~ m#/usr/src/linux#

Wednesday, June 4, 2008

Google Shell

I haven't played around with this very extensively, but I'm a fan of just about any sort of shell, so I thought this was sort of fun.

A non-google sponsored Google Shell.

Tuesday, June 3, 2008

Star Power

This is a very simple tip, but some people may not know it. If your cursor is positioned on a given word in Vim and you press the `*' key, it will jump to the next word of the same name.

Monday, June 2, 2008

Hashes Under The Hood

If you've ever worked with hashes before and wondered how they work behind the scenes, I've written a very straightforward Ruby implementation of a hashing class. The "lookup" method in particular is pretty easy to follow and should give a decent understanding of how hash retrieval works in general.

Simple Hashing Implementation

Edit: Looks like Ruby Garden is temporarily down. Hopefully it will be back up soon.

Friday, May 30, 2008

A Different Mark Jump

Most people know how handy marks are and most use them to jump to the exact location marked. You can also jump to the first character on the marked line rather than the exact location. Just use '{a-z} rather than `{a-z}.

Thanks to Nate for this tip as well.

Thursday, May 29, 2008

Two Copies, Same Buffer

Oftentimes I like to have two copies of the same window open in a horizontal split, so I can look at different pieces of code in the same file. Vim has a handy shortcut to quickly split the current file in a new window. Just hit ctrl-w s. A side bonus is that you don't have to save the file first unlike using :new filename.

Thanks to Nate for the tip!

Wednesday, May 28, 2008

Easier Shared Key Authentication

I use ssh shared key authentication to access a good number of servers I work on. In an effort to make things more secure, the shared keys usually have passwords associated with them. Generally speaking, I try to keep remote sessions alive inside of screen or just minimized on my current workspace, but I'm a habitual window closer, so I end up having to login again pretty regularly. Luckily, openssh has a facility that saves me from typing my passwords over and over again. I keep all my keys inside /home/myuser/.ssh for convenience sake. When I start my desktop session, I cd into that directory and do ssh-add *. It prompts me to type each distinct password once, and that's it. After that, the ssh agent has the password stored in memory until I terminate my desktop session.

Tuesday, May 27, 2008

Running Out of Space?

Running out of space on your HD? I am on a few of my machines. I found the following small Perl script that does a great job helping me locate redundant files. It recurses through a given root directory and looks for files of the same size. When it finds two files of the same size, it does an md5sum on them to determine if they're duplicates. It runs pretty quickly and is easy to tweak to pipe into other scripts.

Find duplicate files.

Thursday, May 15, 2008

Goodbye Console Beep

I guess the console beeps for a reason, but I find it annoying for the most part, and I can generally tell when something isn't working. Here's the procedure to disable console beep in X and on the console.

# Console
setterm -blength 0

# X Windows
xset b off

Start of Match

When searching or writing macros in Vim, it's sometimes helpful to have a search position your cursor at the end of the matching term rather than the beginning. You can use the start of match atom to accomplish this.

/some term\zs

This would match "some term" and position your cursor at the end of the match. See (help :regexp) for other atoms and general regexp goodness.

Thanks to Chris Sutter for contributing this tip.

Tuesday, May 13, 2008

Zsh Alias Suffixing

Here's a handy feature I just noticed with Zsh. You can specify a suffix when creating an alias, and execute an arbitrary command on the given file suffix.

alias -s php=vim
alias -s java=vim
alias -s pl=vim

Now typing "foo.php" on the command-line will open the file in vim automatically. You can get fancy and do stuff like:

alias -s org=firefox-2

Type "vim.org", and your browser goes to the URL.

Monday, May 12, 2008

Bash Substring Manipulation

I recently completed a project for work to automate pulling specific subversion revisions of code and publishing them to various servers. While relatively simple in practice, a number of operational dependencies had to be considered, and it had to be written in bash. I've done a bit of shell programming in the past, but never really written a "system" in shell. Not surprisingly, I learned a few things in the process, which I'll be sharing on this blog as they come to mind. One thing I learned, which will come in handy down the road, is that bash has a number of builtin substring operations. As an example, I had to trim a number of paths to insure that there wasn't a trailing forward-slash at the end. Obviously, there are a few ways of doing this, but I found a very convenient and simple idiom that I'll be adding to my toolbox.

${string%substring}

removes shortest match of substring from end of string

so, if mydir="/home/travis/"

echo "${mydir%/}" will produce /home/travis
echo "${mydir%/*/}" will produce /home

What's nice as opposed to just chopping the last character off the string is that if the string is already formatted without a trailing slash, nothing will change.

mydir="/home/travis"
echo "${mydir%/}" will produce /home/travis

Friday, May 9, 2008

Demystifying File Attributes

Unix based operating systems store three attributes regarding file access and modification. At least one of these attributes is a common source of confusion, and while it's generally a harmless mistake to misinterpret it's meaning, knowing what it actually means could save you some headache down the road.

atime (access time):

This is the last time a file was accessed or read. It's also updated when a file is executed.

view with: ls -lu

mtime (modification time):

This is updated whenever a file is modified.

view with: ls -l

ctime (change time):

This attribute is regularly confused for "creation time", which is totally wrong. It's actually not possible to determine a file's creation time on a Unix based OS. Rather, ctime actually means "change time". This flag is set whenever a files owner or permissions change or when the contents change.

view with: ls -lc

Thursday, May 8, 2008

Reverse Lines in a File

You can quickly reverse all lines in a file using the following ex command.

:g/^/m0

This is particularly useful for files that are in chronological order such as logs or email mbox files. Thanks to Skyler for submitting the tip.

Wednesday, May 7, 2008

Number Pad Madness

A friend of mine was trying to use the numpad inside Vim using Putty (Win32 SSH Client). Unfortunately, this wasn't working for him and was just echoing a bunch of alphabetic characters to his term. After looking online for a few minutes, we found a fix.

Go into Terminal->Features-> and check in "Disable Application keypad mode". Hopefully this helps some Windows users out there.

Tuesday, May 6, 2008

Right Help, Right Mode

Vim's help system is incredibly comprehensive and dare I say, "kick ass". One thing that initially frustrated me though was that searching for a keystroke (i.e., :help ctrl-p) would always return the documentation for normal mode. You can search help for any mode by prepending an identifier to your query (:help i_ctrl-p) would display documentation for ctrl-p from insert mode. (:help v_ctrl-c) would explain ctrl-c from visual mode. Also, remember you can tab-complete help to make finding the right document easier.

Monday, May 5, 2008

Completion

This is one of my favorite features in Vim. In fact, I use it so often that it didn't occur to me until recently that some people may not take advantage of it. If you're familiar with getting around on a modern shell (bash, csh, zsh, etc...), you're almost certainly acquainted with command completion. Vim offers a similar facility that works on the current buffers you have open (although it can do ex command completion as well ;-). When working in your current window, simply type a portion of a string (i.e., hereIsAReallyLong...) then hit ctrl-n. Vim should display a menu of possible completion options. Scroll up and down the menu with ctrl-p and ctrl-n. Hit enter when you find the entry you want to insert. This saves a lot of typing on longer method names and strings and also helps avoid spelling typos. If you find ctrl-p and ctrl-n too cumbersome to type or simply want to make completion feel more like it does on the shell, you can grab the SuperTab Plugin and use that instead.

Side tip: inside of ex mode, you can just use the tab key to complete your ex commands.

Friday, May 2, 2008

Finding Recent SVN Commits

Here's a quick Perl script I whipped up to ease finding newly committed files in our subversion repository. It's a little cleaner than what the client provides and nice to quickly find changes.

#!/usr/bin/perl -w

#
# quick script to succinctly display SVN changes since a given date
# usage: svnsince.pl YYYY-MM-DD HH:MM
#

use strict;
use Date::Calc qw(Date_to_Time);

if (!defined($ARGV[0]) || $ARGV[0] !~ /^\d\d\d\d-\d\d-\d\d$/) {
die("Invalid date. Usage: $0 YYYY-MM-DD HH:MM.");
}

if (!defined($ARGV[1]) || $ARGV[1] !~ /^\d\d:\d\d$/) {
die("Invalid time. Usage: $0 YYYY-MM-DD HH:MM.");
}

my $date_thresh = date_to_epoch($ARGV[0], $ARGV[1]);
my $svn_file = "";
my $svn_date = "";
open(SVN, "svn info -R |") || die("Can't run svn client: $!");

while (<SVN>) {
if (/^Path: (.+)/) {
$svn_file = $1;
}

if (/^Last Changed Date: (\d\d\d\d-\d\d-\d\d) (\d\d:\d\d)/) {
$svn_date = date_to_epoch($1, $2);
if ($svn_date >= $date_thresh) {
print "$1 $2 $svn_file\n";
}
}
}
close(SVN);

sub date_to_epoch {
my ($date, $time) = @_;
my ($year, $month, $day) = split(/-/, $date);
my ($hour, $minute) = split(/:/, $time);
return Date_to_Time($year, $month, $day, $hour, $minute, 0);
}

Remove Empty Lines

Here's a quick sub to remove all the empty lines in a file. I use this to cleanup after macros that leave a bunch of blanks.

:%s/^\n//

Wednesday, April 30, 2008

Character Above, Below

If you're in insert mode, and you want to insert the character above the cursor, hit ctrl-y. To insert a character below the cursor, hit ctrl-e. Alternatively, if you want to grab entire words above and below, you can use the following mappings (which use text objects).

inoremap <C-Y> <Esc>klyiWjpa
inoremap <C-E> <Esc>jlyiWkPa

Rebuild a Debian Package

If you use Debian, Ubuntu, or some other Debian derived Linux distribution, there may come a time when a packages default configuration options don't meet your needs. This generally leaves you two options. You can either build from source and no longer have the luxury of the package manager maintaining your package, or you can rebuild the package with the custom options included in the build. This is easier than it sounds and really only involves a few steps. In this example, I'll rebuild netcat.

1) make sure you have the packages required to re-build a .deb

apt-get install devscripts build-essential fakeroot

2) fetch the source for the package to re-build

apt-get source netcat

3) fetch dependent packages for the build

sudo apt-get build-dep netcat

4) unpack vendor specific source archives

# determine appropriate dsc file
ls *dsc
netcat_1.10-33.dsc
# run dpkg-source on dsc file
dpkg-source -x netcat_1.10-33.dsc

5) cd into source directory

cd netcat-1.10

6) modify source if necessary, apply patches, etc...

7) set custom build options if desired (optional)

# fake DEB_BUILD_OPTIONS here, just for the sake of example...
DEB_BUILD_OPTIONS="--enable-sockets" CC=gcc-3.4 fakeroot debian/rules binary

8) build the package

dpkg-buildpackage -rfakeroot -b

9) install the package ;-)

sudo dpkg -i ../netcat_1.10-33_i386.deb

Quick Paste

You may have noticed that pasting outside text into Vim from insert mode can lead to awkwardly stair-stepped text. You may also know that this is easily avoidable via :set paste from normal mode. I paste from outside often enough, that I've added the following to my vimrc making it that much easier.

set pastetoggle=<F5>

Repeat Last Macro

There are times I want to repeat a macro just a few times. I know you can do count@{0-9a-z":*} but sometimes I don't feel like counting. You can use @@ to repeat the last macro, and it's an easy command to type very quickly, so it fits the bill.

Bash For Loops

I use bash and zsh quite a bit in system admin tasks. One thing I find myself needing to do repeatedly is run a command in a loop a given number of times. I'm very well acquainted with cron, but there are often instances where I prefer to run a bash loop inside of a screen session while logging to a file. This gives a lot more control over bleeding edge scripts, and it's easier to "check in" and see what's going on. Here are some simple ways to loop with bash:

# C style for loop
for ((i=0; i<=100; i+=1)); do
# some command
sleep 1
done

# quick loop

for I in 1 2 3; do
echo $I
sleep 1
done

# infinite loop

while [ 1 ]; do
# some repeated command
sleep 60
done

# file loop

for I in `ls *.txt`; do
echo "file $I"
done

Wednesday, April 23, 2008

Vim Visual Reference

For any visual learners in the crowd, here's a very nice visual Vim reference. There are a lot of cheat sheets out there, and I may add a few more to this post down the road.

Vim Visual Reference.

Friday, April 18, 2008

Finding a File's Owner

I generally gravitate towards Debian derived Linux distributions; although, I spend quite a lot of time on the Red Hat side of the tracks as well. My primary reason for liking Debian and Ubuntu is the package management. I plan on doing a whole series on apt and dpkg tricks in the future, but for the moment, here's a really easy way to find out what package a file belongs to.

travis@travis-desktop:/home/travis% dpkg -S /bin/bash
bash: /bin/bash

Only install from tarballs as a last resort, and you should be able to track down over 90% of the files on your system.

Stripping Non-Printable Characters

Here's a quick way to strip non-printable characters in PHP. This is pretty handy for cleaning data before putting it in a DB.

$val = preg_replace('/[^\r\n\t\x20-\x7E\xA0-\xFF]/', ' ', $val)


Being a Perl compatible regexp, you can use it in your language of choice so long as it supports PCRE.

UPDATE:

Steve Laniel reminded me that you can use a Posix regexp to do roughly the same thing:

$val = preg_replace( '/[^[:print:]]/', '', $val )

This is a lot simpler for most cases; although, the patterns are slightly different (Posix is \x20-\x7E).

Thursday, April 17, 2008

Easier Search + Sub

I can't say how many times I've tried to work out a complex substitute command in Vim only to have the pattern fail and not match what I wanted. I've recently come across an easier way to accomplish a substitution on a pattern that's already guaranteed to match. Say you have the following data in your editor.

"this is a line 12345"

Execute a search (instead of a sub):
" match digits
/\d\+

Now execute a sub with no pattern:

%:s//7890/g

The result would be:

"this is a line 7890"

Obviously this is a simple example, but I find this technique handy whenever I'm working out complex patterns.

Be a Better Programmer

Years ago, I became interested in Lisp and embarked on a path from which I would never return. I learned a good chunk of the language and spent a lot of late nights sitting inside the interactive shell provided by the CMUCL Lisp interpreter. While I don't do much Lisp programming these days, I've since incorporated elements of functional programming into all the code I write. Two basic tenants I've adopted as widely as possible are, "rely on return values whenever possible" and "avoid side effects". You can apply those tenants to OOP, procedural programming or really any style. Basically, avoid passing a pointer (or reference) to a function (or method) unless performance dictates otherwise. Rely on a single return value whenever possible. Returning an array is fine if it makes sense in the context of the method (array_unique() for example), but if your method is returning two distinctly different values, it might be a good idea to refactor. Avoiding side effects means that every method you write should do one thing. If your method prints and returns a data structure, consider refactoring into two methods (one that returns, one that prints). Return values should be as consistent as possible across your program.

The Ruby programming language works on the "principle of least surprise". You don't have to be a Ruby programmer to follow this very simple but effective rule. No matter what language you're writing in, try to think of what the "average programmer" would expect your method to accept and return. There's a time and a place for condensing twenty lines of code into two or three insanely terse pieces of syntactic sugar, but simplicity generally rules the day.

Python, Perl, Ruby, and PHP provide a lot of the facilities that made Lisp so popular to begin with. One such facility is an interactive shell. When I'm writing code, I don't waste time guessing exactly how a method or piece of code is going to behave. I have an interactive shell running alongside my editor window most of the time. This allows me to quickly evaluate a methods behavior if I'm not sure exactly what it will return. Here are instructions on firing up interactive shells in those languages.

If you don't currently program interactively, I highly suggest trying it. You'll get your work done faster and there will be less bugs at runtime.

Python:

Being a language with batteries included, all you have to do is invoke the python interpreter, and it drops you directly into an interactive shell.

travis@travis-desktop:/home/travis% python
Python 2.5.1 (r251:54863, Mar 7 2008, 04:10:12)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 1 + 1
2

Perl:

Perl's debugger provides an interactive shell. It doesn't have readline support by default, which is a minus, but it still gets the job done effectively.

travis@travis-desktop:/home/travis% perl -d # press ctrl-d after you hit enter (sends EOF)

Loading DB routines from perl5db.pl version 1.28
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

Debugged program terminated. Use q to quit or R to restart,
use o inhibit_exit to avoid stopping after program termination,
h q, h R or h o to get additional info.

DB<1> x 1 + 1
0 1

Ruby:

Ruby uses a program called irb to provide an interactive shell. Depending on your Ruby distribution you may have to install it separately using your package manager of choice. Once it's installed, irb provides readline support and is on par with the python's interactive shell.

travis@travis-desktop:/home/travis% irb
irb(main):001:0> 1 + 1
=> 2

PHP:

I have pretty mixed feelings about PHP, but that's another rant entirely. PHP includes an interactive mode out of the box, but a fatal error will drop you back to the command-line. Either way, it's better than nothing.

travis@travis-desktop:/home/travis/Apps% php -a
Interactive mode enabled

<?
echo 1 + 1, "\n";
2
?>

A better solution is to use this third-party app:

http://jan.kneschke.de/projects/php-shell/

Once it's installed, you have a full-featured interactive shell with readline support. It also handles exceptions gracefully.

travis@travis-desktop:/home/travis% php-shell.sh
PHP-Shell - Version 0.3.1, with readline() support
(c) 2006, Jan Kneschke <jan@kneschke.de>

>> use '?' to open the inline help

>> 1 + 1
2

Wednesday, April 16, 2008

Slammed

Sorry, no tip in the last day or so. I've been completely slammed at work launching our new product, Grooveshark Lite.

Monday, April 14, 2008

History

Most people know you can use the Unix history command to see what's been done on the command-line recently.

travis@travis-desktop:/home/travis% history
9985 ls
9986 grep -ri pidgin
9987 grep -ri pidgin *
9988 cd ..
9989 ls
9990 cd .gnome2
9991 grep -ri pidgin *
9992 cd ..
9993 cd .purple
9994 ls
9995 vim prefs.xml

What some people don't know, is you can execute anything in the history list very easily. Just use !number where number is the item number on the left-hand side of the history output.

!9995 would execute `vim prefs.xml'.

Friday, April 11, 2008

Using Awk to Grab a Column

A lot of times when hacking on the command-line, I need to filter a particular column coming in from a pipe. A good example would be grabbing a pid from ps output or cat'ing a file with mysql process status and grabbing mysql thread id's for a subsequent kill. I generally use awk to do this because it's installed everywhere, and it makes doing this very simple. Take the following example:

user@host.org [~]# ps aux
root 2 0.0 0.0 0 0 ? S Mar09 1:04 [migration/0]
root 3 0.0 0.0 0 0 ? SN Mar09 0:10 [ksoftirqd/0]
root 4 0.0 0.0 0 0 ? S Mar09 1:14 [migration/1]
root 5 0.0 0.0 0 0 ? SN Mar09 0:10 [ksoftirqd/1]


Now to grab the pid's (2,3,4,5):

[~]# ps aux | awk '{print $2}'

2
3
4
5

One step further to kill those pid's (you wouldn't want to kill these, but...)

[~]# ps aux | awk '{print $2}' | xargs kill -15

Last Jump Location

This technique was used in the "quick uncomment" tip I posted recently, but I figured I would mention it explicitly. A lot of times when I'm editing a file, I'll set a mark at my current location because I need to do something like jump to the bottom of the file and change something, and I anticipate needing to return to the current line. If you know you're going to quickly return to the current position, there's no need to set a mark. Just hit `` and it will return you to the position you were at before the last jump.