Thursday, August 14, 2008

Recovering a Deleted Ext3 File

A co-worker accidentally deleted a .java file containing several hours worth of work. The file resided in his home directory, which conveniently had it's own dedicated partition. We managed to recover the file using the following (hackish) technique.

1) Take the machine to single user mode.

telinit 1

2) Unmount the partition containing the data.

umount /dev/sda2

3) Identify a semi-unique pattern located in the file.

His code contained a string that matched "Dictionary()".

4) Determine how much context around the string you need.

We chose 250 lines before and after the match.

5) Run grep on the partition.

grep --binary-files=text -250 "Dictionary()" < /dev/sda2 > /tmp/dump.bin

If you're lucky, the data will be somewhere in the file. We extracted the Java code using Vim, wrote a new file to /root, and went back to runlevel 2. One warning, remember not to write the recovered file to /tmp as it's often cleared when changing runlevels and rebooting.


Anonymous said...

6. Make sure you put subversion in place

*joke* :)

Travis Whitton said...

Good point. We actually use svn for just about everything here, but svn only helps if you remember to commit ;-).

Ben said...

So I guess this becomes harder when the filesystem puts the file in multiple (discontiguous) blocks. But luckily for text/code, such files frequently fit within one block.

I wonder how much sport there is in recovering multiple-block deleted files. It would be neat if there was a utility out there that knew the on-disk formats of a handful of filesystems, and once you pointed to a string/byte match that you know the missing file to contain (a problem in itself) the utility could assemble the rest of the file. If the next-block pointers were simple enough, then a shell script with some dd could do it....

Travis Whitton said...

Ahh, "next-block pointers". If only ext3 didn't clobber them when a file was deleted.

Here's a quote from one of the ext3 developers.

`In order to ensure that ext3 can safely resume an unlink after a crash, it actually zeros out the block pointers in the inode, whereas ext2 just marks these blocks as unused in the block bitmaps and marks the inode as "deleted" and leaves the block pointers alone.'

Sadly, things were easier in ext2 in this respect.

Travis Whitton said...

I should also mention that this guy seems to have found some ways around the issues I just mentioned and has developed a tool that may assist in recovery.

Ext3 Grep.

graywh said...

A coworker and I were able to recover several deleted files from an ext3 parition. Since I knew the inode number for a deleted directory, I could search the journal log for block numbers and manually read the data from the disk with dd. A directory block stores a file's inode number after the previous file's name in reversed byte order (4a 40 e2 = 14827594). We manually pieced together non-adjacent blocks with cat and removed the unused space of the last block using vim. This method works for files up to 12 blocks large.

atoztoa said...

I have always wanted an utility to undelete files. I had been in deep trouble due to some rm. Thanks...

Check Effective Use of VIM - Part 1