Tag: fs

Approaches to data recovery

There are a lot of howtos and tutorials for using data recovery tools in Linux, but far less on how to choose a recovery tool or approach in the first place. Here’s an overview with suggestions for which route to go or tool to use:

Cause Outlook Tools
Forgotten login password Fantastic Any livecd
This barely qualifies as data recovery, but is included for completeness. If you forget the login password, you can just boot a livecd and mount the drive to access the files. You can also chroot into it and reset the password. Google “linux forgot password”.
Accidentally deleting files in use Excellent lsof, cp
When accidentally deleting a file that is still in use by some process — like an active log file or the source of a video you’re encoding — make sure the process doesn’t exit (sigstop if necessary) and copy the file from the /proc file handle. Google “lsof recover deleted files”
Accidentally deleting other files Fair for harddisks, bad for SSDs testdisk, ext3grep, extundelete
When deleting a file that’s not currently being held open, stop as much disk activity as you can to prevent the data from being overwritten. If you’re using an SSD, the data was probably irrevocably cleared within seconds, so bad luck there. Proceed with an fs specific undeletion tool: Testdisk can undelete NTFS, VFAT and ext2, extundelete/ext3grep can help with ext3 and ext4. Google “YourFS undeletion”. If you can’t find an undeletion tool for your file systems, or if it fails, try PhotoRec.
Trashing the MBR or deleting partitions Excellent gpart (note: not gparted), testdisk
If you delete a partition with fdisk or recover the MBR from a backup while forgetting that it also contains a partition table, gpart or testdisk will usually easily recover them. If you overwrite any more than the first couple of kilobytes though, it’s a different ballgame. Just don’t confuse gpart (guess partitions) with gparted (gtk/graphical partition editor). Google “recover partition table”.
Reformatting a file system Depends on fs e2fsck, photorec, testdisk
If you format the wrong partition, recovery depends on the old and new file system. Try finding unformat/recovery tools for your old fs. Accidentally formatting a ext3 fs to ntfs (like Windows helpfully suggests when it detects a Linux fs) can often be almost completely reverted by running fsck with an alternate superblock. Google “ext3 alternate superblock recovery” or somesuch.

Reformatting ext2/3/4 with ext2/3/4 will tend to overwrite the superblocks, making this harder. Consider PhotoRec.

Repartition and reinstall Depends on progress
If you ran a distro installer and accidentally repartitioned and reformatted a disk, try treating it as a case of deleted partitions plus reformatted partitions as described above. Chances of recovery are smaller the more files the installer copied to the partitions. If all else fails, PhotoRec.
Bad sectors and drive errors Ok, depending on extent ddrescue
If the drive has errors, use ddrescue to get as much of the data as possible onto another drive, then treat it as a corrupted file system. Try the fs’ fsck tool, or if the drive is highly corrupted, PhotoRec.
Lost encryption key Very bad bash, cryptsetup
I don’t know of any tools made for attempting to crack a LUKS password, though you can generate permutations and script a simple cracker if you have limited number of permutations (“it was Swordfish with some l33t, and a few numbers at the end”). If you have no idea, or if your encryption software uses TPM (rare for Linux), you’re screwed.
Reformatted or partially overwritten LUKS partition Horrible
LUKS uses your passphrase to encrypt a master key, and stores this info at the start of the partition. If this gets overwritten, you’re screwed even if you know the passphrase.
Other kinds of corruptions or unknown FS Indeterminable PhotoRec, strings, grep
PhotoRec searches by file signature, and can therefore recover files from a boatload of FS and scenarios, though you’ll often lose filenames and hierarchies. If you have important ASCII data, strings can dump ASCII text regardless of FS, and you can grep that as a last resort.

If you have other suggestions for scenarios, tools or approaches, leave a commment. Otherwise, I’ll wish you a speedy recovery!

What’s up with directory hard link counts?

Ever considered the hard link count from ls on directories? 

 
vidar@kelvin ~/src $ ls -l
total 108
drwxr-xr-x  4 vidar vidar  4096 2009-11-22 12:52 aml-lsb
drwxr-xr-x 13 vidar vidar  4096 2009-12-13 16:00 delta3d_REL-2.4.0
drwxr-xr-x 23 vidar vidar  4096 2010-02-02 18:22 linux-2.6.32.7
...

For files, this is the number of hard links. You can use find / -samefile filename to find all files that point to the same file inode.

So what does this number mean for directories? Exactly the same thing.

Users, including root, are blocked from creating directory hard links out of the kernel’s mortal fear of cyclical directory trees (or should I say directory graphs?). The kernel still creates them though, specifically in the form of the “.” entry in the directory itself, and “..” in each subdirectory.

An empty directory /foo/bar will have two links, /foo/bar itself, and /foo/bar/.. When creating a subdirectory /foo/bar/baz, you will get the additional hard link /foo/bar/baz/... In other words, the hard link count is the number of subdirectories plus two.

Here’s a party trick for listing directory hard links in bash:

vidar@kelvin ~/src $ ls -ld aml-lsb/{,.,*/..}
drwxr-xr-x 4 vidar vidar 4096 2009-11-22 12:52 aml-lsb/
drwxr-xr-x 4 vidar vidar 4096 2009-11-22 12:52 aml-lsb/.
drwxr-xr-x 4 vidar vidar 4096 2009-11-22 12:52 aml-lsb/bin/..
drwxr-xr-x 4 vidar vidar 4096 2009-11-22 12:52 aml-lsb/lib/..
vidar@kelvin ~/src $

Clearly, each of them refers to the same thing, and the numbers add up (if they don’t, shopt -s dotglob)

As a side note, you can use mount --rbind to fake a directory hard link. This will remount a directory and all submounts on some other directory, but will prevent cycles.

You can also use mount --bind to remount without submounts. This can be useful for when you want to copy the contents of a directory that has another file system mounted over it. This is most commonly /dev, which is over-mounted with udev early in the boot process. Many people don’t realize that they have an entire /dev they’ve never seen!