“Linux ate my ram!”

About once a week I hear some poor newbie scream in terror as he discovers that his box is just seconds away from a gruesome death with barely a few megabytes of memory left. How could this have happened, it was fine when I booted it this morning, why does a Linux box need 2GB of memory just to run Apache, more bloated than Vista, etc, etc.

Then you explain about the wonders of disk caching, and invariably the first question is always “How do I disable it?”

All in all, it’s a lot of repetition.

To avoid this, I registered LinuxAteMyRam.com which features a big flashing “Don’t Panic” sign and answers the most frequent questions as reassuringly as possible.

The goal is to allow people to appreciate Linux’s disk cache for the brilliant, unobtrusive and effective optimization it is, so it skips over some details like the swappiness setting.

If you have thoughts or suggestions, do comment.

Project: Screenshot diary

So to try something new, I’ll write about a little scripting project you can try for laughs and learning. If you find this too basic, you can browse the “Advanced Linux-related things” category (and there’s an RSS feed for just those posts as well).

Now, if you know that someone is taking your picture, you try to smile and look natural (but invariably fail, with a strained smile and rigid pose as if you were caught grave robbing). The equivalent in screenshots is to either close all your apps (if you like your background image) or run a bunch of random ones (if you don’t), and then opening the program menu two or three levels. Judging from most screenshots you see, people are constantly contemplating which of their many lovely apps to run next:
Screenshot just as described Screenshot just as described Screenshot just as described

How about this for an idea: Take a new screenshot at random intervals while you actually use the desktop.

Not only do you always have a natural looking screenshot if anyone should ask, but you get basically a little timelapse of your activity. I set up such a system in 2004, and it’s more fun than it should be to flip through them all!

To do this, we’ll make a script and stick it in the crontab. Since cron can only run things at fixed intervals, we’ll use short intervals and make the script randomly choose if it should take a screenshot or not. When it does, it’ll put it with a timestamp into some directory.

Open ~/bin/takeshot. First the shebang, and in 99 out of 100 cases, we’ll just exit:

#!/bin/bash 

if (( RANDOM % 100 ))   
then
	exit 0
fi

Let’s define a good place for our screenshots:

directory=~/screenshots

Since the crontab runs independently of our X11 session, we have to specify which display to use. :0 is the first display, which on a single user box is probably the only one:

export DISPLAY=:0

While the screensavers are very nice, I don’t really want screenshots of them. Xscreensaver comes with a tool that can be used to check if the screen is currently blanked. For simplicity we use the short-hand && notation rather than a full if statement:

xscreensaver-command -cycle 2>&1 | grep -q 'cycling' && exit 0

This only works for xscreensaver, not for other screen saver packages such as xlock or the KDE screensavers. Feel free to skip.

Now let’s create the output directory if it doesn’t exist, and define the filename to use:

mkdir -p "$directory"
output="${directory}/shot$(date +%Y%m%d%H%M%S).png"

This gives us a filename with the current date and time, such as ~/screenshots/shot20090314232310.png.

Now to actually take the screenshot. There are tons of utilities for this, but the two main ones are ‘import’ from ImageMagick; and xwd (from X.org) plus NetPBM to convert it. Import is simpler to use, but I’m a fan of NetPBM for its modularity. Plus NetPBM produces png files that are half the size of ImageMagick’s. Here are both ways:

# Using ImageMagick 
import -win root "$output"
## NetPBM Alternative: 
# xwd -root | anytopnm | pnmtopng > "$output"

Now chmod +x ~/bin/takeshot and try running it a few times (you might want to temporarily delete the zeroes in “100” to speed things up). Check that the screenshots are there.

Now add it to cron. Run crontab -e and add

*/10 * * * * ~/bin/takeshot &> /dev/null

Save and exit whichever editor crontab -e invoked for you.

The script should now be taking a screenshot on average every 100*10 minutes, or 17 hours of actual use time. You can adjust either factor up and down (or make an even more clever scheme) to get more or less screenshots.

To summarise the script:

#!/bin/bash 

if (( RANDOM % 100 ))   
then
	exit 0
fi
directory=~/screenshots
export DISPLAY=:0

xscreensaver-command -cycle 2>&1 | grep -q 'cycling' && exit 0

mkdir -p "$directory"
output="${directory}/shot$(date +%Y%m%d%H%M%S).png"

# Using ImageMagick 
import -win root "$output"
## NetPBM Alternative: 
# xwd -root | anytopnm | pnmtopng > "$output"

Here are some random screenshots of mine from different years and wms:

Bunch of terminals in Fluxbox Bunch of terminals in KDE Bunch of terminals in Ion3 Bunch of terminals in Ion3, now widescreen

dd is not a backup tool!

Pretty much all Linux newbies will at some point be dazzled by the amazing powers of dd, and consider using it for backups. DON’T! Allow me to elaborate:

  1. dd must be run on an unmounted device. The point of using dd is usually to get a snapshot, but it’s not a snapshot if the system keeps running and modifying the FS while it’s being copied! The “snapshot” will be a random collection of all the states that the data and metadata were in during the 30+ minutes it took to copy.
  2. It’s hard to restore on a file by file basis. You hardly ever want to restore everything, usually you just want one file or directory that was accidentally deleted, or all files except the ones you’ve been working on since the backup was taken.
  3. It’s hard to restore to new hardware. If you suffer a massive disk crash, you will indeed want to restore everything. If you’re restoring to the same size disk, and you don’t decide that you want less swap or a bigger root partition while you’re at it, you can now easily restore and thank the gods that most FS don’t rely on disk geometry anymore. If you try to restore to a smaller disk on a secondary/old computer, you’re just screwed. If you upgrade to a larger disk (by far the most likely scenario), you’ll be playing the partition shuffle for a while to get use of the new space.
  4. It’s highly system dependent, and requires root to extract files. You can’t use your mum’s Wintendo or even your school’s Linux boxes to get out that geography report. And if you’re sick of Linux after it botched your system, you can’t switch to FreeBSD or OSX.
  5. You can’t do incremental backups. You can’t properly back up just the information that has changed. This all but kills network backups, and dramatically reduces the number of snapshots you can keep.

So when is dd a decent choice for backups?

Take a snapshot of a new laptop that doesn’t come with restoration disks, so that you can restore it if you sell the laptop to a non-geek or if the laptop needs servicing (it’ll make life easier for clueless techies, and companies have been known to use Linux as an excuse for not covering hardware repairs).

Create a disk image right before you try something major that you want to be able to reverse, such as upgrading to the latest Ubuntu beta to see if the new video driver works better with your card. Or right before installing Puppy Linux to write a little review about it. Restoring the image will be easier than downgrading/reinstalling, and you won’t have done any work in the mean time.

Image a computer and teach the kids how to install an operating system in a realistic scenario.