This page is provided as a convenient way to download. It should also be helpful for people on dial-up to have one big page to read. A list of the other “AllOfs” can be found at http://linuxbasics.org/course/book/allallofs.

Please take into considerations that once you save this page, it cannot keep up with the dynamics of the wiki. If you find something to be corrected or improved, you need to check or edit the actual current wiki-page.

The current version of this document can be found at http://linuxbasics.org/course/book/index.

This book is licensed and may be distributed under the terms of the GNU Free Documentation License. A copy of the license can be found at http://linuxbasics.org/course/book/allof_appendix

Usually, these snapshots are updated daily between 05:00 and 05:15 UTC

This snapshot was created: Sun Jul 20 06:08:07 CEST 2008

• 9.1. Introduction
• 9.2. Moving your data to a backup device
  • 9.2.1. Making a copy on a floppy disk
  • 9.2.2. Making a copy with a CD-writer
  • 9.2.3. Backups on/from jazz drives, USB devices and such
  • 9.2.4. Backing up data using a tape device
  • 9.2.5. Tools from your distribution
• 9.3. Using rsync
  • 9.3.1. Introduction
  • 9.3.2. An example - rsync to a USB storage device
• 9.4. Encryption
  • 9.4.1. General remarks
  • 9.4.2. Generate a key
  • 9.4.3. About your key
  • 9.4.4. Encrypt data
  • 9.4.5. Decrypting files
• 9.5. Summary
• 9.6. Exercises

Accidents will happen sooner or later. In this chapter, we’ll discuss how to get data to a safe place using other hosts, floppy disks, CD-ROMs and tapes. We will also discuss the most popular compressing and archiving commands.

Upon completion of this chapter, you will know how to:

• Make, query and unpack file archives
• Handle floppy disks and make a boot disk for your system
• Write CD-ROMs
• Make incremental backups
• Create Java archives
• Find documentation to use other backup devices and programs
• Encrypt your data

Prev: Exercises
Home
Next: Introduction

Although Linux is one of the safest operating systems in existence, and even if it is designed to keep on going, data can get lost. Data loss is most often the consequence of user errors, but occasionally a system fault, such as a power failure, is the cause, so it’s always a good idea to keep an extra copy of sensitive and/or important data.

In most cases, we will first collect all the data to back up in a single archive file, which we will compress later on. The process of archiving involves concatenating all listed files and taking out unnecessary blanks. In Linux, this is commonly done with the tar command. tar was originally designed to archive data on tapes, but it can also make archives, known as tarballs.

tar has many options, the most important ones are cited below:

• -v: verbose
• -t: test, shows content of a tarball
• -x: extract archive
• -c: create archive
• -f archivedevice: use archivedevice as source/destination for the tarball, the device defaults to the first tape device (usually /dev/st0 or something similar)
• -j: filter through bzip2, see Section 9.1.1.2

It is common to leave out the dash-prefix with tar options, as you can see from the examples below.

• Use GNU tar for compatibility
  • The archives made with a proprietary tar version on one system, may be incompatible with tar on another proprietary system. This may cause much headaches, such as if the archive needs to be recovered on a system that doesn’t exist anymore. Use the GNU tar version on all systems to prevent your system admin from bursting into tears. Linux always uses GNU tar. When working on other UNIX machines, enter tar --help to find out which version you are using. Contact your system admin if you don’t see the word GNU somewhere.

In the example below, an archive is created and unpacked.

gaby:~> ls images/
me+tux.jpg  nimf.jpg

gaby:~> tar cvf images-in-a-dir.tar images/
images/
images/nimf.jpg
images/me+tux.jpg

gaby:~> cd images

gaby:~/images> tar cvf images-without-a-dir.tar *.jpg
me+tux.jpg
nimf.jpg

gaby:~/images> cd ..

gaby:~> ls */*.tar
images/images-without-a-dir.tar

gaby:~> ls *.tar
images-in-a-dir.tar 

gaby:~> tar xvf images-in-a-dir.tar 
images/
images/nimf.jpg
images/me+tux.jpg

gaby:~> tar tvf images/images-without-dir.tar 
-rw-r--r-- gaby/gaby  42888 1999-06-30 20:52:25 me+tux.jpg
-rw-r--r-- gaby/gaby   7578 2000-01-26 12:58:46 nimf.jpg

gaby:~> tar xvf images/images-without-a-dir.tar 
me+tux.jpg
nimf.jpg

gaby:~> ls *.jpg
me+tux.jpg  nimf.jpg

This example also illustrates the difference between a tarred directory and a bunch of tarred files. It is advisable to only compress directories, so files don’t get spread all over when unpacking the tarball (which may be on another system, where you may not know which files were already there and which are the ones from the archive).

When a tape drive is connected to your machine and configured by your system administrator, the file names ending in .tar are replaced with the tape device name, for example:

tar cvf /dev/tape mail/

The directory mail and all the files it contains are compressed into a file that is written on the tape immediately. A content listing is displayed because we used the verbose option.

The tar tool supports the creation of incremental backups, using the -N option. With this option, you can specify a date, and tar will check modification time of all specified files against this date. If files are changed more recent than date, they will be included in the backup. The example below uses the timestamp on a previous archive as the date value. First, the initial archive is created and the timestamp on the initial backup file is shown. Then a new file is created, upon which we take a new backup, containing only this new file:

jimmy:~> tar cvpf /var/tmp/javaproggies.tar java/*.java
java/btw.java
java/error.java
java/hello.java
java/income2.java
java/income.java
java/inputdevice.java
java/input.java
java/master.java
java/method1.java
java/mood.java
java/moodywaitress.java
java/test3.java
java/TestOne.java
java/TestTwo.java
java/Vehicle.java

jimmy:~> ls -l /var/tmp/javaproggies.tar
-rw-rw-r-- 1 jimmy   jimmy   10240 Jan 21 11:58 /var/tmp/javaproggies.tar

jimmy:~> touch java/newprog.java

jimmy:~> tar -N /var/tmp/javaproggies.tar \
-cvp /var/tmp/incremental1-javaproggies.tar java/*.java 2> /dev/null
java/newprog.java

jimmy:~> cd /var/tmp/

jimmy:~> tar xvf incremental1-javaproggies.tar
java/newprog.java

Standard errors are redirected to /dev/null. If you don’t do this, tar will print a message for each unchanged file, telling you it won’t be dumped.

This way of working has the disadvantage that it looks at timestamps on files. Say that you download an archive into the directory containing your backups, and the archive contains files that have been created two years ago. When checking the timestamps of those files against the timestamp on the initial archive, the new files will actually seem old to tar, and will not be included in an incremental backup made using the -N option.

A better choice would be the -g option, which will create a list of files to backup. When making incremental backups, files are checked against this list. This is how it works:

jimmy:~> tar cvpf work-20030121.tar -g snapshot-20030121 work/
work/
work/file1
work/file2
work/file3

jimmy:~> file snapshot-20030121
snapshot-20030121: ASCII text

The next day, user jimmy works on file3 a bit more, and creates file4. At the end of the day, he makes a new backup:

jimmy:~> tar cvpf work-20030122.tar -g snapshot-20030121 work/
work/
work/file3
work/file4

These are some very simple examples, but you could also use this kind of command in a cronjob (see Section 4.4.4), which specifies for instance a snapshot file for the weekly backup and one for the daily backup. Snapshot files should be replaced when taking full backups, in that case.

More information can be found in the tar documentation.

• The real stuff
  • As you could probably notice, tar is OK when we are talking about a simple directory, a set of files that belongs together. There are tools that are easier to manage, however, when you want to archive entire partitions or disks or larger projects. We just explain about tar here because it is a very popular tool for distributing archives. It will happen quite often that you need to install a software that comes in a so-called compressed tar ball. See Section 9.3 for an easier way to perform regular backups.

Data, including tarballs, can be compressed using zip tools. The gzip command will add the suffix .gz to the file name and remove the original file.

jimmy:~> ls -la | grep tar
-rw-rw-r-- 1 jimmy  jimmy    61440 Jun  6 14:08 images-without-dir.tar

jimmy:~> gzip images-without-dir.tar 

jimmy:~> ls -la images-without-dir.tar.gz 
-rw-rw-r-- 1 jimmy  jimmy    50562 Jun  6 14:08 images-without-dir.tar.gz

Uncompress gzipped files with the -d option.

bzip2 works in a similar way, but uses an improved compression algorithm, thus creating smaller files. See the bzip2 info pages for more.

Linux software packages are often distributed in a gzipped tarball. The sensible thing to do after unpacking that kind of archives is find the README and read it. It will generally contain guidelines to installing the package.

The GNU tar command is aware of gzipped files. Use the command

tar zxvf file.tar.gz

for unzipping and untarring .tar.gz or .tgz files. Use

tar jxvf file.tar.bz2

for unpacking tar archives that were compressed with bzip2.

The GNU project provides us with the jar tool for creating Java archives. It is a Java application that combines multiple files into a single JAR archive file. While also being a general purpose archiving and compression tool, based on ZIP and the ZLIB compression format, jar was mainly designed to facilitate the packing of Java code, applets and/or applications in a single file. When combined in a single archive, the components of a Java application, can be downloaded much faster.

Unlike tar, jar compresses by default, independent from other tools - because it is basically the Java version of zip. In addition, it allows individual entries in an archive to be signed by the author, so that origins can be authenticated.

The syntax is almost identical as for the tar command, we refer to info jar for specific differences.

• tar, jar and symbolic links
  • One noteworthy feature not really mentioned in the standard documentation is that jar will follow symbolic links. Data to which these links are pointing will be included in the archive. The default in tar is to only backup the symbolic link, but this behavior can be changed using the -h to tar.

Saving copies of your data on another host is a simple but accurate way of making backups. See Chapter 10, Communications, for more information on scp, ftp and many more.

In the next section we’ll discuss local backup devices.

Prev: Fundamental Backup Techniques
Home
Next: Moving your data to a backup device

On most Linux systems, users have access to the floppy disk device. The name of the device may vary depending on the size and number of floppy drives, contact your system admin if you are unsure. On some systems, there will likely be a link /dev/floppy pointing to the right device, probably /dev/fd0 (the auto-detecting floppy device) or /dev/fd0H1440 (set for 1,44MB floppies).

fdformat is the low-level floppy disk formatting tool. It has the device name of the floppy disk as an option. fdformat will display an error when the floppy is write-protected.

emma:~> fdformat /dev/fd0H1440
Double-sided, 80 tracks, 18 sec/track. Total capacity 1440 kB.
Formatting ... done
Verifying ... done
emma:~>

The mformat command (from the mtools package) is used to create DOS-compatible floppies which can then be accessed using the mcopy, mdir and other m-commands.

Graphical tools are also available.

Figure 9-1. Floppy formatter

After the floppy is formatted, it can be mounted into the file system and accessed as a normal, be it small, directory, usually via the /mnt/floppy entry.

Should you need it, install the mkbootdisk utility, which makes a floppy from which the current system can boot.

The dd command can be used to put data on a disk, or get it off again, depending on the given input and output devices. An example:

gaby:~> dd if=images-without-dir.tar.gz of=/dev/fd0H1440
98+1 records in
98+1 records out

gaby~> dd if=/dev/fd0H1440 of=/var/tmp/images.tar.gz
2880+0 records in
2880+0 records out

gaby:~> ls /var/tmp/images*
/var/tmp/images.tar.gz

Note that the dumping is done on an unmounted device. Floppies created using this method will not be mountable in the file system, but it is of course the way to go for creating boot or rescue disks. For more information on the possibilities of dd, read the man pages.

This tool is part of the GNU coreutils package.

• Dumping disks
  • The dd command can also be used to make a raw dump of an entire hard disk.

On some systems users are allowed to use the CD-writer device. Your data will need to be formatted first. Use the mkisofs command to do this in the directory containing the files you want to backup. Check with df that enough disk space is available, because a new file about the same size as the entire current directory will be created:

[rose@blob recordables] df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/hde5              19G   15G  3.2G  82% /home

[rose@blob recordables] du -h -s .
325M    .

[rose@blob recordables] mkisofs -J -r -o cd.iso .
<--snip-->
making a lot of conversions
<--/snip-->
98.95% done, estimate finish Fri Apr  5 13:54:25 2002
Total translation table size: 0
Total rockridge attributes bytes: 35971
Total directory bytes: 94208
Path table size(bytes): 452
Max brk space used 37e84
166768 extents written (325 Mb)

The -J and -r options are used to make the CD-ROM mountable on different systems, see the man pages for more. After that, the CD can be created using the cdrecord tool with appropriate options:

[rose@blob recordables] cdrecord -dev 0,0,0 -speed=8 cd.iso
Cdrecord 1.10 (i686-pc-linux-gnu) (C) 1995-2001 Joerg Schilling
scsidev: '0,0,0'
scsibus: 0 target: 0 lun: 0
Linux sg driver version: 3.1.20
Using libscg version 'schily-0.5'
Device type    : Removable CD-ROM
Version        : 0
Response Format: 1
Vendor_info    : 'HP      '
Identification : 'CD-Writer+ 8100 '
Revision       : '1.0g'
Device seems to be: Generic mmc CD-RW.
Using generic SCSI-3/mmc CD-R driver (mmc_cdr).
Driver flags   : SWABAUDIO
Starting to write CD/DVD at speed 4 in write mode for single session.
Last chance to quit, starting real write in 0 seconds. 
Operation starts.

The time required for writing will depend on your CD-writer. Upon finishing the job, you will get a confirmation message:

Track 01: Total bytes read/written: 341540864/341540864 
          (166768 sectors).

There are some graphical tools available to make it easier on you. One of the popular ones is xcdroast, which is freely available from the X-CD-Roast web site and is included on most systems and in the GNU directory. Both the KDE and Gnome desktop managers have facilities to make your own CDs.

These devices are usually mounted into the file system. After the mount procedure, they are accessed as normal directories, so you can use the standard commands for manipulating files.

In the example below, images are copied from a USB camera to the hard disk:

robin:~> mount /mnt/camera

robin:~> mount | grep camera
/dev/sda1 on /mnt/camera type vfat (rw,nosuid,nodev)

If the camera is the only USB storage device that you ever connect to your system, this is safe. But keep in mind that USB devices are assigned entries in /dev as they are connected to the system. Thus, if you first connect a USB stick to your system, it will be on the /dev/sda entry, and if you connect your camera after that, it will be assigned to /dev/sdb - provided that you do not have any SCSI disks, which are also on /dev/sd*. On newer systems, since kernel 2.6, a hotplug system called HAL (Hardware Abstraction Layer) ensures that users don’t have to deal with this burden. If you want to check where your device is, type dmesg after inserting it.

You can now copy the files:

robin:~> cp -R /mnt/camera/* images/

robin:~> umount /mnt/camera

Likewise, a jazz drive may be mounted on /mnt/jazz.

Appropriate lines should be added in /etc/modules.conf and /etc/fstab to make this work. Refer to specific hardware HOWTOs for more information. On systems with a 2.6.x kernel or higher, you may also want to check the man pages for modprobe and modprobe.conf.

This is done using tar (see above). The mt tool is used for controlling the magnetic tape device, like /dev/st0. Entire books have been written about tape backup, therefore, refer to our reading-list in Appendix B for more information. Keep in mind that databases might need other backup procedures because of their architecture.

The appropriate backup commands are usually put in one of the cron directories in order to have them executed on a regular basis. In larger environments, the freely available Amanda backup suite or a commercial solution may be implemented to back up multiple machines. Working with tapes, however, is a system administration task beyond the scope of this document.

Most Linux distributions offer their own tools for making your life easy. A short overview:

• SuSE: YaST now includes expanded backup and restore modules.
• RedHat: the File Roller tool provides visual management of (compressed) archives. They seem to be in favour of the X-CD-Roast tool for moving backups to an external device.
• Mandrake: X-CD-Roast.
• Most distributions come with the BSD dump and restore utilities for making backups of ext2 and ext3 file systems. This tool can write to a variety of devices and literally dumps the file(s) or file system bit per bit onto the specified device. Like dd, this allows for backing up special file types such as the ones in /dev.

Prev: Introduction
Home
Next: Using rsync

The rsync program is a fast and flexible tool for remote backup. It is common on UNIX and UNIX-like systems, easy to configure and use in scripts. While the r in rsync stands for remote, you do not need to take this all too literally. Your remote device might just as well be a USB storage device or another partition on your hard disk, you do not need to have two separated machines.

As discussed in Section 3.1.2.3, we will first have to mount the device. This is done as root:

root@theserver# mkdir /mnt/usbstore

root@theserver# mount -t vfat /dev/sda1 /mnt/usbstore

• User-friendly
  More and more distributions give access to removable devices for non-privileged users and mount USB devices, CD-ROMs and such automatically.

Note that this guideline requires USB support to be installed on your system. See the USB Guide for help if this does not work. Check with dmesg that /dev/sda1 is indeed the device to mount.

Then you can start the actual backup, for instance of the /home/karl directory:

karl@theserver:~> rsync -avg /home/karl /mnt/usbstore

As usual, refer to the man pages for more.

Prev: Moving your data to a backup device
Home
Next: Encryption====== 9.4. Encryption ======

Encryption is synonym to secrecy. In the context of backups, encryption can be very useful, for instance if you need to leave your backed up data in a place where you can not control access, such as the server of your provider.

Apart from that, encryption can be applied to E-mails as well: normally, mail is not encrypted and it is often sent in the open over the netwerk or the Internet. If your message contains sensitive information, better encrypt it.

On Linux systems you will find GnuPG, the GNU Privacy Guard, which is a suite of programs that are compatible with the PGP (Pretty Good Privacy) tools that are commercially available.

In this guide we will only discuss the very simple usage of the encryption tools and show what you will need in order to generate an encryption key and use it to encrypt data for yourself, which you can then safely store in a public place. More advanced usage directions can be found in the man pages of the various commands.

Before you can start encrypting your data, you need to create a pair of keys. The pair consists of a private and a public key. You can send the public key to correspondents, who can use it to encrypt data for you, which you decrypt with your private key. You always keep the private key, never share it with somebody else, or they will be able to decrypt data that is only destined for you. Just to make sure that no accidents happen, the private key is protected with a password. The key pair is created using this command:

willy@ubuntu:~$ gpg --key-gen
gpg (GnuPG) 1.4.2.2; Copyright (C) 2005 Free Software Foundation, Inc.
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions.  See the file COPYING for details.

gpg: directory `/home/willy.gnupg' created
gpg: new configuration file `/home/willy/.gnupg/gpg.conf' created
gpg: WARNING: options in `/home/willy/.gnupg/gpg.conf' are not yet
 active during this run
gpg: keyring `/home/willy/.gnupg/secring.gpg' created
gpg: keyring `/home/willy/.gnupg/pubring.gpg' created
Please select what kind of key you want:
    (1) DSA and Elgamal (default)
    (2) DSA (sign only)
    (5) RSA (sign only)
Your selection? 1
DSA keypair will have 1024 bits.
ELG-E keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n month
      <n>y = key expires in n years
Key is valid for? (0) 0
Key does not expire at all
Is this correct? (y/N) y

You need a user ID to identify your key; the software constructs the
user ID from the Real Name, Comment and Email Address in this form:
    "Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"

Real name: Willy De Wandel
Email address: wdw@mvg.vl
Comment: Willem
You selected this USER-ID:
    "Willy De Wandel (Willem) <wdw@mvg.vl>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
You need a Passphrase to protect your secret key.
					 
Passphrase:

Now enter your password. This can be a phrase, the longer, the better, the only condition is that you should be able to remember it at all times. For verification, you need to enter the same phrase again.

Now the key pair is generated by a program that spawns random numbers and that is, among other factors, fed with the activity data of the system. So it is a good idea to start some programs now, to move the mouse cursor or to type some random characters in a terminal window. That way, the chances to generate a number that contains lots of different digits will be much bigger and the key will be more difficult to crack.

When your key has been created, you will get a message about the fingerprint. This is a sequence of 40 hexadecimal numbers, which is so long that it is very, very hard to generate the same key twice, on any computer. You can be rather sure that this is a unique sequence. The short form of this key consists of your name, followed by the last 8 hexadecimal numbers.

You can get information about your key as follows:

willy@ubuntu:~$ gpg --list-keys
/home/willy/.gnupg/pubring.gpg
------------------------------
pub     1024D/BF5C3DBB 2006-08-08
uid                    Willy De Wandel (Willem) <wdw@mvg.vl>
sub     4096g/A3449CF7 2006-08-08

The key ID of this key is “BF5C3DBB”. You can send your key ID and your name to a key server, so that other people can get this info about you and use it to encrypt data for you. Alternatively, you can send your public key directly to the people who need it. The public part of your key is the long series of numbers that you see when using the –export option to the gpg command:

gpg --export -a

However, as far is this guide is concerned, we assume that you only need your key in order to encrypt and decrypt data for yourself. Read the gpg man pages if you want to know more.

Now you can encrypt a “.tar” archive or a compressed archive, prior to saving it to a backup medium or transporting it to the backup server. Use the gpg command like this:

gpg -e -r part-of-uid archive

The -e option tells gpg to encrypt, the -r option indicates who to encrypt for. Keep in mind that only only the user name(s) following this -r option will be able to decrypt the data again. Part of the name is sufficient for identification. An example:

willy@ubuntu:~$ gpg -e -r Willy /var/tmp/home-willy-20060808.tar

Using the -d option, you can decrypt files that have been encrypted for you. The data will scroll over your screen, but an encrypted copy will remain on disk. So for file formats other than plain text, you will want to save the decrypted data, so that you can view them with the appropriate program. This is done using the -o option to the gpg command:

willy@ubuntu:~$ gpg -d -o /var/tmp/home-willy-decrypt.tar /var/tmp/home-willy-20060808.tar.gpg

You need a passphrase to unlock the secret key for
user: "Willy De Wandel (Willem) <wdw@mvg.vl>"
4096 ELG-E key, ID A3449CF7, created 2006-08-08 (main key ID BF5C3DBB)

gpg: encrypted with 4096-bit ELG-E key, ID A3449CF7, created 2006-08-08
        "Willy De Wandel (Willem) <wdw@mvg.vl>"

• No password = no data
  • If you can not remember your password, the data is lost. Not even the system administrator will be able to decrypt the data. That is why a copy of important keys is sometimes kept in a sealed vault in a bank.

Prev: Using rsync
Home
Next: Summary====== 9.5. Summary ======

Here’s a list of the commands involving file backup:

Table 9-1. Backup commands

Prev: Encryption
Home
Next: Exercises

• Make a backup copy of your home directory in /var/tmp using the tar command. Then further compress the file using gzip or bzip2. Make it a clean tarred file, one that doesn’t make a mess when unpacking.
• Format a floppy and put some files from your home directory on it. Switch floppies with another trainee and recover his/her floppy in your home directory.
• DOS format the floppy. Use the mtools to put and delete files on it.
• What happens to an unformatted floppy when you want to mount it into the file system?
• If you have any USB storage, try to put a file on it.
• Using rsync, make a copy of your home directory to another local or remote file system.

Prev: Summary
Home
Next: Networking