
Last update 1999-03-03

Datbkr Tape Backup system

Dave Cinege <dcinege@psychosis.com>
----------------------------------------------------------------------------

FTP site
----------------------------------------------------------------------------
News:
Datbkr version 1999030301 released. Fixes a minor bug in the host name
check. Changed some bash centric things to make the script more portable.

Datbkr version 1998100401 released. Fixes a minor bug in the log file
timestamp. Was using %m(month) instead of %M(minute). Opps!

Grab the new version of tarblocks.
#1 I put the wrong source file here...it had a display bug. : P
#2 Special files (/dev/) were screwing up the offsets. They stor major,minor
where the size normally is. Now I only check size for regular files.
----------------------------------------------------------------------------

FAQ

What is Datbkr?
Datbkr is the end result of the evolution of some scripts and programs
that I created over several months to handle my back-up needs.

What makes Datbkr special?
Datbkr is very light weigth, and simple to use. Besides this it is
designed to provide seemless backup's over an encrypted ssh (secure shell)
link.

What does it work with?
Datbkr itself is comprised of a multicall, tar based, sh script, and a
small program called tarblocks. It was developed on Debian Linux 2.0
w/bash 2.0, but should work with most any unix with GNU utils and sh shell.

What tape drives does it work with?
It has been designed to work with *partitionable* dat drives.
The partition is used to keep the tape index and back-up log
report. You also must have an mt command that can handle this (mt-st)

How does it work?
It is all very basic..check for a tape, ask to proceed, ssh into dat server,
etc etc. The script is very well done and seemless, with command line
parameters to override defaults. Right now it is made to keep only one
archive per tape.

Do I need this?
It works very well for simple large filesytem backups on either your local
or remote machines. It is geared towards one shot server backup's, not
incremental or multivolume backups.

Why do I want this over my own tar script?
It shows progress in MB written. It can estimate archive size.
It creates nice backup reports that are mailed to you. It is a sh script
that can be customized as needed. It creates an output file with block
listings for quick single file retrieval. Setting up seemless ssh backups
with logging that can be cron'ed is a real motherfucker.

----------------------------------------------------------------------------

Installation and set up:
Download the following -
        datbkr
        mt-st
        tarblocks

compile mt-st           (see readme)
compile tarblocks       (gcc -o tarblocks tarblocks.c; strip tarblocks)

Copy datbkr, mt-st, and tarblocks to /usr/local/bin/
Symlink datbkr as datbkr-mt and datbkr-rshtar
(ln -s datbkr datbkr-mt; ln -s datbkr datbkr-rshtar)

Edit datbkr. There is a user configuation section at the top of the script.
You can cut and paste this section into the file /etc/datbkr.conf if
you don't want to edit the script directly. (recommended)
This file will be sourced by the script and override any defaults.

Make sure the hostname is set correctly, or else datbkr will assume
you want to access a remote dat server and fail.

Testing
This part expects you already can acess your dat, and you have
a blank tape inserted.

First see if mt-st is working OK for you:
        # /usr/local/bin/mt-st -f /dev/tape tell
        At block 0.

Now try the datbkr wrapper:
        # datbkr-mt tell, status, offline
        At block 0.
        [status info]
        [tape ejects]

Insert the tape again. Now the moment of truth:
        # datbkr-mt mkpart 10, status, setpart 1, status

This should create a 10MB partition, print status, go to partition 1,
print status. The status reports show the current partition. Make sure
the second status says partition=1.

If this does not work for you I don't know what to tell you.  : P

Backing up
If all is working well let's try your first back-up:
        # datbkr -e -s -d /tmp
        Tape check: OK

        All data will be erased on this tape!
        Proceed? (y/N) y

        Check for tape partitions: OK
        Erase index partition: Done
        Erase data partition: Done
        Changing directory to: /tmp
        Estimating size: 7.8M
        Backing up: *
        Written:      8701K,              8910336 bytes
        Back up complete.
        Change to index partition: Compress index: Write index to tape: Done
        Ejecting tape: Done

        All Operations Finished.

Wasn't that nice? Don't forget to look in your mail box for that nice
backup report!

Restoring data
Reinsert the tape....let's make sure all the data got on there:
        # cd /tmp; mkdir tmp1; cd tmp1
        # datbkr-mt setpart 1
        # tar xvf /dev/tape
        tape.log
        tape.dir.gz

Now let's dump the backup:
        # datbkr-mt setpart 0
        # tar xvf /dev/tape
        [file listing]

If it does not dump or you get tape.log and tape.dir.gz again, the script
had a problem changing the partitions on your tape. email me.
Otherwise...that's it

Let's take a look at the top of your tape.dir:
        rec          0: drwx------ root/root         0 1998-08-29 10:47 kfm-cache-0/
        rec          1: prw------- root/root         0 1998-08-29 13:47 kfm_0_229_0.0
        rec          2: prw------- root/root         0 1998-08-29 13:47 kio_0_229_0.0
        rec          3: -rw------- root/root   5001065 1998-06-28 21:13 tape.dir
        rec       9772: -rw------- root/root      1012 1998-08-28 21:15 tape.log
        rec       9775: -rw------- root/root       559 1998-06-28 21:13 tape.status
        rec       9778: drwxr-x--- root/root         0 1998-08-29 13:09 usr1/
        ....

Notice the record sizes are 512 byte (tar) blocks. These can be used to easily
do QFA (quick file access) restores. You do this by:
        # datbkr-mt seek [ RECORD / (TAPEBLKSIZE / 512) ]

By default datbkr uses a 512 byte tape block size. This is 1:1 with the tar
records. IE to start extracting with 'tape.status':
        # datbkr-mt seek 9775
        # tar xvf /dev/tape

If you do alot of incremental restores, the above can be automated with
some light scripting.

Usage
You can toggle various options on the command line.
        # datbkr -h
        datbkr 1998082901   call as: datbkr, datbkr-mt or datbkr-rshtar

        Usage: datbkr [OPTION]

          -d CDTODIR    change dir to    (defaults ./)
          -i INCLUDE    tar include list (defaults *)
          -b BLKSIZE    tape block size
          -f TAPEDEV    tape device
          -p            don't prompt to continue
          -s            estimate size and print
          -e            autoeject tape
          -m            autopartition tape
          -v            verbose console output
          -q            suppress console output

          -h            display this help and exit

        Usage: datbkr-mt [-b BLKSIZE] [-f TAPEDEV] mt-arg[, mt-arg2, ...]

        Usage: datbkr-rshtar -- internal use

What about remote access via ssh?
The basic set up is the same on the remote machines.
The only changes you need to make are on your dat server.

NOTE:owner/group and permissions are critcal in making this work
correctly and secure. Be thorough!

First make sure the tape /dev files are correct:
        crw-rw----   1 root     tape       9, 128 May 27  1997 nst0
        lrwxrwxrwx   1 root     tape            4 May 26 17:48 tape -> nst0
The owner group and permissions are important!
Now create a user called dat (see RSHUSER setting in config file)

/etc/passwd:
        dat:x:200:26::/home/dat:/bin/sh
/etc/shadow:
        dat:*:10405:0:99999:7:::

dat's default group should be tape (on debian machines this is gid 26)

Create dat's home directory and set permissions:
        drwxr-s---   3 dat      tape         1024 Aug 29 01:34 dat/
Create a .ssh dir in /home/dat/:
        drwx------   2 dat      tape         1024 Jun 28 10:30 .ssh
In .ssh create an authorized keys file:
        -rw-------   1 dat      tape          665 Jun 28 13:13 authorized_keys

In this file you place the *public* root ssh keys of all the remote machines
you want to be able to access the dat server. Those key's must NOT
have a passphrase or else datbkr will hang waiting for a password.

Finally change the permissions on the dat servers copy of datbkr:
        -rwxr-s---   1 root     tape         7148 Aug 29 10:53 datbkr

You should be all set! Run datbkr from the remote machine just like you did
on the dat server.

Make sure to set your host and sshuser information correctly in the conf file.
Also make sure to make 1 initial connect to the dat server with ssh. If you
don't the script will hang waiting for an answer to:
        Host key not found from the list of known hosts.
        Are you sure you want to continue connecting (yes/no)?

----------------------------------------------------------------------------
Yet another miraculous page by me that defies the boundries of HTML,
graphics, and grammar.
