Backup Program for Knoppix HDD Installed System

[Cuddles]

First of all, I looked at KBackup - don't think this is going to do the job...

I want a backup that "comes standard" on the Live CD, so that, in the case of a system wide crash, I just need to pop the Live CD in, and perform a restore to a new hard drive, etc... to get back in business...

This is what I am looking for -=-
(1) Hopefully the "backup" program comes on the Live CD of Knoppix v3.3 (2.4.22-xfs)
(2) I want to purchase a second hard drive, possibly the same size, or larger, than my current "knoppix" installed drive, 40 gig -=- and "save" backup "files" to this new hard drive. That way, if a system crash happens, I can get another "main" hard drive, boot the Live CD, run the restore from the Live CD, and pointing the restore "files" from the secondary hard drive, used to store previous backup data.
(3) Would prefer to have some form of "compression" on the backup - that way a backup "file" of previous backups, are not just a "collection" of backed-up files on my system, but also take less space as well.

This backup/restore program can be something that just does a tar, or zip, or gz of all files, and stores them into a single "backup file", or it could be a "full-blown" backup/restore program. I am quite sure that I can set some kind of "scheduler" event to have whatever it is, run at specific times of the week, and at specific times of these days. [?]

Whatever this backup/restore program is, I would want to backup "everything", system, devices, drivers, etc... I am looking for a complete solution, not a couple of files one. Time is not critical - considering the backups would be done during a "off time" for the system, just as long as the backup process doesn't take more than 8 hours. Current backup would be around 5 gig of data, and may go as high as 10, on days in which my business is processing a few digital sound files... As I said, the backup should be a "image" of the system, not just of "user" areas. This is needed for faster "crash" recovery.

I have been running for awhile now, without a backup stratagy, and fear the longer I go without one, as in the case of my Windows experience, the longer you don't, the worse it gets, when something disasterous happens.

Thank you to anyone, and everyone, for any assistance on this issue,
Cuddles

[nvgringo]

have you tried partimage? It is included with Knoppix. I think it is intended to do what you want to do but I have not had much luck with it.

[Markus]

You could do it with tar. For instance to backup /etc to 2:nd hd:s 1:st partition do:
tar cjf /mnt/hdb1/etc-backup.tar.bz2 /etc
Make sure to test it, 'cause I haven't. I use RW:s.

[Dave_Bechtel]

--Simple backup script, small enough to fit on floppy/USB thumbdrive: (this is for hd-installed, but you can adapt it. I've tested it, including full restore, and it works. The only problems I've had are having to recreate /proc and the /mnt/ dirs. Oh, and don't forget to re-do lilo. Having a "syslinux" floppy with your current kernel does come in handy in emergencies.)

BEGIN /root/bin/bkpsys

#!/bin/bash

#mount /dev/hda12 /mnt/bkps
mount /mnt/bkps
cd /mnt/bkps/local/p901/p901-knoppix-hdb1
ls -l

bkpfname='hdb1-knoppix-kernl2425--20040306-wvdial-acting-weird'

# Cmd not found?
#debsort
dpkg -l >~/DEBInstalled.list

# XXX BACKUP DEB CACHE BEFORE RUNNING $0!!
#apt-get clean

echo '==Updatedb...'
time updatedb

echo '==Backing up...'
time tar \
  --exclude=/mnt/* --exclude=/proc \
  --exclude=/cdrom --exclude=/floppy \
  --exclude=/.journal --exclude=/lost+found \
  -cvpf - / | \
   gzip -9cv >$bkpfname.tar.gz

ls -l

You could do it with tar. For instance to backup /etc to 2:nd hd:s 1:st partition do:
tar cjf /mnt/hdb1/etc-backup.tar.bz2 /etc
Make sure to test it, 'cause I haven't. I use RW:s.

[Cuddles]

Very interesting, Dave_Bechtel,

So, you exclude the following areas:
/mnt/*
/proc
/cdrom
/floppy
/.journal
and
/lost+found

Can understand excluding cdrom (though I would also have to include in this listing, cdrom1, cdaudio, and dvd - since all of these are mount points to my two cdrom devices)

Can understand lost+found, considering this would be the same as (in a Windows backup) including the Recycle Bin, or wastebasker

Can understand excluding floppy, since data stored on that would be nonsense to backup anyway

I can see why you wouldn't want to backup the .journal information ( this being the super-block area files, right? ( the ones used in a ext3 file system? ) )

I think I understand why saving the mnt location is not needed ( considering that the only thing I have in mnt is floppy mount point ) -=- but it also has the boot floppy information, wouldn't that need to be stored/restored to make a working system though???

Now, the tough one, I have seen many things in the proc location, and going by what I have seen, this area, I think, would need to be backed up, wouldn't it? If you want to get a complete "image" backup, I would think this area would need to be included in that, am I right???

Last thing, SysLinux ??? You mean the floppy that was created when I did the hdd install ??? ( I did this the first time I did a hdd install from the cd, but on the last one, I didn't ) If this is the thing you speak of, can I use the "previous" floppy, or can I "recreate" this floppy, by some command, or is SysLinux "a command" to build this bootable system floppy???

One more detail...
My (assumption) on this resore process...

(1) would want to run (from the boot floppy) cfdisk, to get the Linux and Linux Swap partitions on the drive.
(2) would want to run the "restore" process, to get what "backup" script was used to store my drive image from.
(3) would want to get lilo back on the drive. ( would this need to have a copy of my current lilo.conf file stored on the floppy, or would this already be in the "backup" script? )
(4) would want to run lilo on the drive, thus ensuring that that the MBR is updated to boot Linux.
(5) remove floppy used to boot the restore process with, and re-boot into new drive, and restored Linux.

Am I correct on this???
Cuddles

[Dave_Bechtel]

--Note: Lost+found dir is not akin to the Recycle Bin; it's where the ext2/ext3 fsck puts files that are "found goofy" during a filesystem check. Similar to FILE0001.DD after a "scandisk" in Windoze. For more info, see:
clicky

--Excluding /mnt -- I don't use automounters. They go against The Unix Way(TM) Besides that, excluding /mnt also excludes NFS / network mount points from the backup.

--The object of the script is only to backup the base filesystem. (Actually the script could be refined more if you have separate mountpoints for /usr, /var, et al - you could exclude those as well, and back them up simultaneously in the background.)

--AFA restoring; yes, I have to recreate all my /mnt dirs "by hand" before rebooting, but unless you have a *lot* of them (and are willing to pretty much take the system down to runlevel 1, and also unmount everything but your backup destination dir, and exclude THAT as well...) it's less of a pain to just exclude everything under /mnt.

--I have a little alias to make /mnt dirs with a NOTHERE file in them, wait for a keypress, and then mount the associated filesystem. When the filesystem isn't mounted, you see the NOTHERE file; and vice versa. It helps when creating new /mnt points; just define it in /etc/fstab 1st, save that, and use the alias to create the mountpoint and (optionally; you can always hit Ctrl-C) mount it:

BEGIN /root/bin/newmnt

#!/bin/sh

mkdir /mnt/$1
touch /mnt/$1/NOTHERE

ls -l /mnt/$1

echo PK to mount $1
read

mount /mnt/$1

--There *was* one nice feature of the "dump" backup command, before Linus decreed it unsafe -- dump would backup all the directories and files under /mnt, including the NOTHERE files - but *not* the mounted filesystems attached to the directories. It was very handy. However, Linus eventually deprecated the "dump" command, and besides that there has never been a "dump" available for Reiserfs.

--Regarding /proc - yes, you need the directory but NOT the files in it (at least for backup/restore purposes.) They are all created at boot time, or dynamically when drivers/modules are loaded. You can find more info about /proc here, assuming you have the kernel sources installed:
/usr/src/linux/Documentation/filesystems/proc.txt

(Actually, you used to be able to get away with not having a /proc at all; but I dunno / wouldn't want to try it now.)

--Syslinux; see ' man syslinux '. Basically a LILO replacement that loads the kernel from floppy disk; slower, but sometimes preferable. (And of course you could always burn a syslinux floppy image to CDR/CDRW.) I always format Syslinux floppies with the ' superformat ' command 1st.

--You might want to keep the original Knoppix syslinux bootfloppy around, but yes you can re-use it to boot your current hdinstall if you want (that's what I did.) You can always make another Knoppix bootfloppy from the CD image file.

--Note: If you use syslinux to boot your system, you need to re-copy the resulting kernel file to the floppy whenever you change / recompile your kernel. Filesystem naming convention, AFAICTell, follows the 8.3 rule so you should keep that in mind. Personally, I use a descriptive name like "krnl2425" instead of "vmlinuz" and update the syslinux.cfg file accordingly.

--Restoring... For #3, this is where having a syslinux floppy comes in handy. Use it to boot your restored system, and then run lilo.

Very interesting, Dave_Bechtel,

So, you exclude the following areas:
/mnt/*
/proc
/cdrom
/floppy
/.journal
and
/lost+found

Can understand excluding cdrom (though I would also have to include in this listing, cdrom1, cdaudio, and dvd - since all of these are mount points to my two cdrom devices)

Can understand lost+found, considering this would be the same as (in a Windows backup) including the Recycle Bin, or wastebasker

Can understand excluding floppy, since data stored on that would be nonsense to backup anyway

I can see why you wouldn't want to backup the .journal information ( this being the super-block area files, right? ( the ones used in a ext3 file system? ) )

I think I understand why saving the mnt location is not needed ( considering that the only thing I have in mnt is floppy mount point ) -=- but it also has the boot floppy information, wouldn't that need to be stored/restored to make a working system though???

Now, the tough one, I have seen many things in the proc location, and going by what I have seen, this area, I think, would need to be backed up, wouldn't it? If you want to get a complete "image" backup, I would think this area would need to be included in that, am I right???

Last thing, SysLinux ??? You mean the floppy that was created when I did the hdd install ??? ( I did this the first time I did a hdd install from the cd, but on the last one, I didn't ) If this is the thing you speak of, can I use the "previous" floppy, or can I "recreate" this floppy, by some command, or is SysLinux "a command" to build this bootable system floppy???

One more detail...
My (assumption) on this resore process...

(1) would want to run (from the boot floppy) cfdisk, to get the Linux and Linux Swap partitions on the drive.
(2) would want to run the "restore" process, to get what "backup" script was used to store my drive image from.
(3) would want to get lilo back on the drive. ( would this need to have a copy of my current lilo.conf file stored on the floppy, or would this already be in the "backup" script? )
(4) would want to run lilo on the drive, thus ensuring that that the MBR is updated to boot Linux.
(5) remove floppy used to boot the restore process with, and re-boot into new drive, and restored Linux.

Am I correct on this???
Cuddles

[Cuddles]

Ok, lets try another approach to this issue...

Since I am uncomfortable at the thought of doing a "image backup" of my Knoppix 3.3 (v2.4.22-xfs) system, and the solutions, so far, do not appear to be getting "everything" needed to get the system back without having to do "manual" rebuilding... Lets try something else...

Anyone
know of a "backup" package that can use - say - a floppy boot disk to get the system up to a backup/restore process - which would be installed on that floppy, and be able to access a "backup" destination hard drive, to perform a complete "image" restore of a failed hard drive...

I have a 41gig hard drive, being used as my "system" drive, 40.5gig for the system/file structure, and 500meg for the swap - and a 80gig hard drive - being used as the "backup storage destination, 78gig setup as Linux Ext3, and 2gig setup for Win98 (for Wine and DOS usage)

My "main" Linux system drive (hda1) is in Ext3...

What - I guess - I want now is - a backup package/program that can be booted from a floppy, and also run off the hda1 system drive - backup a complete "image" of the system to "files" on the destination device (hdc2 - Linux Ext3) - and if, God forbid, a system crash, the boot floppy will be able to gain access to the hdc2 device, perform a restore "completely" to the device hda1...

Does anyone use a "nice" backup program that may be used this way as well? I feel rather naked without knowing I don't have a backup strategy in place - even though I just went through a complete "gutting" and "re-install" to my hard drive to get ALSA working - I would PREFER not to do THAT again...

Thanks in advance,
Cuddles

[Dave_Bechtel]

--I think you're missing the forest for the trees here... Knoppix CD has lots of software on it, no need for floppy-based backup/boot disk.

--My suggestion:
o Use ' partimage ' to backup your install, compressed, to the 2nd HD. (Yes, this is an image backup, and best done from the CD-boot. If you resize the backed-up partition, it prolly won't be able to restore; something to be aware of.)

--In addition to that, back up "critical files" on a more frequent basis. This way you can restore file-by-file if you need to. I use a 100MB Zip disk for mine, you can use an existing directory on 2nd HD and then burn to CDR if you want. Here is my bkcrit script (always run as root, don't forget to chmod +x it)

BEGIN bkpcrit

#!/bin/sh
# Backup critical files (hopefully)

dest='/mnt/zippy'
mount $dest

##[ -e "$dest/NOTHERE" ] && echo "$dest NOT MOUNTED" && exit 99

# "If" checking for NOTHERE file...

dest=$dest'/bkpcrit-linux-p233-knoppix-hde3'
echo $dest = PK
read

mkdir $dest

# Copy this bkp script to zippy
cp $0 $dest
cp ~/localinfo.dat $dest

# Bkp symlinks
bkpsymlinks &
wtonit=$!

dpkg --get-selections > ~/dpkg-selections.txt
## To get your currently installed packages. Once you re-install then:
# dpkg --set-selections < dpkg-selections.txt
## To restore the saved selections to the package system.
#
## Then
# dselect update
## To update the database and
# apt-get dselect-upgrade
## To perform an upgrade using the the restored packages list.

time {
tar cpvzf $dest/bkp-ETC-debian.tar.gz /etc
tar cpvzf $dest/bkp-DEV-debian.tar.gz /dev
tar cpvzf $dest/bkp-rootbin-debian.tar.gz /root
tar cpvzf $dest/bkp-davesrc-debian.tar.gz /home/dave/src
tar cpvzf $dest/bkp-davebin-debian.tar.gz /home/dave/bin
tar cpvzf $dest/bkp-usr-local-debian.tar.gz /usr/local
tar cpvzf $dest/bkp-var-dpkg-status.tar.gz /var/lib/dpkg
tar cpvzf $dest/bkp-var-dpkg-backups.tar.gz /var/backups
# Dotfiles
cd /root
tar cpvzf $dest/bkp-root-dotfiles--restore-locally.tar.gz .[^.]*
cd /home/dave
tar cpvzf $dest/bkp-dave-dotfiles--restore-locally.tar.gz .[^.]*
}

echo 'Waiting for symlinks to finish ('$wtonit')...'
wait $wtonit;
cp /SYMLINKS.tar.gz $dest

ls $dest -al
df $dest
exit

--And the companion script...

BEGIN bkpsymlinks

#!/bin/bash
# ===== Find all symlinks and keep a list for later

# This should be root so it can run while bkpcrit runs
Bkpdir='/'

echo ' *** Finding all symlinks... '
#find / -type 'l' -name '*' -exec /bin/ls -l "{}" ";" > /Allsymlinks.list

# Methodology for this taken from ' info tar ' somewhere
# -mount = Don't descend into other filesystems
# -print0 seps with NULL char instead of LF

# For some reason, this not work in ram...
time find / -type 'l' -name '*' -mount -print0 > $Bkpdir/Rootsymlinks.NULLlist &
time find /usr -type 'l' -name '*' -mount -print0 > $Bkpdir/Usrsymlinks.NULLlist &
wait;

cat $Bkpdir/Rootsymlinks.NULLlist $Bkpdir/Usrsymlinks.NULLlist \
 >$Bkpdir/Allsymlinks.NULLlist
##echo "$rootsyms" "$usrsyms" > /Allsymlinks.NULLlist

echo ' *** Backing up all symlinks *** '
time tar -cvz --null \
 --files-from=$Bkpdir/Allsymlinks.NULLlist \
 --file=/SYMLINKS.tar.gz

## --file=$Bkpdir/SYMLINKS.tar.gz

rm $Bkpdir/Rootsymlinks.NULLlist
rm $Bkpdir/Usrsymlinks.NULLlist
rm $Bkpdir/Allsymlinks.NULLlist

Ok, lets try another approach to this issue...

Since I am uncomfortable at the thought of doing a "image backup" of my Knoppix 3.3 (v2.4.22-xfs) system, and the solutions, so far, do not appear to be getting "everything" needed to get the system back without having to do "manual" rebuilding... Lets try something else...

Anyone
know of a "backup" package that can use - say - a floppy boot disk to get the system up to a backup/restore process - which would be installed on that floppy, and be able to access a "backup" destination hard drive, to perform a complete "image" restore of a failed hard drive...

I have a 41gig hard drive, being used as my "system" drive, 40.5gig for the system/file structure, and 500meg for the swap - and a 80gig hard drive - being used as the "backup storage destination, 78gig setup as Linux Ext3, and 2gig setup for Win98 (for Wine and DOS usage)

My "main" Linux system drive (hda1) is in Ext3...

What - I guess - I want now is - a backup package/program that can be booted from a floppy, and also run off the hda1 system drive - backup a complete "image" of the system to "files" on the destination device (hdc2 - Linux Ext3) - and if, God forbid, a system crash, the boot floppy will be able to gain access to the hdc2 device, perform a restore "completely" to the device hda1...

Does anyone use a "nice" backup program that may be used this way as well? I feel rather naked without knowing I don't have a backup strategy in place - even though I just went through a complete "gutting" and "re-install" to my hard drive to get ALSA working - I would PREFER not to do THAT again...

Thanks in advance,
Cuddles

[turbine]

Yo, Cuddles:

I use partition commander to do what you are asking for. Here's the benefits:

1. You can either partition your existing hard drive in half, using one of the halves as the backup, or, as in your case, use the entire spare disk as a backup.

2. Backups and restores are ultra simple. You just copy the entire partition.

3. The program runs off of it's own boot diskette and is thus independent of Linux, Windows, or any other os.

4. It's relatively cheap. I paid $30 for mine, version 6. I think they're up to version 8 now for about $42.

[firebyrd10]

Yo, Cuddles:

I use partition commander to do what you are asking for. Here's the benefits:

1. You can either partition your existing hard drive in half, using one of the halves as the backup, or, as in your case, use the entire spare disk as a backup.

2. Backups and restores are ultra simple. You just copy the entire partition.

3. The program runs off of it's own boot diskette and is thus independent of Linux, Windows, or any other os.

4. It's relatively cheap. I paid $30 for mine, version 6. I think they're up to version 8 now for about $42.

Unless it lets you broswe the backuped image for perticular files, then its a waste of money as partimage does the same thing, minus the boot floppy (but you can use knoppix to boot and restore)

As for the wasted space, yes if the partition your restoring to is bigger then you won't be able to use the space. But you can use ext2resizer http://ext2resize.sourceforge.net/ to fix that. (works with ext3 also)

As said before, making an image every month or so and backuping important files every other week is pretty safe bet.