Backup System w/RAID

[davewmerrill]

I have the follwing idea:

- take a desktop
- install 4 drives of the same size
- boot knoppix
- configure the 4 drives into a RAID array (say via software raid)
- use the resulting RAID array as a backup device

Has anyone seen any HOW-TO's online discussing such a thing (creating the RAID array) and is there an easy way to boot knoppix with a cd where it automatically mounts the RAID array?

[Harry Kuhman]

There are software RAID drivers for Linux, raid 0, 1, 4 and 5 at least should be available. You can find a wealth of information about this on the internet. This (http://tldp.org/HOWTO/Software-RAID-HOWTO.html) is the first hit that a simple search turned up.

As far as I know, the RAID drivers are not included in Knoppix. I can't imagine any good reason for wanting to do this with Knoppix, or any live CD, over a real installed Liunx system anyway. I would suggest using a system like Debian to build such a server. It will end up being far more stable, flexible and expandable.

Using a system like this purely as a backup device would be a good system. In designing a backup system, one important thing that people often omit is making the backups extremely convenient. An important truth about backups is that they will not get made as often as needed (if at all) if the backup process is cumbersome, such as swapping multiple tapes. But it is important to note a few issues here. RAID provides good protection against a single hard drive failure. But it does not protect against user error, malware, or other hardware problems (such as power supply or case overheating if multiple drives are in the same case running off the same supply). For this reason it's good to have some for of second stage backup (perhaps even off-site backup depending on the value of the data).

Of course, Linux RAID support is not at all the same as being able to read a RAID array that was created by software drivers under Windows. From your question I expect that you already know this, I only mention it because we see a lot of lost souls come here trying to recover data from a RAID 0 array created under Windows that they have no backup of, and I would not want to give them the impression by my post that Knoppix or Linux will recover such an array.

[davewmerrill]

I agree with your issues on single points of failure (assuming this is some generic desktop w/2 IDE interfaces):

- 1 power supply (if it goes then you're done)
- drives A & B on ide-interface1 and drives C & D on ide-interface2 (either ide interface does down and your RAID array is done)

The reason I was thinking of doing this w/a live CD was that you could get an extra drive in there (4 data drives rather than 1 os drive and 3 data drives).

I was imagining this as a poor (yet wildly linux savy) man's option for a backup device. A POS desktop w/4 reasonably sized drives RAIDed together, boot up a live CD, mount the RAID array and allow rsync-via-ssh mojo to it (all by persistent settings).

Thanks for the URL. I've also found a few references to sections of O'Reilys book Knoppix Hacks that look interesting.

[davewmerrill]

Couple quick URLs I've found:

http://www.silug.org/lists/silug-discuss/200412/msg00062.html
http://www.millibit.de/plugscript/doc/knoppix_ataraid_howto.html
http://www.samhart.com/cgi-bin/classnotes/wiki.pl?UNIX02/Install_Knoppix_On_A_Software_RAID

[Harry Kuhman]

....... - 1 power supply (if it goes then you're done)...
There is a difference between the supply failing and loss of data. If it's only a backup system then the supply failing usually will not harm the drives, and you can replace the supply and hopefully have your backup system back. But for really mission critical systems I have built redundant drives each in their own case with their own power supply and multiple computers (SCSI rather than IDE for the external case support).


.......The reason I was thinking of doing this w/a live CD was that you could get an extra drive in there (4 data drives rather than 1 os drive and 3 data drives). ...
Well, if this is your basic old system with 2 IDE ports then you are not going to be able to have 4 IDE drives and an IDE CD drive. I don't see any need for keeping the OS and data on different disks, as long as you can boot into the raid system. If not, then an OS drive or at least an OS partition would be needed. If you have enough IDE connections for 4 hard drives and a CD drive (by my count that's 3), you could slave another small hard drive onto the IDE connection supporting the CD and use that for the OS and still have your data RAID.

[davewmerrill]

Well, if this is your basic old system with 2 IDE ports then you are not going to be able to have 4 IDE drives and an IDE CD drive.

Gah, good point. One could try booting from an external CD-ROM I guess...

[cbagger01]

Well, if this is your basic old system with 2 IDE ports then you are not going to be able to have 4 IDE drives and an IDE CD drive.

Gah, good point. One could try booting from an external CD-ROM I guess...

The easy solution is to boot KNOPPIX from a USB drive, aka a "thumbdrive". There are instruction on how to do this in the forums. You will need a minimum 1GB size drive because the KNOPPIX CD version is 699MB in size.

If your computer has USB 2.0 (high speed) ports, then performance will be very fast, comparible to a "poor man's install".

[Harry Kuhman]

The easy solution is to boot KNOPPIX from a USB drive, aka a "thumbdrive". There are instruction on how to do this in the forums. .....
Thumb drives can wear out from the kind of repeated access an OS like Knoppix puts on them. OK for an emergency recovery device to carry around, not advised for day to day use. And booting USB can be very problematic with any but the most recent computers. Another option that wasn't mentioned yet but would be much simpler and cheaper would to add an inexpensive PCI card with one or two more IDE ports (around ten bucks U.S. from Tiger and others) and then not worry about the 4 IDE device limitation. One could boot from the CD this way, but it would still be much better to build a file server like this with an installed OS such as Debian (on another hard drive outside of the RAID array) rather than with any Live CD.

[cbagger01]

The easy solution is to boot KNOPPIX from a USB drive, aka a "thumbdrive". There are instruction on how to do this in the forums. .....
Thumb drives can wear out from the kind of repeated access an OS like Knoppix puts on them. OK for an emergency recovery device to carry around, not advised for day to day use. And booting USB can be very problematic with any but the most recent computers. Another option that wasn't mentioned yet but would be much simpler and cheaper would to add an inexpensive PCI card with one or two more IDE ports (around ten bucks U.S. from Tiger and others) and then not worry about the 4 IDE device limitation. One could boot from the CD this way, but it would still be much better to build a file server like this with an installed OS such as Debian (on another hard drive outside of the RAID array) rather than with any Live CD.

Flash memory based thumbdrives can wear out from repeated writes such as to a swap partition and should not be used for this purpose.

However, in this situation the thumbdrive would be used like a virtual CDROM or poorman's install and would be used in the same way as a CDROM drive (read only). USB boot support gained popularity with the frequent elimination of the internal floppy drive from computers around 2001-2002 timeframe. It is also possible to perform a KNOPPIX USB boot on an older computer that does not support BIOS-based USB booting. The solution is to create a KNOPPIX boot floppy and then initial boot from floppy with switchover to livecd or USB thumbdrive or poormans install.

I agree that the "right" way to do things is to buy a server class machine with a supported OS installed such as a Dell Poweredge/Debian Linux solution for example.

But it is also possible to slap together a 4 disk IDE Linux Software RAID array fileserver from some parts sitting in the attic such as a PC, 4 hard drives, and a thumbdrive with a KNOPPIX USB install.

[davewmerrill]

Here's an initial recipe that I've been able to work out w/a test system of 3 IDE hard drives and 1 IDE CD-ROM:

Initial Setup (the first time you boot and setup the RAID array)

modprobe md
mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/hd[bcd]1
mdadm --query --detail /dev/md0 (watch the status of the array)
mkfs.ext3 -c -j /dev/md0
mkdir /mnt/raid5
mount -t ext3 /dev/md0 /mnt/raid5/

Subsequent Reboots
modprobe md
mdadm --assemble /dev/md0 /dev/hd[bcd]1
mdadm --query --detail /dev/md0 (check the status of the array)
mkdir /mnt/raid5
mount -t ext3 /dev/md0 /mnt/raid5/

I've been able to run an rsync to the array, shut it down, bring it back up and find the data still there.