copying from ntfs to ext hdd fat32 - stalled

[dav2900]

Using Knoppix 4.0.2, I boot into my PC and correctly see my internal RAID drive (NTFS - from a crashed XP install) and I can see my data.
I have an external firewire 300 Gb HDD, preformatted as FAT32.
I changed my FAT32 to be writable.
When I copy files from NTFS to FAT32 using Konqueror, it begins the process but a few weird things happen:
1. status changes from transfer speed (i.e. 30 Mb/s) to "stalled." Sometimes the copy continues; other times it fails.
2. if the copy manages to complete, the status of my FAT32 drive often changes not allowing me to write anything further to it. I try unmount, mount, etc...but it doesn't work until I completely reboot again and start over.

I have about 240 Gb of data to retrieve and FAT32 is fine since I need a Windows install to see the files in the future (after I rebuild my PC).

I've also delete the partition using FDISK and recreated it using knoppix as vfat. That worked OK but didn't change my above results.

Any other ideas??
:?

[Harry Kuhman]

1- I'm not sure how you are seeing the RAID drive properly in Linux. Almost all RAID drive currently available require special RAID software drivers to be installed under Windows. If you need these drivers under Windows and Linux does not have RAID drivers that prefectly match the WIndows RAID geometry then you will get strange results (and you seem to be getting strange results).

2- Some files that can exist on a NTFS system can not exist on a FAT32 system. Are there any files 4 gig or larger on your system? If so they cannot be moved to a FAT32 system as it simply does not support files 4 gig or larger.

[dav2900]

Hmm.. Knoppix seems to see it out of the box but in a flakey manner, as you said.
Other boot CDs like Phlak also see my RAID setup. I just assumed they included various drivers...
How can I tell Knoppix to use a driver that I have (from a floppy), for example?

The files all conform to NTFS & FAT32 requirements: size, naming, etc. so that is not the problem.

[Harry Kuhman]

This is purely speculation, but I expect that the live Linux systems you have tried are just seeing the first disk of the RAID array and see enough of a disk structure there that they think they see a partition and files in it, but since the RAID geometry differes from the actual first hard disk that they are not copying files correctly (and will not try to access information beyond that first disk).

As to the drivers, I don't know. I'm not sure if it can be done at all, and if these are Windows drivers and not drivers written for Linux then I would not expect it could be made to work. Even if there are Linux drivers for the RAID controler I would want to find out if they deal with the drive geometry in the same way the Windows drivers do (they very well could, but it is dangerous to just assume that they do).

[ckamin]

I agree with Harry's speculation, that Linux is probably seeing just the first disk in the raid array, depending on how it's configured. Not knowing what kind of raid setup you have, striped, mirrored, Raid 0, Raid 1, etc., I will assume the worst that you have only two drives with no redundancy. Linux is probably not the best tool for what you need to do. I would suggest looking at "BartPE" and seeing if you can load your Raid Drivers under Windows using the BartPE disk to salvage your files. One other alternative would be to get Windows working again. There are many Linux "rescue" , "forensic", or "recovery" disks out there, but I am unaware of any that can effectively deal with a Windows NTFS raid array. That does not mean that one does not exist, just that I have not seen one that could.

[dav2900]

Thanks for your feedback, guys. Your theory seems to make sense, although that is not good new for me. <g>
I am using LSI Logic's Megaraid 150-4 controller card. I have 4 SATA 200 Gb drives in a RAID-5 config.
LSI has Linux drivers; I could call LSI to confirm no risks in loading them but I would still need to know HOW to load them using knoppix.
I tried BartPE with UBCD and at first, it loaded the RAID fine but then stopped working after 1 day and I haven't gotten it to load it properly since.
I also installed XP on another internal 80 Gb IDE drive and that install went fine until I loaded the RAID card driver. After that, the XP install wouldn't load properly anymore!
Knoppix is the only bootable CD i have currently that continues to at least see my RAID drive, although in a wacky way.
This is crazy. Sometimes I think the safety of RAID-5 isn't worth all the problems in reading the drive in these types of situations; it adds another level of complexity.
Maybe I'll post on UBCD forums and try to troubleshoot there unless someone else can offer a suggestion here.

[Harry Kuhman]

Sometimes I think the safety of RAID-5 isn't worth all the problems in reading the drive in these types of situations; it adds another level of complexity.
I agree with you there. Raid-5 is good for dealing with an actual drive failure (which does happen). And if it isn't adding too much software overhead then it can help improve disk performance. But given Windows XP's pretty bad track record of having disks that are suddenly unreadable by XP (I had once happen myself and I saw a friend's system do this just this week, although Knoppix was able to read much but not all of the data and copy it to a USB flash drive, in addition to the many times I see reports of the same thing happening here), you only invite additional problems by running an disk system that depends on special Windows drivers.

RAID is not a replacement for regular and complete backups. Hardware failures can and have taken out multiple drives. Virusus and other malware can destroy the information on a RAID array just as well as they can on a regular hard drive. And XP may well be the unsafest OS of any for hard disks, at least until Vista comes out.

[dav2900]

Just thought I'd post an update....
IMO, I dont think your theory is correct since I have been able to copy over 100 Gb from RAID to my ext. HDD. Also, any file in the RAID set is striped across all drives so if I could only see one drive, it would've never let me copy a file over (provided it's greater than the block size I have set, which is 64k, I believe).
Anyway, it turns out that on two of the four HDDs, I have found media errors, as reported by the RAID controller card.
That may explain why some copy jobs fail while others are OK.
I'll call LSI tomorrow to see what they suggest to 'fix' the media errors besides possibly having to replace the drives.

[Harry Kuhman]

Thanks for the update. I'm glad to hear that things seem to be working, but you will be the first person that I know of who came to these forums with a RAID problem and then managed to get his files back with Knoppix. Maybe you have the rare hardware RAID conroler that does not cut corners and depend on Windows drivers but rather does all of the raid function in firmware on the controler and presents the RAID array to the computer as one nice big hard drive. If that turns out to be the case please be sure to let us know about it.

But are you sure that what you are copying is really a good file? Just a file of the right size with the first part of the contents correct is not a valid test. I would suggest that you check a large file on the backup to be sure that it is good. And don't go by that "64 K 'block' size" you mentioned; the raid array should not move on to the next disk until a track is full on the first disk, not a block or cluster. I could even imagine it staying on one drive for a full cylinder, particularly since the "track" is not really a full track in modern drives. So check one or more very large files to be sure that either their contents are the same as you know they should be by an md5 checksum (such as if you had a Knoppix.iso file out there on the RAID array) or that all of the data is good (if you have a movie or hour long TV show, watch it to be sure the data is not corrupt many minutes into the show).

[dav2900]

I will check with LSI and let you know what they say.

I have opened many files successfully in my other XP installation - MS Office docs, MPG, AVIs, JPGs. All the way up to 4 Gb AVIs...

I have managed to isolate a few directories where I am unable to copy from - that must be where the media errors lie. Not too many considering I had ~ 240 Gb of files recovered (copied).

[Harry Kuhman]

I have opened many files successfully in my other XP installation - MS Office docs, MPG, AVIs, JPGs. All the way up to 4 Gb AVIs...
Most doc or jpg files are too small to be a valid test. The MPG or AVI would be a good test, but just because you can open and view the first part of a mpg does not mean the rest is good, it would be far better to watch it or skip to the end of the MPG and see if it really is the correct end of the MPG. Good luck.

[dav2900]

LSI said the hardware talks to the OS through the driver so Knoppix must be loading the correct Linux driver, which LSI offers. So far, I've copied ~240 Gb of data and identified some dir's I cant copy because of the media errors.
I'll run a consistency check soon which may be able to move the data to another location on the drive but this process may take 1-2 days to complete. Once I get as much as I can, I have to run Maxtor's diag utility to get the error code to apply for a replacement but to do that, I'll need to buy another SATA card since the LSI doesn't allow the Maxtor utility to see each drive individually. Could easily be another 1-2 weeks before I have everything done to BEGIN the rebuilding process...sheesh!

Anyway, in case anyone is interested, I'll post results as I go along.