Persistent Home, will my existing windows partition be safe?

[Garibaldi]

I wanted to be able to keep the changes that I make to knoppix. I am thinking about doing persistent home, but I want to know if it will corrupt or erase or do anything to damage or make my current ntsc windows xp home operating system unaccessible. Thanks.

[eco2geek]

Do you have a USB key or a partition formatted as FAT32 (or ext2 or ext3) to save it on?

If your hard drive partition(s) is (are) formatted as NTFS, you will not be able to save a persistent home on it (them), because Linux doesn't adequately support writing to NTFS yet.

Knoppix probably wouldn't let you mount a persistent home image from an NTFS partition even if you took the risk and saved it there.

If you don't have a FAT32 (or ext2, or ext3) formatted partition available, your best bet is to use a USB key (they're usually formatted as FAT or FAT32).

And as long as you don't write to NTFS, no, it won't do a darn thing to "harm" Windows.

[Garibaldi]

Here is my exact situation. I have a HP Pavilion desktop. It has a 160 GB harddrive, which has 2 partitions. The first one (C) is 144 GB and contains the windows os and all program files. The other partition (D) is The "HP Recovery" Partition. This is a 16 GB partition with 786 mb free. It contains a copy of the windows xp home OEM version of the os, incase you need to reinstall. The C Partition is NTSC, but the D is FAT32, could I put Knoppix with Poor Man's Install onto this partition, using the 786 mb free space? I'm not too concerned about keeping the hp recovery data (it would be nice though), because hp sent me a cd copy of it too. If this is possible, I have a couple questions. First, in the guide to performing a poor man's install, it says under What You Need To Know 2. that you need to make your disk writable and unmount it. I tried on my D partition and neither of these options are available to me. I did share it though. That is about the only option that they give you. Also, I have usb flashdrive/thumbdrive. I tried using it in Knoppix but it says something like "cannot mount." Its only 32 megs so its not the greatest storage device for an os. Thanks for all the help.

[j.drake]

OK, I think I can help. It just so happens that I have a very similar computer. The key to your dilemma is that the recovery partition is actually hda1, not 2, even though it's D:. Your C: drive is actually hda2. When you try to write to hda2, Knoppix is telling you that you can't, because hda2 is actually your NTFS partition, not your FAT32 partition. It confused the heck out of me, too. If you want to verify (and you should), open up each, and see that the hp folders are in hda1, and things like Program Files are in hda2.

As for myself, I put on a second hd. :wink:

I don't think I'd chance resizing the NTFS partition with QtParted or CFdisk. You might be able to siphon away some of that C: drive with Partition Magic, or some other Windows based program, but I wouldn't use linux. If you don't already have Partition Magic, I wouldn't buy it, because you could buy a nice second hd for the same bucks, and have plenty of room. I just bought a 250 GB last week for $150. You could probably double your disk space for the same cost as buying partition magic, then partition it with QtParted in Knoppix however you want.

As for your thumbdrive, try letting Knoppix reformat it as FAT32. That's what I had to do. Look at the "mkfs" command (http://www.onlamp.com/linux/cmd/cmd.csp?path=m/mkfs), and the filetype will be "vfat" (what Linux calls FAT32). Before you do that, though, see if you can mount it from the console. Was it plugged in when you booted? If not, plug it in, open a console, and type lsmod. Should probably come up as sda1. Try "sudo mount -t vfat /dev/sda1 /mnt/sda1". If that won't work, THEN try redoing the file system with the mkfs command.

jd

[Garibaldi]

Here is what I did:
1. I had 786 mb free on the recovery partition, I created a 785 mb persistent home.
2. I rebooted with knoppix home=/dev/hda1 tohd=/dev/hda1, which gave me an error saying that it couldn't copy- not enough freespace, which makes since because I used it all for my persistent home
3. I ran windows and deleted the knoppix.img, my persistent home from the recovery partition
4. Restarted knoppix and ran knoppinx tohd=/dev/hda1, which copied all the files
5. It then loaded knoppix, and I had 106 mb left on the recovery partition, I made a persistent home of 100 mb on the recovery partition. I then rebooted
6. I booted up with knoppix home=search fromhd=/dev/hda1, which gave me an error, filesystem not found, and then gave me very limited options, with pressing the reset key to restart.
I don't know what I am doing wrong. It seems to copy the files to hda1 fine, but then says that there is no filesystem. It also says in the tutorial not to have the persistent home & fromhd on the same partition, why?

[j.drake]

It also says in the tutorial not to have the persistent home & fromhd on the same partition, why?

You can, but you won't be able to unmount the partition if you need to (since that's where the system is running).

I'm not sure what's wrong either, can you still boot the fs by leaving off the home= cheatcode?

jd

[Garibaldi]

I'm not sure what's wrong either, can you still boot the fs by leaving off the home= cheatcode?

I don't know about that. Last night I tried booting one last time with only "knoppix fromhd=/dev/hda1" but it still couldn't find the fs. How much space is needed for the fs? Like I said, I only have 786 mb. Also, what should I do to accoplish this (which of these do I need to do to do this: persistent home, poormans install, save config file): I want to be able to download and install new programs into linux and be able to keep my settings from time to time. Also, I would like it to have good performance. I am guessing that I will need to do all three of those to get that. Finally, how would you recommend me making a persistent home folder, since I can't use the same partition as I'm booting from? I do have a flashdrive that's only 32 megs, not very good. I have it in when I boot, and knoppix shows it on the desktop as sda1, sda2, sda3, and sda4. When I try to click on these it gives me an error on "cannot mount, no fs specified." Maybe I need to format it, could you give me some instructions on how to do that? Thanks.

[j.drake]

Well, bear in mind that the file system just barely fits onto a 700MB CD, and that's with compression. The "tohd" flag takes the this and recreates the file structure on hd. So, although I think there's still some compression, I doubt that it would fit.

Options:

1. Forget "tohd" and "fromhd", boot with live CD, make your hda1 writable, and create a persistent home there. Reboot (still live CD) and save config. Now you are running liveCD with a persistent home and saved config, which is almost as good as poor man's . If your HP is like mine, you probably have two optical drives, one of which is either a CD or DVD burner, and the other one either a CD or DVD reader. On a reboot, when the hp splash is up, poke the F1 key to get into BIOS, go to the boot section, and make sure that your reader is in the boot order where you want it. That way you can still run from CD, and have 786 MB available for your home, plus have one optical drive available for reading and writing. I used this config for a while, and it works fine. The only downside I see is that it is slightly slower that the poor man's. You can still download and install programs from Klik and save files. When you boot, you'll simply enter the codes for your choice of kernel, home and configs.

2. Buy a second hd, and format and partition it to accept whatever you want in linux. That way, if you get to the point where you feel comfortable enough to try a dual boot, you can hdinstall to the second hd

3. Find a program that can safely partition away more of your NTFS drive to FAT32, and try poor man's again.

jd

[firebyrd10]

I don't think I'd chance resizing the NTFS partition with QtParted or CFdisk. You might be able to siphon away some of that C: drive with Partition Magic, or some other Windows based program, but I wouldn't use linux. If you don't already have Partition Magic, I wouldn't buy it, because you could buy a nice second hd for the same bucks, and have plenty of room. I just bought a 250 GB last week for $150. You could probably double your disk space for the same cost as buying partition magic, then partition it with QtParted in Knoppix however you want.
jd
Qtparted uses the plugin ntfsresizer which is very reiable. Given knoppix uses a very outdated version of both ntfsresizer and qtparted so you might want to look into http://www.sysresccd.org/ which has the most uptodate version of any live-cd distro. If needed knoppix can resize without any problems.

[Garibaldi]

Qtparted uses the plugin ntfsresizer which is very reiable. Given knoppix uses a very outdated version of both ntfsresizer and qtparted so you might want to look into http://www.sysresccd.org/ which has the most uptodate version of any live-cd distro. If needed knoppix can resize knoppix without any problems.
Can I run Qtparted from windows? I would like to be as safe as possible, bascially have there be about a 1% chance of something going wrong. If that's the case with Qt Parted then please fill me in with some instructions on how to resize my 144 GB NTSF Partition, siphoning off about 15 GB.

j.drake, Thanks for all the help. I'll probably end up just putting a persistent home on with config for now. I've been booting my Knoppix from the E: drive which came with my pc and is my dvdrw drive. Since it came with the machine I'm sure hp wouldn't put out the bucks to make it 52x, I think its about 24x. I have an Iomega usb cdrw drive with 48x, could I boot knoppix with the cd in that (it might be faster)? Also about the BIOS settings, I could try it but I don't want to take a chance about screwing anything up to the point of not being able to get window again (just to be safe). Since your interested in other oses besides windows, have you tried or heard of pear pc? see www.pearpc.net :)

[j.drake]

No, I think what you are doing is fine. All I was suggesting is that since most HP's that I've seen have two optical drives - one which can read or write (your dvd burner), and one which can only read - I was suggesting running the live CD from the one which can only read, since that would give you the most flexibility (whichever one you use to run Knoppix cannot be used for anything else during the Knoppix session, so if you use the read only drive for running the Knoppix live CD, you can still read AND write with the other).

jd

[Garibaldi]

One last problem j.drake:
I made my persistent home 776 out of 786 megs. I then got knoppix to load my persistent home and config. I checked out my persistent home folder and it says that the size is 345 megs, with 1 meg used. Where did the other 430 megs go? Also, once I save my config file will it automatically update each change that I make or will I have to update it after I make a change? Thanks.

[j.drake]

I don't pretend to know the mechanics of how PH and configs work, so I don't know about the file size issue.

As far as whether you have to update, I don't think so - at least I don't. One thing I've found is that if I save the config in the same drive as my PH, is seems as if I don't have to use the my config bootcode - just the "home=" bootcode. For example, I recently redid my linux drive (hdb) after upsizing my NTFS Windows drive. I have my PH and configs on hdb1, save my Knoppix file system on hdb5, swap on hdb6, and hdb3 isFAT32, for whatever I want to share between OSes. When I boot, my bootprompt is "knoppix26 fromhd=/dev/hdb5 home=/dev/hdb1"

jd

[firebyrd10]

Can I run Qtparted from windows? I would like to be as safe as possible, bascially have there be about a 1% chance of something going wrong. If that's the case with Qt Parted then please fill me in with some instructions on how to resize my 144 GB NTSF Partition, siphoning off about 15 GB.


No I don't think there is a windows version. But if you want to use knoppix or the other cd then I can still help you.


After you get qtparted up, click on the hardrive and let it search it.
Find your NTFS partition and right click it. Select resize and take off about 15 gig. Select ok then go up the the menu and select commit.(Or whatever there is,)

[Garibaldi]

firebyrd10: I only have about 20 GB free, is that okay? Also, it the chance that QTparted will screw up my ntsf very small? Also, I haven't defraged in a while, do I have to? I was thinking of doing the poor man's install, so I guess the main os will take about 2 GB leaving me 13 free for persistent home. Any help you could give would be great. Also, (I'm sure this is a dumb question) how do I install downloaded programs (not klik). I just downloaded mplayer, and there is a file called install, but I don't think it has a extension. For console, do I type something like cd (from DOS) and my directory? Thanks

[firebyrd10]

firebyrd10: I only have about 20 GB free, is that okay? Also, it the chance that QTparted will screw up my ntsf very small? Also, I haven't defraged in a while, do I have to? I was thinking of doing the poor man's install, so I guess the main os will take about 2 GB leaving me 13 free for persistent home. Any help you could give would be great. Also, (I'm sure this is a dumb question) how do I install downloaded programs (not klik). I just downloaded mplayer, and there is a file called install, but I don't think it has a extension. For console, do I type something like cd (from DOS) and my directory? Thanks
Yea, although ntfsresizer has little chance of screwing up your partition, defragging will help. Makes things faster too. Also if you see you only see 1 gig free on qtparted but you know that you have alot more then that free, you need to defrag.

[Garibaldi]

Yea, although ntfsresizer has little chance of screwing up your partition, defragging will help. Makes things faster too. Also if you see you only see 1 gig free on qtparted but you know that you have alot more then that free, you need to defrag.

Is ntfsresizer and qtparted the same thing? Also, if it did screw up my hdd, could I still recover the info if I plugged the hdd into another computer? What steps can I take to lessen the chances of it screwing up my hdd?

[firebyrd10]

Yea, although ntfsresizer has little chance of screwing up your partition, defragging will help. Makes things faster too. Also if you see you only see 1 gig free on qtparted but you know that you have alot more then that free, you need to defrag.

Is ntfsresizer and qtparted the same thing? Also, if it did screw up my hdd, could I still recover the info if I plugged the hdd into another computer? What steps can I take to lessen the chances of it screwing up my hdd?
Qtparted is just a GUI to a program called parted which does all the work.

Ntfsresizer? Just thing of that as a plugin.

As with any program that screws around with a hardrive like this, you are recommended to back up your data. I don't think putting the hardrive in another PC would fix things, but you may still be able to get your data, depending on what went wrong.

To help lessen the chance of something going wrong, make sure you hardrive is defragged (try to get most of the data near the front of the drive) Make sure the hardrive is unmounted and not being used. Don't run any other programs. Do one thing at a time (If you want to resize a partition then add a ext2 partiton, first resize, then make the new one).
Last, try to get the most up to date version of both (qtparted and ntfsresizer), Check out my link earlier in the thread.


Thats about it. Just the general stuff.

[Garibaldi]

firebyrd10: Thanks, I'll let you know... My 144 gb drive is almost full so my defrag is going to take forever...

[Irgu]

If your hard drive partition(s) is (are) formatted as NTFS, you will not be able to save a persistent home on it (them), because Linux doesn't adequately support writing to NTFS yet.

Not really true. There are three totally different NTFS drivers for Linux. The 2.4 kernel has a crappy and limited driver and the 2.6 has a much more advanced one but there is a backport to 2.4 as well. Write is completely disabled in 2.4 and Knoppix doesn't use the advanced driver's backport. The advanced driver supports read-write persistent homes SAFELY for over two years now. Several other distros have this feature for a long time. The first one was Phat Linux in 2002, running Linux fully read-write from NTFS. Corrently the most popular such distro is TopologiLinux but there are more.


And as long as you don't write to NTFS, no, it won't do a darn thing to "harm" Windows.

Still not everything is implemented in the advanced driver but what's released that's fully working and very stable. For example I've also resized quite a lot of NTFS on Linux and never experienced any corruption or lost data.

[Garibaldi]

Not really true. There are three totally different NTFS drivers for Linux. The 2.4 kernel has a crappy and limited driver and the 2.6 has a much more advanced one but there is a backport to 2.4 as well. Write is completely disabled in 2.4 and Knoppix doesn't use the advanced driver's backport. The advanced driver supports read-write persistent homes SAFELY for over two years now. Several other distros have this feature for a long time. The first one was Phat Linux in 2002, running Linux fully read-write from NTFS. Corrently the most popular such distro is TopologiLinux but there are more
So are you saying that when I boot I could type knoppix 26 tohd=/dev/hda2 (hda2 is my NTSF) and I could install and run knoppix from my ntsf partition without screwing up my windows and being able to read/write? Or does that just apply to other versions of linux, if so, could I get some links? Thanks.

[eco2geek]

(This is where I wish we could get definitive answers from someone like Fabian, or Klaus Knopper himself -- the people who wrote the scripts that make Knoppix run.)

Garibaldi, first off, when you use "tohd"/"fromhd" and "bootfrom", Knoppix runs from a read-only image. It's just like running off the CD, except faster. It's not read-write.

The difference between "fromhd" and "bootfrom" is that "fromhd" uses the ~700MB /KNOPPIX/KNOPPIX file from the CD, while "bootfrom" uses the entire ISO image.

In my experience, Knoppix is written so it won't install using "tohd" or run using "fromhd" (the flip side of "tohd") with an NTFS partition. I'm not that good at bash scripting, but it looks to me like the "linuxrc" script in "minirt26.gz" checks the partition type, and simply won't allow you to do it unless your partition is ext2, ext3, vfat, or reiserfs.

You can use "bootfrom" and boot the ISO image from an NTFS partition, but again it's as read-only as running off a CD.

So the short answer would be, no.

Someone in Japan has recently modified Knoppix 3.4 to run a "fromhd"-like poor man's install off an NTFS partition -- see this thread (http://www.knoppix.net/forum/viewtopic.php?t=13209) for the download link. There's an installer included that installs "GRUB for Windows" (looks a lot like the one from Topologilinux), that loads after NT Loader. It's still read-only, though.

I've used Topologilinux (http://www.topologilinux.com), and it's very good. It creates a couple of large files on your NTFS drive -- one for swap and one for the actual installation -- and allows you read/write access within those files. No partitioning necessary.

You just have to be careful to mark them "don't touch me" when you defrag!

I don't know technical details of how Topologilinux does its magic or understand how Linux NTFS drivers work. AFAIK, Irgu is correct in saying that Linux has tools to allow you to resize NTFS partitions safely, as well as to safely write within files that already exist on the partition. But from what I hear, creating and deleting files on an NTFS partition using Linux is still not recommended.

[Irgu]

(This is where I wish we could get definitive answers from someone like Fabian, or Klaus Knopper himself -- the people who wrote the scripts that make Knoppix run.)
Based on Klaus comments on the devel list and forums, apparently he confuses or not enough interested in the several NTFS driver status (crappy NTFS driver, new NTFS driver, Captive NTFS, user space utilities, etc) thus he took a defensive strategy using the default driver from the kernel. In case of 2.4 this is the crappy one (also you can't use it for home) and the better one in 2.6 (you could use it for home). And optionally you can also install Captive NTFS. I don't know Fabian's opinion.

To make things more complicated, the 2.6 kerne'sl geometry detection code and Parted has a serious bug. When you repartition a disk then it can make Windows unbootable. QTParted uses Parted and when it's run on 2.6 kernels it can cause this problem with both FAT and NTFS. However since today at least 80% of the Windows computers use NTFS (or all new Windows installs basically) thus people report QTParted on 2.6 kernels trashed Windows NTFS. They make and spread the wrong conclusion NTFS is still unreliable altough the problem now is really Parted. Luckily Parted was fixed recently.


You just have to be careful to mark them "don't touch me" when you defrag!
Why? It shouldn't matter. Any pointer?


I don't know technical details of how Topologilinux does its magic or understand how Linux NTFS drivers work.
It mounts NTFS read-write then loopback read-write mount the large "container" files for Linux root fileystem and swap from the NTFS partition. Basically what the NTFS driver does is only overwriting these files what is safe. At an upper layer this is seen as reading and writing everywhere.

This is also the reason why defragmentation shouldn't matter because the Linux NTFS driver will find the part of the files anywhere, no matter where the defragmenter relocates them.


AFAIK, Irgu is correct in saying that Linux has tools to allow you to resize NTFS partitions safely, as well as to safely write within files that already exist on the partition. But from what I hear, creating and deleting files on an NTFS partition using Linux is still not recommended.
You can't create or delete files (since W2K and up). This functionality was disabled in the crappy driver about 3 years ago and neither is implemented in the new driver yet. So basically you can recommend to create or delete files because it's impossible, no driver code will run at all, only the "permission denied" message part.

However you can use Captive NTFS to create and delete files but it has its own problems, like legal, slow, complicated, only certain XP drivers work, it doesn't work with 2.6 kernels, can trash NTFS if it's not umounted properly or otherwise, 100 MB file size limitations, etc.

[eco2geek]

Why? It shouldn't matter. Any pointer?

No. I guess that was just my assumption at the time. Scratch that. (If you use Topologilinux's "GRUB for Windows" -- I use it to boot Knoppix -- you do have to reinstall it when you defrag, but that's not the same thing.)


You can't create or delete files (since W2K and up). This functionality was disabled in the crappy driver about 3 years ago and neither is implemented in the new driver yet. So basically you can recommend to create or delete files because it's impossible, no driver code will run at all, only the "permission denied" message part.

Interesting. My question is, how come the Linux kernel developers were able to implement vfat so well, but not NTFS? Is vfat just better documented, or is NTFS harder to "reverse engineer," or...?


However you can use Captive NTFS to create and delete files but it has its own problems, like legal, slow, complicated, only certain XP drivers work, it doesn't work with 2.6 kernels, can trash NTFS if it's not umounted properly or otherwise, 100 MB file size limitations, etc.

Also very interesting. Didn't know that Captive didn't work with kernel 2.6 or had 100MB file limitations.

[Irgu]

My question is, how come the Linux kernel developers were able to implement vfat so well, but not NTFS? Is vfat just better documented, or is NTFS harder to "reverse engineer," or...?
NTFS is much, much more complex than FAT, for example see http://www.ntfs.com/ntfs_vs_fat.htm

From the above table it also turns out that NTFS supports features that the main Linux filesystems usually don't (e.g. compression, encryption, streams, unicode).

There are many Linux filesystem developers, furthermore several companies support Linux native filesystems development (IBM, Red Hat, SUSE, SGI, Namesys, etc) by employing many developers (some over 10, just to work on Linux filesystems!). But NTFS is developed by only a couple of guys in their spare time. Hence the development is quite slow, if nothing would happen.