Knoppix 6.7.0: A View from the Forest

[Forester]

... no, no!

Hi Werner,

I am glad to see that someone else is at least reading this thread. I have read your pages on remastering several times and I do have questions but I've never tried it so why should you answer ? No, I don't use Knoppix installed to hard drive but that is pretty feeble excuse.

I've read a fair way through the thread you started on your laudable attempt to 'update' the Remastering How To but that rambled off onto a long discussion of virtual machines. That discussion took place after I stopped using Knoppix and stopped attempting to contribute to this forum. The irony was that all the time I was attempting to contribute I was using Knoppix in Virtual Box under Windows 7.

My first experience was negative. I got a job to write some PHP scripts for Linux with some people who thought cross-development from Windoze was cool. So I reached for the latest Knoppix only find Plan A blocked by M$ fixing something they called a security loophole and Plan B blocked by the 'withdrawal' of the home= cheatcode from Knoppix. So I had be be inventive and spent a lot of time tinkering with the init script. Virtual box never gave me any trouble. I think it is a really good piece of software even if the 'user interface' is designed to be so easy to use it's insulting.

At home I am privileged. I haven't run Windoze seriously in years: just occasionally inside a virtual machine, of course. I find virtual machines handy: I have Knoppix 6.7.0 LiveCD, LiveUSB and even installed to hard drive all running in virtual machines. I don't have disk space problems either. I've several machines with gigs of LVM space I can carve up any way I need for testing purposes. This means I struggle to appreciate all this nonsense about the safety or otherwise of nfts.

Back on topic (I hope), I understood utu was trying to put together some kind of proposal for a "poor man's" mastering to float past KK. Something as dumb as the current install the USB and install to hard drive scripts. Something the average Knoppix user could use. Bearing in mind that, by definition, half of users are below average.

I don't know that this is possible but I'd be happy to discuss the possibilities if only to understand remastering better. After all, this is only a hobby. I've tried to start with something simple that might be extended rather than start with something ambitious that needs simplifying. The starting point is an method of shuffling files from read/write persistent store to the read-only compressed iso image. As for progression, there is naturally Plan A, Plan B and Plan C.

Plan A considers how far the simple approach could be made to go. What are its real limitations ? How generally useful could we make it ? The greater its potential utility, the more reason KK might have to say OK.

Plan B gives up. Let's call the approach "reflashing" or something catchy and see if KK thinks the simple idea is worthy of distribution.

Plan C is about finding out what utu's real problem is. I suspect here 'backup and recovery' are probably key. We will know a whole lot more when utu explains the special properties he has discovered in the ntfs file system organisation.

My simple approach certainly has limitations.

One is the use of persistent store. I understand that with a hard drive installation there is no persistent store. The simple view is the approach is limited to (real or virtual) Live USB installations.

Another is the output. The approach specifically is about remastering the compressed iso file only. It does not include generating the final iso file that can be burnt to disk. That could be added easily but requires 'additional' disk space and that is a problem.

Another is scale. We begin with adding and removing a few packages. This can easily be extended to include some individual changes if one wants. However, what if you try replace every single package ? I saw somewhere someone tried to replace all 32-bit packages with their 64-bit equivalents. That would require a very big persistent store. Bigger than the 4 Gb limit implied by the current install to USB script. Rewriting that script is off-topic.

Of course you are not going to need a persistent store anywhere near that big unless you are remastering the DVD. The DVD hits another limit. The original remastering uses the original cloop and is a two stage pipe. That should work for most people. The old wiki notes say you need 1 Gb of memory (which, when those notes were written, certainly wasn't the norm). There are complaints from that era from folks whose attempts a remastering failed mysteriously. I think that is because they ran out of memory.

For a the DVD you generally have to scale up by a factor of 7 or 8. Who has 8 Gb of memory on their personal workstation or laptop ? Even with swap space ('additional' disk space by another name) you can't use 8 Gb of memory without using a 64-bit kernel (and possibly some applications compiled for 64-bits).

My way round this was to use squashfs because that's a one stage process with no pipe and so doesn't require lots and lots of memory. But squashfs isn't even an option in the Knoppix distribution.

Hmm ... so the simple approach either has to limit itself to CD sized installations (at which point several people, myself included, lose interest) or use 'additional' disk space. Even then we have a bit of a problem. If you want to remaster a LiveCD, your iso has to fit and a tool, however simple, should check for this. If you just want to remaster a LiveUSB there is no such limitation. The limit is actually 4 Gb unless we rewrite the install to USB script. The DVD is only 3.5 Gb. Oops.

Stopping people running out of memory or at least giving them a sensible error message when they do could be the hardest part of producing a tool that the average Knoppix user could use.

What about the vexed question of 'additional storage' ? The first time I tried remastering I was horrified by how much disk space it required just to add java and eclipse to the Knoppix DVD. I have plenty of disk space spare (none of it ntfs) so I wasn't to worried.

However, I realised that many people might not. They might be tempted to borrow a few gigs from their Windoze partition. I can't recommend that but I recognise that those who need to will rationalise until they are convinced it will be OK. That's for the individual conscience but I don't think it would be OK in a script for inclusion in a distribution. Many people still come to Knoppix because Windoze has trashed its file system. I'm told that is what Windoze does. Would KK accept a script that might be even suspected of complicity ? It would like a fireman setting fires.

If, fortunately this is not likely, I was forced at gun point to remaster Knoppix using a machine with Windows 7 as its hard drive operating system, I would use a virtual machine. That would use big ntfs files inside of which were Linux file systems. I would see this as as safe as it gets since Windows 7, not Knoppix, would be read and writing these files. It seems from the long thread that discussed virtual machines that utu found his Windows trashed his file system more often when he used virtual machines,

Hmm ... somewhere I wrote that I think all you can prove from statistics about file system trashing is that inexperienced people make more mistakes. That's the point: any tool included in the Knoppix distribution has to be safe in the hands of the inexperienced.

[utu]

Something the average Knoppix user could use. Bearing in mind that, by definition, half of users are below average.

Enough, already, of this kind of spam.
This is carrying the MEAN-value theorem to a new level of MEANness.
Give us some good TECHNICAL ideas here.