Knoppix 6.7.0: A View from the Forest

[Forester]

Hi utu,

Thanks for letting me know a new Knoppix was out. I've been busy playing with Debian and thinking I'll get back to Knoppix when a new release came out. Of course I wasn't looking when it did.

I read the thread http://www.knoppix.net/forum/threads/29397-Knoppix-V6.7.0? with interest until it wandered way off topic. You raised a number of questions I felt where not answered. No point in replying there.

So, 6.7.0 is not all that different from 6.4.4. People sounded disappointed. Is that because it's 6.7.0 and not 6.4.7 ? I think the Open Source community are trying to standardise the significance of version numbers. Under that scheme 6.4.7 would be bug fixes only and that, I guess, is not true. I guess Knoppix 7 will be based on Debian Wheezy.

Yes, there's probably all the bugs fixes between Debian 6.0.0 and 6.0.2.

Yes, there's a new kernel (2.6.39.3 instead of 2.6.37) but I don't know what that means. There is one new kernel cheatcode on the bootline (hpsa.hpsa_allow_anty=1) but I don't know what that means either.

Yes, there's a new X server (1.10.2 instead of 1.9.4 RC 1) but the experimental nouveau driver hasn't got a version bump. Maybe folks with Nvidia drivers will notice an improvement since the Knoppix releases notes don't really warn about this any more.

Firewire has reappeared in the kernel configuration. There used to be two firewire implementations in the kernel. Knoppix stuck to the old one until it was withdrawn in 2.6.37 and along with it went the Knoppix support. I don't have any firewire devices so I can't report that it is working.

The majority of changes in knoppix-autoconfig are beefing up of the recognition of the graphic and sound 'cards'. I also saw the comment:


# For now, it is OK to start network-manager before udev is completeSo that's why your system starts quicker. For now. ;)

I note also the keyboard has been fixed for those who use a UK keyboard. I don't but I tried it anyway and the cheat code lang=uk doesn't lock up the keyboard the way it did with Knoppix 6.4.4.

The cheatcodes ? Well, lang=uk wasn't documented and it still isn't. I checked the EN edition, CD and DVD. I downloaded both using the torrent. Along with the iso file, the torrent also downloads a copy of the cheatcodes. This file still lists home= so it has not been updated. Oops. However, when you get into the iso file, the copy in the KNOPPIX subdirectory was updated in February. There is no description of home= but the description of bootfrom=, which isn't in Knoppix 6.4.4 either, is still there. Hmmm ... so I still would not trust the file. :(

However, it isn't all bad ...

The changes in init are almost all labelled GvR. The biggest is an implementation of bootfrom= Hurray ! :-D

The withdrawal of home= and bootfrom= made Knoppix 6 so much less flexible than Knoppix 5 and other Live CDs that I've wondered if it was a deliberate attempt to make Knoppix unfashionable.

What's the big deal ?

OK. I download the Knoppix iso. I want to do a USB install but with Knoppix 6.4.4 and earlier I have to burn the iso to CD/DVD first. The first thing I'm going to do is remaster the USB. I'm never going to use the CD/DVD again - not without a home= cheatcode - so it's a waste.

The bootfrom= cheatcode allows one to boot the iso directly without burning a CD using what is called a loopback.cfg (see http://www.supergrubdisk.org/wiki/Loopback.cfg). The loopback.cfg are what those folks who try to put 100 distributions on a single disk use.

So, using my Debian installation, I copied the Knoppix 6.7.0 CD iso file to the root directory of a disk partition with enough space - it happened to be /dev/sda9. I added the following to my menu.cfg file:



menuentry 'Knoppix 6.7.0cd' {
set root='(hd0,msdos9)'
loopback loop /debian/KNOPPIX_V6.7.0CD-2011-08-01-EN.iso
echo 'Loading Knoppix 6.7.0 ...'
linux (loop)/boot/isolinux/linux bootfrom=/dev/sda9/KNOPPIX_V6.7.0CD-2011-08-01-EN.iso [other cheat codes from the syslinux bootline for knoppix]
echo 'Loading initial ramdisk ...'
initrd (loop)/boot/isolinux/minirt.gz
}
I reboot and select Knoppix from the grub boot menu and Knoppix booted. From there I could install the USB flash as normal.

Neat, if you run a proper Linux: I used the grub2 bootloader. I don't think you can do this with the syslinux bootloader used by the Knoppix LiveCd or LiveUSB and I know you can't do this with what is now called 'grub legacy'. Oops. Even Knoppix 6.7.0 still uses 'grub legacy' so if all you've got is Windoze and Knoppix installed on your hard drive this won't work for you.

Perhaps Knoppix uses 'legacy grub' because the install to HDD script needs it and no one wants to upgrade that old script. :rolleyes: I tried removing 'legacy grub' and installing grub2 but Knoppix wasn't having it. Oh well.

The obvious change in 6.7.0 among the applications is the withdrawal of the browser and e-mail client I've used for 10 years now ever since their first beta and my first dial-up internet connection. That's progress I suppose.

I still think Knoppix is a good distribution considering how much effort goes into it.

Ciao.

[utu]

Nice to have you back, Professor Forester.



The changes in init are almost all labelled GvR. The biggest is an implementation of bootfrom= Hurray ! :-D


Are you acquainted with GvR?

Also, hope you'll help me get my homework corrected on a new re-mastering thread
I've started. I've got a lot of questions for you.

Ciao.

[utu]

I note also the keyboard has been fixed for those who use a UK keyboard. I don't but I tried it anyway and the cheat code lang=uk doesn't lock up the keyboard the way it did with Knoppix 6.4.4.



dyfi could probably use your help:
http://www.knoppix.net/forum/threads/29433-Keyboard-Settings?highlight=uk+keyboard

[Forester]

Hi Utu,

I see that some folks aren't too happy that Iceweasel/Firefox disappearing from Knoppix (and that KK has even asked for their opinion !). I also read mutterings that some are unhappy with the size of LibreOffice and wondered about chucking bits they did not want.

I had similar issues with Debian Squeeze but I didn't get Firefox direct from Mozilla nor did I start mixing packages from Wheezey and Sid in with my nice stable Squeeze. I went to Debian backports instead. This ensures I'm using packages built for my system so, I hope, minimises the risk of incompatibilities and sub-optimisation. So, I thought, why not do the same with Knoppix 6.7.0 ?

Iceweasel and Icedove

My understanding is that these are Firefox and Thunderbird simply re-branded for 'copyright' reasons. Otherwise nothing added and nothing taken away. Only they tend to be out-of-date versions and set to be more so as Mozilla intends to crank out a new release every three months.

You can find the Debian Mozilla Team's response at http://mozilla.debian.net/. There are some simple instructions there that allow you install up-to-date-as-you-like (yes, there's a choice) versions of Iceweasel and Icedove built for Squeeze, which is what Knoppix 6 is (was ?) largely based on.

Following the instructions, I added the backports repositories to my sources.list by adding a file I named backports.list to the directory /etc/sources/source.list.d:


deb http://mozilla.debian.net/ squeeze-backports iceweasel-release
deb http://mozilla.debian.net/ squeeze-backports icedove-release
deb http://backports.debian.org/debian-backports squeeze-backports main
Following the instructions, I then added the additional key for the mozilla.debian.net repository:


wget -O- -q http://mozilla.debian.net/archive.asc | gpg --import
gpg --check-sigs --fingerprint --keyring /usr/share/keyrings/debian-keyring.gpg 06C4AE2A
gpg --export -a 06C4AE2A | sudo apt-key add -
sudo apt-get update
If you try this, following the original instructions on http://mozilla.debian.net/, not my copy.

I then installed the 'release' versions of Iceweasel and Icedove:


apt-get install -t squeeze-backports iceweasel icedoveThat got me Iceweasel 6 and Icedove 5. I was able to install the (en-gb) internationalisation for Iceweasel but not for Icedove (it seems the latter is still for Icedove 3.2).

I hope this has stabilised. When I tried a couple of months ago I got Iceweasel 5 (no internationalisation available) and Iceweasel 3.1 (with internationalisation), which came from the main backports repository, not the Debian Mozilla repository.

LibreOffice

Debian Squeeze hasn't got LibreOffice so Knoppix uses the Wheezy version. I took off the Wheezy version and put on the version from Squeeze Backports and I did it so as to determine how much space you might save by installing just the word processor and spread sheet applications. This process regressed from LibreOffice 3.3.4 to 3.3.3.

Now for the bad news: it took three attempts. The first failed because of repository incompatibilities, second because I ran out of 'persistent' store. It failed when I started with an empty 'persistent' store of 512 Mb. :( It was OK with 1 Gb.

First I removed the existing LibreOffice:


libreoffice=$(dpkg -l | awk "/libreoffice/"'{print $2}')
sudo apt-get purge ${libreoffice}That, it told me, freed 301 Mb of diskspace.

Next (don't do this) I did:


sudo apt-get autoremoveTo remove other 'no longer required packages'. That freed another 42.2 Mb.
This was a mistake: most of these packages need reinstalling when I put on the Squeeze Backports version and that failed due to some version incompatibility.

Start again. Put back just the word processor and spread sheet applications.


sudo apt-get install -t squeeze-backports libreoffice-writer libreoffice-calcThis used 256 Mb of space suggesting a saving of around 45 Mb.
Just to confirm:


sudo apt-get install -t squeeze-backports libreoffice
sudo apt-get install -t squeeze-backports libreoffice-l10n-en-gb
sudo apt-get autoremove
Added an extra 35 Mb and another 11 Mb and freed 1 Mb giving a total of 45 Mb to the nearest Mb.

If you want to save some space when remastering and your English happens to be an American dialect, remove all the German language internationalisation, help and spelling packages. That's all as in not-just-for-LibreOffice-but-for-all-packages.

Repositories

The really bad news is the repositories. Especially if you are using an installation with 'persistent store'.

Just doing a:


sudo apt-get updateuses 160 Mb and after the installation of the Squeeze Backports version of LibreOffice alone a further 167 Mb of packages are sitting in the apt cache. You certainly would not include the latter in a re-mastering and I doubt you'd include the former.

Then


sudo apt-get upgradetells me that already 142 packages are out of date. Ouch. Most of these will be from Wheezy or Sid as there is a lot more churn in these respositories than in Squeeze (by definition).

Worse the upgrade would only upgrade 63 of these: the other 79 were held back. I suspect this has something to do with a pending upgrade to the gcc compiler and the C++ libraries. To my mind one thing 'stable' means is you don't go messing with the compiler.

To complicate matters further, it wanted to upgrade the LibreOffice packages but half of them would be held back. Install the Squeeze Backport version made no difference. I'm sure I could pin LibreOffice to 'workaround' this one but that is beyond my current experience.

My conclusion is that upgrading, and therefore remastering Knoppix, is for braver men than me with far more spare time to burn.

Have fun. ;)

[Forester]

Nice to have you back, Professor Forester.

Don't count on it.


Are you acquainted with GvR?

No. I've lived in Flanders (and can recommend it) but I never met anyone called Gilles van Ruymbeke.


... a new re-mastering thread I've started.

What yet another thread ? I haven't read all the others yet.

The long thread (that started on Knoppix 6.7.0 and wandered off topic) I read at times reminded me of the four blind men describing an elephant. I'd like to suggest that if you're going succeed in your efforts you will need to come up with a description that not only KK recognises as an elephant but that the great majority of Knoppix users, young and old, would recognise too.

And you should come up with the description before you try to make the elephant.

Too often (and I count myself among the guilty) folks go off and make something in the belief that when it is done it will be self-explanatory. Saves all that thinking, designing, explaining, and convincing that ought to go before the doing.

I think you need to define what you are doing before you start. A quick glance at your new thread and I see you've tried. Well done. But a second glance and I see it is all about 'how', not about, 'what' or even 'why'.

Am I making any sense ?

You start with:



Re-mastering will require many gigabytes to accomplish.
Oh.

Please don't give me any credit at all in your efforts. My effort was carefully construction NOT to require any external disk space. That was the only original contribution I made. The rest was plagiarised from elsewhere.

It was just another view of the elephant. Obviously not a popular one.



Also, hope you'll help me get my homework corrected ...


Could you explain the special properties you attribute to ntfs-3g ? At the moment I only see a stick with two wrong ends. I'm sure there is a ramble of thread somewhere I could read if I wanted to get really confused but that wouldn't really help you.

Ciao.

[utu]

Greetings, Forester

I wonder if we might return to your thread,
http://www.knoppix.net/forum/threads/29120-USB-Remasterig-Using-Minimum-Real-Estate?highlight=forester
and specifically your program doremaster.

Of the nine views, three are mine.

I like all the windworks leading up to the program,
but I can't make it work. Stuck at square one.

I have two 2 Gb SanDisk LiveUSBs both made from Knoppix 6.7 LiveCD
one at /dev/sdb1, one at /dev/sdc1. Both boot ok by themselves

Can't survive the first test of 'tri-state logic'.

[utu]

And you should come up with the description before you try to make the elephant.

Too often (and I count myself among the guilty) folks go off and make something in the belief that when it is done it will be self-explanatory. Saves all that thinking, designing, explaining, and convincing that ought to go before the doing.

I think you need to define what you are doing before you start. A quick glance at your new thread and I see you've tried. Well done. But a second glance and I see it is all about 'how', not about, 'what' or even 'why'.

Am I making any sense ?

Greetings again, Forester

I was tempted to respond in kind and define some elephants, but
my quest is not that grand. I'm only asking for the addition of
two modest additions to the Knoppix menu. These two scripts would
use already built-in elements of Knoppix for the most part.

We all use the menu function 'Install KNOPPIX to flash disk' from
the Knoppix LiveCD Menu. Have you tried it from the LiveUSB menu?
That's another story. I want you to focus on this customer's
request for just a couple of NEW menu items.

One new LiveUSB menu choice, actually produce something useful for
an improved clone, which would create (1) a new-and-improved compressed
KNOPPIX file containing the net combined net contents of the source's
KNOPPIX file and the source's persistence file; and (2) a renewed 'empty'
persistence file. This product should be bootable.

The second LiveUSB menu choice would produce a similar LiveCD product,
but we will forego the persistence aspect (for now, anyway).

I don't personally care any more whether these results require additional
storage to PREPARE the magic for the LiveUSB. I still want the PRODUCT
to fit on a 2 Gb LiveUSB started initially from a LiveCD iso.
I want to make as good a use of the LiveUSB real estate as Klaus does
with the LiveCD real estate.

The new LiveCD from LiveUSB menu choice is to provide some belt-and-
suspenders back-up to tide us over those times we clobber our r/w
media we've come to rely on with our clumsiness. 'us' and 'our' are royal singular.

Your doremaster idea seems to be about as elegant an idea as I could ask
for. But I can't make it work.

[utu]

I never met anyone called Gilles van Ruymbeke.


You might be interested in this thread:

http://www.knoppix.net/forum/threads/11589-ISO-boot-from-FAT-NTFS-USB-%28GRUB.exe-grldr-from-boot.ini%29?highlight=ruymbeke:

Ciao.

[Forester]

I wonder if we might return to your thread,
and specifically your program doremaster.

Of the nine views, three are mine.


I had a very quick look at the script over on your thread http://www.knoppix.net/forum/threads/29435-Re-mastering-Proof-of-Principle-Effort (http://www.knoppix.net/forum/threads/29435-Re-mastering-Proof-of-Principle-Effort)and at the scripts I have here in my archive.

Here's the irony ... the script I published for you and dinosoep to try out was very much a cut down version and it seems that folks have been busy ever since trying to add back what I took out. :)

It wasn't that I didn't want to share. The script was flexible and powerful. The more powerful, the more rope with which one could hang oneself and the greater the difficulty of describing how to use it safley.

So I dumbed it down for you. It seemed I hadn't dumbed it down enough but I couldn't see how I could dumb it down any further. I was flummoxed.

I have tried, on and off this forum, to help people who are not exactly 'computer literate'. Their very best attempt to articulate their problem is "It doesn't work". You have to be very patient with such people. After asking them a lot of dumb questions you figure out that "doesn't work" means that their PC boots Windows when they expect it to boot Knoppix and that by "it" they mean Knoppix, not Windows or their PC. After a lot of heading scratching, you suggest they open the CD drive, turn the CD over and try again. Surprise, now it works.

The feed back I got from you was along the lines of "It doesn't work. When you fix it could you add the following features ...".

I was nonplussed. I expected something more constructive from a Senior Member on this forum. My mistake. I apologise.

Had you written "I've changed the script as suggested. Here's a diff -Naur showing my changes. Can you see where I've gone wrong ?" I expect I would have (then, yes, now, maybe not).

Had you written "When I run the script I get the following output ... [copy/paste]. This doesn't look right. What should I expect to see ?" I expect I would have been able to tell you immediately.

Instead I get "Stuck at square one". All that told me was you hadn't got far enough to have a new LiveUSB that wouldn't boot (I expected that one) or you'd run out of 'disk space' half way through (which I now expect you'd hit first).

Folks I have helped spread out on a wide spectrum. At one end are those who genuinely want to understand their problem so they can solve it themselves. I have all the time in the world for them. At the other end are those who want someone else to take responsibility for their problem. Not interested in helping them and I'm not sorry to say so.

On this occasion you definitely came over as being at the wrong end of the spectrum.

There may be feedback on that thread from other people. Their feedback might be more useful but I'm not interested. I published the script for you. All I know is it doesn't work. Forget it. Life is too short.

There are several people on these forums who have meddled with remastering and are computer literate enough to make their own way. They perhaps don't have much incentive to collaborate. I guess I'm one of those and you aren't. I sympathise but I cannot empathise. Somewhere on the long, rambling, thread that started talking about 6.7.0 someone replied to a question of yours with the suggestion you try it for yourself. I can empathise with the irritation that prompted that response but I can't sympathise. You deserve better. But then so do we all.

[utu]

I can see you are exasperated, Forester.
Cool off. This is a hobby for some of us.

I'd hoped to tap into your obvious professional expertise.
If I'd found anyone else using your magic technique,
I would have tried to get some help from them, having
already had a similar ration from you before.

The only others I am aware of working the problem
seem either to need or to prefer a separate, and to my mind
LARGE, workspace to do their remastering. Some intrepid
souls even use Windows OS partitions with abandon.

I am primarily interested in how you manage to do otherwise.
I'm impressed at your cleverness and programming clarity.
Only six other downloads of the program I'm complaining
about. Have you heard anything from the others about your
unique take on remastering?

Have you approached KK with your idea? I'm sure he has
the chops to read and execute your code and give you an
opinion. I'd really like to follow such a dialog between
yourself and KK on the advisability and genius which your
scheme exemplifies.

There does exist an intelligent, friendly, civil level of
discourse between 'elephants' and professional level bash
scripting.
You haven't found it yet. I'm here to help you in that regard.

[Forester]

Greetings again, Forester

Hi utu,

I quite liked your description (post #7, this thread) of what you want the first time I read it. It is a fair reflection of what I think we discussed way back when.

I don't know that others will understand it quite the same way. Some might not recognise it as 'remastering', some might wonder what is so special about this set of requirements. I seem to remember KK himself wondering why bother.

I still think you have a valid point to make but it needs to be able to made convincingly.

When I think of what is involved in remastering I see a 'selection' of what to include, 'assembling' what is selected, 'generating' a new image from the assembly and, of course, 'testing, testing, testing'. Others might see it differently - they might include publishing the new image, for example. I think the process is iterative and I'd see testing as continual, rather than something that is done at the end.

In the middle of all this, the 'generation' bit may not seem all that important but that was all we were really talking about. Perhaps our approach is a bit to simplistic. I don't think so but you would need to convince the skeptics.

You start with a working LiveUSB installation. You use synaptic to add a few packages and may be remove a few others. That covers 'selection'. You've also covered 'assembly': everything is in your persistent store. You've been playing with the new packages as you've installed them. That covers 'testing' too (for all practical purposes). So all you need is the 'generation' bit and that isn't something you can do with a click or two of the mouse.

This may not be 'remastering' as KK knows it but it seems to be a reasonable scenario (at least) for the 'ordinary' user's first attempts at remastering. I think there is a case of providing a simple (to use) tool for this.

Do this make sense ? Is there any point in my continuing ? It only gets harder from here on.

The next questions you might be asked as about the scope the method covers (like is this LiveUSB only ?). There is no problem with there being limitations but it helps if you know what they are. Open discussion may help widen the scope at little cost and bring others on board.

P.S. Still interested in the special properties of ntfs for remastering.

[utu]

Greetings, again Forester.

Are we on the same wavelength again? I think so.
The word 're-mastering' is itself a problem.

I surely don't imply the process KK must go through to make a new version of Knoppix.
Surely, he includes some new source compilation; I can forego that.
He brings in new kernels and makes all the drivers that go with that.
Lots of stuff I'm sure I don't even know about.
I'm talking about things a non-pro user might want to tide himself over between
official Knoppix versions. Something just little more elegant than persistence plus
backup.

What I would settle for in 'selection' might simply include a few Synaptic additions,
some minor home-grown script additions, plus the sundry tweaks one puts into tailoring
one's desktop. Nail these down and pack them into the compressed image. Have the
option to store that as a snapshot on a liveCD as well as a LiveUSB. I don't find it
useful to move all my stuff into VirtualBox or VMWare to try out new things, especially
on a Windows workspace. Been there; done that.

I have the feeling that handling Synaptic removal of things in KKs original compressed
file may be off-limits for any simple solution. I can give that up. Or, I can try it and
throw out the result and go back to a previous snapshot of everything that worked before.

The 'testing' phase I use is just everyday use of my LiveUSB. Whatever works, I keep
in persistence. If it doesn't work, I don't keep it. If it doesn't work and ruins my
current LiveUSB, I back up to an earlier version for which I have a (compressed) copy
of KNOPPIX-DATA that works and bring that in as update.tar.gz. Moving KNOPPIX-DATA into
KNOPPIX occasionally would solve the back-up problem automatically, and more elegantly.

I see 'Generating' as the neat scripts you and Capricorny and Werner use to re-generate
a new KNOPPIX file from an old one plus the accumulated 'additions' in persistence.
And re-plenishing the persistence file, of course.
Creating such a script is beyond both my capabilities and interest, but I'd like to
have the capability of such a script in my working LiveUSB. Here's my take on the
possibilities I know about:

The following table outlines my current estimate of the state of progress along these lines.
It would have been a little prettier if odf formats were allowed on the forum.
I will be glad to correct any errors in this representation.



iso Product Workspace Virtual Infra- Principle LiveCD LiveUSB KK KK
Concept Req'd? Req'd? Structure Status Product Product Aware Interest

PCLinuxOS No Immaterial Mandriva In Practice Yes No Probably Prob Not
Forester No Optional Knoppix --?-- No No Prob Not Likely
Capricorny Yes Optional Knoppix Proven No No Prob Not --?--
Schulz Yes Essential Knoppix Proven No No Yes Some

Desired Pref No Optional Knoppix Proven Yes Yes Yes Yes
My intention is to advocate a progression of one or more of these, or similar, concepts toward
a state of visibility sufficent to attract KK's awareness and interest such that that these new
functions might actually appear in some future incarnation of Knoppix.

We probably need a different name than re-mastering for this modest upgrade of Knoppix.
Something like it is called mylivecd in PCLinuxOS.

[Werner P. Schulz]

.. and Werner use to re-generate
a new KNOPPIX file from an old one plus the accumulated 'additions' in persistence... no, no! I don't work with accumulated stuff in any persistence memory. I do all my changes in a real Harddrive Installation and test it within this. That's the way, all people work.

You need a backup concept or a version control system to go back to former state of testing in case of error.

And only then, when all went well, I repeat the changes in the chroot jail using some scripts and build a new ISO.

And why do you insist on working within Windows? Windows is the worst case for handling Linux development. My suggestion, to use VirtualBox is only a poor man solution, it isn't "essential"; the better way is using a separate partition for Knoppix-Installation and boot it.


Greetings Werner * http://www.wp-schulz.de/knoppix/summary.html
Own Rescue-CD with Knoppix (Knoppix V6.7.0 remaster)

[utu]

@ Werner

I hope I haven't offended you with my simplistic representation of your re-mastering approach.

In my table, I am trying to contrast my suggested modest addition to Knoppix with what you
and others are using for much more complex and thorough manipulations. What I'm proposing
is a menu-driven back-up utility of sorts, but capitalizing, I hope, on being able
to occasionally move back-up content into the compressed image, once any changes have
'panned-out'. Plus a 'hard-copy', LiveCD product as well.

My unhappy experience with VirtualBox and VMWare on Windows was a learning experience for me.
At the time, I had no source of workspace storage adequate for re-mastering except the ntfs
partitions of my laptop, nor other large Linux partitions. Since one of our forum members
uses Windows partions for workspace, I thought I could probably at least do some virtual stuff there.
Well, after several occasions that I needed to 'restore' my Win7 image, I decided that's
not for me.

Suffice it to say, if what I'm proposing requires any significant external workspace, then it
probably won't be very popular. I do think Knoppix should have some built-in back-up choices
of some kind.

[Forester]

... no, no!

Hi Werner,

I am glad to see that someone else is at least reading this thread. I have read your pages on remastering several times and I do have questions but I've never tried it so why should you answer ? No, I don't use Knoppix installed to hard drive but that is pretty feeble excuse.

I've read a fair way through the thread you started on your laudable attempt to 'update' the Remastering How To but that rambled off onto a long discussion of virtual machines. That discussion took place after I stopped using Knoppix and stopped attempting to contribute to this forum. The irony was that all the time I was attempting to contribute I was using Knoppix in Virtual Box under Windows 7.

My first experience was negative. I got a job to write some PHP scripts for Linux with some people who thought cross-development from Windoze was cool. So I reached for the latest Knoppix only find Plan A blocked by M$ fixing something they called a security loophole and Plan B blocked by the 'withdrawal' of the home= cheatcode from Knoppix. So I had be be inventive and spent a lot of time tinkering with the init script. Virtual box never gave me any trouble. I think it is a really good piece of software even if the 'user interface' is designed to be so easy to use it's insulting.

At home I am privileged. I haven't run Windoze seriously in years: just occasionally inside a virtual machine, of course. I find virtual machines handy: I have Knoppix 6.7.0 LiveCD, LiveUSB and even installed to hard drive all running in virtual machines. I don't have disk space problems either. I've several machines with gigs of LVM space I can carve up any way I need for testing purposes. This means I struggle to appreciate all this nonsense about the safety or otherwise of nfts.

Back on topic (I hope), I understood utu was trying to put together some kind of proposal for a "poor man's" mastering to float past KK. Something as dumb as the current install the USB and install to hard drive scripts. Something the average Knoppix user could use. Bearing in mind that, by definition, half of users are below average.

I don't know that this is possible but I'd be happy to discuss the possibilities if only to understand remastering better. After all, this is only a hobby. I've tried to start with something simple that might be extended rather than start with something ambitious that needs simplifying. The starting point is an method of shuffling files from read/write persistent store to the read-only compressed iso image. As for progression, there is naturally Plan A, Plan B and Plan C.

Plan A considers how far the simple approach could be made to go. What are its real limitations ? How generally useful could we make it ? The greater its potential utility, the more reason KK might have to say OK.

Plan B gives up. Let's call the approach "reflashing" or something catchy and see if KK thinks the simple idea is worthy of distribution.

Plan C is about finding out what utu's real problem is. I suspect here 'backup and recovery' are probably key. We will know a whole lot more when utu explains the special properties he has discovered in the ntfs file system organisation.

My simple approach certainly has limitations.

One is the use of persistent store. I understand that with a hard drive installation there is no persistent store. The simple view is the approach is limited to (real or virtual) Live USB installations.

Another is the output. The approach specifically is about remastering the compressed iso file only. It does not include generating the final iso file that can be burnt to disk. That could be added easily but requires 'additional' disk space and that is a problem.

Another is scale. We begin with adding and removing a few packages. This can easily be extended to include some individual changes if one wants. However, what if you try replace every single package ? I saw somewhere someone tried to replace all 32-bit packages with their 64-bit equivalents. That would require a very big persistent store. Bigger than the 4 Gb limit implied by the current install to USB script. Rewriting that script is off-topic.

Of course you are not going to need a persistent store anywhere near that big unless you are remastering the DVD. The DVD hits another limit. The original remastering uses the original cloop and is a two stage pipe. That should work for most people. The old wiki notes say you need 1 Gb of memory (which, when those notes were written, certainly wasn't the norm). There are complaints from that era from folks whose attempts a remastering failed mysteriously. I think that is because they ran out of memory.

For a the DVD you generally have to scale up by a factor of 7 or 8. Who has 8 Gb of memory on their personal workstation or laptop ? Even with swap space ('additional' disk space by another name) you can't use 8 Gb of memory without using a 64-bit kernel (and possibly some applications compiled for 64-bits).

My way round this was to use squashfs because that's a one stage process with no pipe and so doesn't require lots and lots of memory. But squashfs isn't even an option in the Knoppix distribution.

Hmm ... so the simple approach either has to limit itself to CD sized installations (at which point several people, myself included, lose interest) or use 'additional' disk space. Even then we have a bit of a problem. If you want to remaster a LiveCD, your iso has to fit and a tool, however simple, should check for this. If you just want to remaster a LiveUSB there is no such limitation. The limit is actually 4 Gb unless we rewrite the install to USB script. The DVD is only 3.5 Gb. Oops.

Stopping people running out of memory or at least giving them a sensible error message when they do could be the hardest part of producing a tool that the average Knoppix user could use.

What about the vexed question of 'additional storage' ? The first time I tried remastering I was horrified by how much disk space it required just to add java and eclipse to the Knoppix DVD. I have plenty of disk space spare (none of it ntfs) so I wasn't to worried.

However, I realised that many people might not. They might be tempted to borrow a few gigs from their Windoze partition. I can't recommend that but I recognise that those who need to will rationalise until they are convinced it will be OK. That's for the individual conscience but I don't think it would be OK in a script for inclusion in a distribution. Many people still come to Knoppix because Windoze has trashed its file system. I'm told that is what Windoze does. Would KK accept a script that might be even suspected of complicity ? It would like a fireman setting fires.

If, fortunately this is not likely, I was forced at gun point to remaster Knoppix using a machine with Windows 7 as its hard drive operating system, I would use a virtual machine. That would use big ntfs files inside of which were Linux file systems. I would see this as as safe as it gets since Windows 7, not Knoppix, would be read and writing these files. It seems from the long thread that discussed virtual machines that utu found his Windows trashed his file system more often when he used virtual machines,

Hmm ... somewhere I wrote that I think all you can prove from statistics about file system trashing is that inexperienced people make more mistakes. That's the point: any tool included in the Knoppix distribution has to be safe in the hands of the inexperienced.

[utu]

Something the average Knoppix user could use. Bearing in mind that, by definition, half of users are below average.

Enough, already, of this kind of spam.
This is carrying the MEAN-value theorem to a new level of MEANness.
Give us some good TECHNICAL ideas here.

[utu]

@ Forester

I found your post #15 about 50% offensively trite with its Bilious excesS; and
about 25% recounting things as they are, but not news.
In the remainder, you glossed over two things important to what I am interested-in.
I would be interested in delving into these areas in a more, but not excruciatingly so,
level of detail:




The approach specifically is about remastering the compressed iso file only.
It does not include generating the final iso file that can be burnt to disk.
That could be added easily but requires 'additional' disk space and that is a problem.


A final USB product can be fabricated via cut-and-paste without 'additional' disk space,
if the required iso file and a replacement persistence file are available.





My way round this was to use squashfs because that's a one stage process with no pipe
and so doesn't require lots and lots of memory.
But squashfs isn't even an option in the Knoppix distribution.


Are you sure squashfs is not 'in' 6.7?
If not, how big a change would that be?
What are the elements of this change?

[Werner P. Schulz]

... it is hard, to classify Foresters excellent analysis as spam. And I can understand him, if he isn't willing to help you in this Principle (http://www.knoppix.net/forum/threads/29435-Re-mastering-Proof-of-Principle-Effort) thread - a huge building site, only some problems solved and far away to become in the future a nice little script for unexperienced users.

[Werner P. Schulz]

@Forester

I have read your pages on remastering several times and I do have questions but I've never tried it so why should you answer ?
I'm glad to hear you've visited my page. But if you have questions, why don't you ask me; there is a link how to contact me.

[Forester]

... it is hard, to classify Foresters excellent analysis as spam.

Thank you Werner for your encouragement. I do rather seem to have upset utu.

The point I failed to make was that KK might not want something that only half of Knoppix users could use. Of course any solution offered to KK would have to satisfy utu. He is the sponsor and possibly the only test bed. Do the math.

utu, the points glossed over.

On the first, I think the answer is yes. I used a USB installation because I needed a portable solution. Yes, I backed it up to hard drive every day. I know many people think back ups are an imposition and don't/won't/can't do them. I had a second USB stick in case I lost the first. I know many people don't. My script 'remastered' from the first to the second without requiring any 'additional' (aka temporary) file space. It seemed to me that that modest investment in a second USB stick might be easier for inexperienced users than repartitioning their hard drive or learning to use virtual machines. That is, of course, just a personal opinion.

On the second, I should have written that squashfs is not supported as an alternative to cloop. The Linux kernel squashfs module is in 6.4.3 and 6.4.4. I have every reason to believe that it is also in 6.7.0 (but haven't checked). However, you can't boot Knoppix from squashfs because the init script does not support it. There is a long thread that discusses this. I posted a patch for init that supported this (and I think dinosoep tried it). The patch works for 6.4.4 but it might not for 6.7.0.

I write scripts for two purposes: for things I don't do often and things I do do often. This is to be less dependent on my poor memory and my poor typing. Scripts are meant to be read and changed. I have little experience of writing scripts for others to use, especially those who can't read and change them. I made a mistake when I published the one utu can't use. I won't do that again.

If I ever do such a thing again, I will provide a GUI front-end but I have no experience of writing such things and have no idea when or if I could start such a project. It is easy to say I want simple GUI script but it is quite something else to write one. After all, when a command line script goes wrong the user can see what has happened and is supposed to figure out what to do whereas with a GUI, the script hides the error and tries to figure out what to do by itself.

My analysis, as far as it has gone, is that the first step is to replace the install to USB script. As it stands, it uses vfat, which has the distinction of being the only hard drive file system I know of that does not support files with holes (or sparse files or whatever you call them). That's why the backup / restore functions I did for 6.4.4 were dismissed as 'too slow'. It imposes other limitations that make doing something nice and simple like utu wants very much more difficult that it should be.