Ramdisk and partial remastering in Knoppix 7

[klaus2008]

I'm interested to hear the experiences of others with Knoppix 7's ramdisk.

Hello utu!

When I read Klaus Knopper's post on the mailing list I decided to try the remastering procedure from the live system myself.

I use a kind of Poor man's install of Knoppix V 7.0.2 inside a VirtualBox 4.1.16. I copied the content of the DVD to the directory knx702 and use Grub as bootloader.

I think that a typical session looks as follows.

1. Boot with "noimage" cheat code.

2. Open root terminal.

3. Issue the following commands.

apt-get update
apt-get install flashplugin-nonfree
apt-get clean

Of course, there are many more useful packages that are not included in the public version of Knoppix 7.0.2 since they are not free like unrar for example.

4. Create the archive file KNOPPIX1 from the ramdisk. The options used for makeisofs are the usual ones taken from the "Remastering howto". I use a temporary file on the virtual HDD since I do not have more than 1 GB of RAM available inside my virtual machine. I exclude some directories with files I consider not useful for the archive. Maybe there are more directories.

mkisofs -R -U -V "KNOPPIX1 filesystem" -P "KNOPPIX1" \
 -hide-rr-moved -cache-inodes -no-bak -pad \
 -x /ramdisk/var/lib/apt/lists -x /ramdisk/dev \
 -x /ramdisk/var/log -x /ramdisk/var/run /ramdisk | nice -5 \
 /usr/sbin/create_compressed_fs -b -B 65536 \
 -f /mnt-system/knx702/isotemp - /mnt-system/knx702/KNOPPIX/KNOPPIX1

5. Reboot and enjoy Flash videos in Iceweasel.

Comments and suggestions are always welcome.

[utu]

Greetings. Klaus2008.
Nice to hear from you.

Glad to see someone carry-through on Klaus K's example.
I thought his example was hard to follow, myself.
Also, it left me puzzled on how to carry-on for subsequent images.

You are using a lot of options, I'd like to know more about each of these.
In some of my prior efforts I tried to cast out anything that had Cache or cache
in its name.

I like this one of Klaus K's idea because I can do it all within the resources
of my 8 Gb SDHC. No additional system, virtual or otherwise, no additional
workspace needed, no hours-long processing needed.

As an unexpected plus, I just cloned a new 8 Gb SDHC, and would-you-believe?
The clone has KNOPPIX1, KNOPPIX2 and persistence right away, ready to go.

Anxious to hear how you progress thru KNOPPIX2, KNOPPIX3, etc.

[Capricorny]

Hello utu!

When I read Klaus Knopper's post on the mailing list I decided to try the remastering procedure from the live system myself.

I use a kind of Poor man's install of Knoppix V 7.0.2 inside a VirtualBox 4.1.16. I copied the content of the DVD to the directory knx702 and use Grub as bootloader.

I think that a typical session looks as follows.

1. Boot with "noimage" cheat code.

2. Open root terminal.

3. Issue the following commands.

apt-get update
apt-get install flashplugin-nonfree
apt-get clean

Of course, there are many more useful packages that are not included in the public version of Knoppix 7.0.2 since they are not free like unrar for example.

4. Create the archive file KNOPPIX1 from the ramdisk. The options used for makeisofs are the usual ones taken from the "Remastering howto". I use a temporary file on the virtual HDD since I do not have more than 1 GB of RAM available inside my virtual machine. I exclude some directories with files I consider not useful for the archive. Maybe there are more directories.

mkisofs -R -U -V "KNOPPIX1 filesystem" -P "KNOPPIX1" \
 -hide-rr-moved -cache-inodes -no-bak -pad \
 -x /ramdisk/var/lib/apt/lists -x /ramdisk/dev \
 -x /ramdisk/var/log -x /ramdisk/var/run /ramdisk | nice -5 \
 /usr/sbin/create_compressed_fs -b -B 65536 \
 -f /mnt-system/knx702/isotemp - /mnt-system/knx702/KNOPPIX/KNOPPIX1

5. Reboot and enjoy Flash videos in Iceweasel.

Comments and suggestions are always welcome.

Just to expand a bit on this:
1. I wonder if VMware Workstation/Player could be an even better platform for this kind of work? And Qemu is rather versatile in my experience, running KNOPPIX directly.
2. Re Utu's question about KNOPPIX2, I also wonder if rolling auxiliary images together might be a better alternative than juggling with several of them. But maybe there isn't any easy way to do this, short of complete remastering?
3. Could setting up a script, which handles the alternatives of writing /ramdisk to an image file or to a cloop, be a rather universal solution to most problems encountered running from USB sticks?
4. Is there any principal difference between cloop and squashfs in this context? I would think squashfs would be even simpler, avoiding the file system setup pipe. Or am I mistaken?

[utu]

.
This is not to take away from heretofore 'standard' re-mastering procedure(s) but to
highlight a subtle refinement in using Knoppix that accomplishes much of what one hopes to
achieve by re-mastering, or making backups of some kind. This alternative is elegant
in minimizing the resources required in re-mastering, including time.
I'm re-wording here to collapse into one post what previously spread over several posts.

Assuming for example that KNOPPIX1 was the previous compressed image update,
to stack another compressed image update with the previous, use the following magic incantation:

cd /
sudo mkisofs -R -U KNOPPIX-DATA | create_compressed_fs -B 131072 -m - - > /mnt-system/KNOPPIX/KNOPPIX2

Then reboot, using 'knoppix noimage' or to see if it all works like it did using the previous
persistence file.

If so, move knoppix-data.img from /mnt-system/KNOPPIX/ to /mnt-system/ temporarily and reboot,
this time without the 'noimage' cheatcode. This should allow you to re-establish persistence.
Then, re-verify that KNOPPIX2 is doing what if ought. When you are sure KNOPPIX2 and persistence
are ok, remove the 'old' knoppix-data.img in /mnt-system/, not the 'new' one in /mnt-system/KNOPPIX/.

You may use PCManFM to move things around, but don't count on PCManFM to remove things;
use root and rm to remove files.
init's mountunion() looks like you may have up to ten such upgrades, if you have enough space
on your LiveUSB

Recapitulation: We can have a full-DVD-size LiveUSB on an 8 Gb Class 10 SDHC device. This device
may be serially updated with compressed image additions without requiring any additional workspace;
without requiring any additional system, real or virtual, to manage this process; and without
any significant time expenditure for these performing these updates.
The convenience of temporary persistence may be maintained, while greatly reducing the magnitude
of the hazard in occasional inadvertent ruination of this 'persistent' image, since these
successive updates preserve well-tested material in more safely situated in more permanent
compressed images.

[Capricorny]

.
Recapitulation: We can have a full-DVD-size LiveUSB on an 8 Gb Class 10 SDHC device. This device
may be serially updated with compressed image additions without requiring any additional workspace;
without requiring any additional system, real or virtual, to manage this process; and without
any significant time expenditure for these performing these updates.
The convenience of temporary persistence may be maintained, while greatly reducing the magnitude
of the hazard in occasional inadvertent ruination of this 'persistent' image, since these
successive updates preserve well-tested material in more safely situated in more permanent
compressed images.

For more heavy installs. I will strongly recommend using a 16 GB stick, for in this scheme nothing is ever purged, which means, for example, that you will get several instances of the apt data, essentially one on each KNOPPIX[N] image. In my own use case, I have by now purged about 2GB of programs/data, and added about 3 GB on 7.0.2. Of course, this is "real world" use of Knoppix, and most users might do less, but if I had used this method, I would have had about 6GB total in cloop images by now, and I advise people to be defensive with regard to the space overhead of different copy/compress/backup methods.

It does not go on indefinitely, for the vast majority of uses, 16GB media will be enough. I have also experimented with >4GB persistent image, for example using several (uncompressed) image files, but I think an adaptation of the method presented here is better in most cases.
I think this is also a nice illustration of what I have come to think is the basic rule of remastering: If you don't know exactly why you need it, and why the alternatives are indequate, don't do it.

I'd like to mention one way this can bu used for (sort of) incremental backups: If you call the KNOPPIX[1|2|3...] images something else, for example knoppix-20120620.cloop, and store it at your backup location, while you continue to use the same knoppix-data.img at next boot, instead of forcing creation of a new image, you may cloop-mount this image and recreate the knoppix-data.img of that day from it. They may of course grow ever larger, but if you purge your persistent image, they may also shrink.

Myself, I make tarballs (.tgz files) of the knoppix-data.img for backups, but this way, I don't have the chance to start afresh with the persistent image without remastering - like I can if I decide to use the cloop backup as KNOPPIX1 and start a new .img file. I should add that I don't use the persistent store for much data - they are stored on a volume mounted on /store in my setup, and this volume is stable across OS versions and backed up separately. Some of it dates back to Slackware days of 1995..