Re-mastering Proof-of-Principle Effort

[utu]

That card should have ca 20MB/s r&w, I think that's fine. I would say just go ahead and try - only beware that if the PC identifies it as an mmcblk device with the built-in reader, you have to either use an external card reader, or modify minirt.gz something in the way of marlas' method, http://www.knoppix.net/forum/threads...mmcblk0-device

I may have to use 0wn to prepare the recombining system. I'll have to look closely at Werner's scripts.

As regards the handling of knoppix-data.img, if you don't have soething special rewritable to keep, you might just let Knoppix create a new one. The main reasons I recreate it upon remastering, is to keep my modified /home outside of the remastered image (I might give it to someone), to save space in the new cloop by purging package data that will be rewritten anyway, and to save one step at the first booting of the new version.

I keep small compressed backups of /home, /etc & 'syslinix.cfg instead of 'the whole thing'. These amount to only ~40 Mb. Works for me.

Squashfs vs cloop is really no big deal in our context. Squashfs image is smaller than the quick-compressed cloop, and takes one less step to create. But when we have enough space on our USB devices, and use scripts, that's not very significant. Klaus K clearly has his reasons for sticking with cloop, but I still haven't got exactly what is most important, with the present versions. I have been using about half and half of them, often not knowing which variety I'm running right now - and that's just the way I like it. I might go for more exclusive squashfs, though. It's just that I then have to modify the minirt, which I also try to avoid as much as posssible.

I feel the same way. I thought you still might be anxious over this.

[utu]

@ Capricorny, relative to 'Marlas method':

Marlas is really brave to dive into minirt.gz and change it.
I don't expect to have Marlas' problem.
I've used SD cards almost exclusively with Knoppix ever since 6.2.1 with no problem.

My portable has four USB ports: three come out as regular USB, one as a Card reader.
I also have an external Sony Card-to-USB adapter.
The portable reads all these just fine.

I do have some problems booting from an external USB hard drive,
but wasn't planning to go that way in this experiment.

[Werner P. Schulz]

I'll have to look closely at Werner's scripts.

I've improved the script kn-recombine. Perhaps it may be interesting for you. Please edit the script as you need.

Greetings Werner

[utu]

Greetings, Werner.

I noticed your site was a little different than I remembered.
I'm sure I'll have some questions for you once I get back
into this topic.

[utu]

.
This is a status report on my experience with an adaptation of Werner's kn-recombine.
kn-recombine is an excellent and generalized algorithm for re-mastering Knoppix LiveUSBs.

My adaptation seeks to further simplify the use of kn-recombine specific to my particular
use of the algorithm. My end use is re-mastering 2 Gb LiveCD-size LiveUSbs with persistence.
I choose not to utilize the ntfs fields of the Windows 7 capability but rather to perform
all the necessary manipulations via external USBs attached to my laptop. Recent cost
reductions and speed increases of USB chips has now made this choice inexpensive & viable.
For $19 one can now create a re-mastering 'appliance' for CD-size Knoppix Flash Drives.

I use a 16 Gb USB card with a Class 10 rating. This seems to be 15 Gb formatted Fat32
initially. I use Gparted to re-partition this to one 1.84 Gb Fat32 partition, and then
create a second partition with the remainder and format that to ext3. I clone my current
2 Gb LiveUSB and its persistence on the Fat32 partition and install Werner's kn-recombine
on the ext3 partition. 1.84 Gb is the net size of my 2 Gb LiveUSBs.

Working with the 16 Gb unit's LiveUSB I find Werner's algorithm will re-master my LiveUSB
in about 41 minutes. In this 'appliance' use of Werner's algorithm, when it comes time to
write to a new flash-disk, the first prompt requests the location of the re-mastered
KNOPPIX file. Don't anticipate (as I did at first) the prompt to ask rather for the
location of the flash-disk to be modified. With a minor re-do of Werner's algorithm,
I have hopes that perhaps the partition location dialog may be further simplified, and
even possibly omitted in an 'appliance' adaptation of the algorithm.

As a bonus, the appliance serves as a backup, it may be noted. Also, my 2 Gb LiveUSBs
are on Class 2 USB cards; I would expect Classes less than 10 may be much too slow
for the remastering use of such a 16 Gb 'appliance'.

[utu]

.
I note there is an 8 GB SanDisk 'Extreme' Class 10,
that is rated at 30 MB/s for $20.
This might be an even better fit for 2 GB re-mastering.

[utu]

.
My UNIONFS is about 2.3 Gb. The new stuff in KNOPPIX-DATA is probably
about about 500 Mb. Compressed KNOPPIX grows from 722 to 953 Mb in
39 minutes during re-mastering with kn-recombine.

The rsync phase of remastering with kn-recombine takes about 22 minutes
with my class 10 LiveUSB, 2.10 Ghz dual cpus & 32-bit kernel.
This is about the same as the result of cp -r UNIONFS /media/sdc2/knx.

Perhaps the rsync phase can be speeded up by first pre-loading recombine*
by expanding the un-re-mastered compressed KNOPPIX into that space before
finishing it off with an rsync againts UNIONFS. Otherwise, rsync is never
getting any benefit out of working on small differences.

I am also wondering, as another alternative to kn-recombine, if it might
be possible to just compress UNIONFS directly to compressed KNOPPIX at
/media/sdc2/knx without mirroring it first to recombine*.

* recombine is a folder (or directory) in /media/sdc2/knx in the
statements above.

[Werner P. Schulz]

If you want short times for "recombine", why don't you use a free partition on HD as I described? The use of a second flash device is a lame duck.

To use

cp -r UNIONFS /media/sdc2/knx

is not as versatile as

rsync -ax /UNIONFS/ $STOR/knx/recombine

"cp" doesn't offer the ability to exclude/include files and folders.

Perhaps the rsync phase can be speeded up by first pre-loading recombine*
by expanding the un-re-mastered compressed KNOPPIX

The un-re-mastered compressed KNOPPIX is already as cloop within the directory /KNOPPIX/.

Greetings Werner
http://www.wp-schulz.de/knoppix/recombine.html

[utu]

Thanks for your comments, Werner.

If you want short times for "recombine", why don't you use a free partition on HD as I described? The use of a second flash device is a lame duck.

I feel more comfortable NOT modifying the HD on my laptop.
The underlying Win7 system works very well, and I don't feel confident enough to repair it if I break it.
$19 provides a safer but slower means for me.

To use

cp -r UNIONFS /media/sdc2/knx

is not as versatile as

rsync -ax /UNIONFS/ $STOR/knx/recombine

"cp" doesn't offer the ability to exclude/include files and folders.

I noticed that in the excludes, but it doesn't seem you are saving many Mbs.
Is there something more important going on here?

The un-re-mastered compressed KNOPPIX is already as cloop within the directory /KNOPPIX/.

If that were de-compressed to /media/xxx/knx/recombine and then rsync-ed it would save
some time, I'd think.

My last question is, can we just compress UNIONFS directly, without making a mirror of it first?

[utu]

'
The 'lame duck' may have the potential of a self-contained
re-mastering capability.

It doesn't need external storage nor another OS to complete
the re-mastering. After the kn-recombine rsync phase,
dismount the 'duck and reboot IT with the option 'noimage'
and proceed with the kn-recombine write operation.

This may require a slight tweak of kn-recombine to allow this,
but it should work since none of the files being re-arranged
are in use on a noimage reboot.