Making use of update*.tar.gz

[utu]

For the record:
...from CHEATCODES AND HINTS FOR KNOPPIX V6.4 (last update: 28.01.2011)
(quote) If you place an update*.zip or update*tar.gz file on the medium
holding the KNOPPIX data, it will be unpacked onto the overlayed filesystem
before starting "init", thus allowing quick reconfiguration of the
system. (unquote)

There is a verifiable capability of Knoppix 6.4.4 to accept additions and
modifications via compressed format code input. These additions and modi-
fications accumulate and are assimilated into the persistent image,
knoppix-data.img.

Reading from about line 738 of the init of minirt.gz for Knoppix 6.4.4, one
may confirm that a file such as update*.tar.gz is scanned-for and will be
admitted for subsequent processing.

What follows is an experiment which may be duplicated which demonstrates a
useful application of this capability.

I have a 2 Gb SDCard which acts as a Live Knoppix 6.4.4 USB, made from a
menu choice on a Knoppix 6.4.4 LiveCD. On first boot, this LiveUSB offered
1181 Mb as the maximum allowed space for persistent storage. I chose all
of that. This choice precluded adding any significant additional code on
the LiveUSB, even if some few files were to be removed to make room. Over
a few months, this initial LiveUSB has accumulated a number of 'refinements'
which I cherish. These are accumulated in its knoppix-data.img.

I intend to show how to 'teach' a second similar SDCard to be (almost) as
smart as the first card in a few easy operations.

To begin, I prepare a second SDCard exactly as the first by using the same
LiveCD menu preparation. On first boot of the second LiveUSB, I choose only
800 Mb for persistent store, leaving 400 Mb unused on this SDCard. This is
space we will needed for update*.tar.gz.

So then I fire up the first smart LiveUSB and do the following:

cd /
sudo tar -cf /tmp/update1.tar KNOPPIX-DATA
cd /tmp
sudo gzip update1.tar

This produces update1.tar.gz which I move to a third usb.

I shut down the smart LiveUSB and start up the new second LiveUSB. Next
I copy update1.tar.gz from the third usb to the same directory that its
knoppix-data.img is on, and reboot the second LiveUSB again.

This boot will tell you update1.tar.gz has been recognized. This boot will
also take a little longer than it did before. You will notice now that the
second LiveUSB is a lot smarter, having a lot of the characteristics of the
first LiveUSB. It is also a little slower to boot and a little fatter.

Here's what you might not anticipate. With the second LiveUSB still live,
DELETE update1.tar.gz, that's right, delete it, then reboot. You should
find that all or most of what update1.tar.gz had to impart to the second
LiveUSB was captured in the second LiveUSB's knoppix-data.img via the
process outlined. Note we now have 400 Mb to play this game with again.

Here's how well this experiment went for me:

GUI Software:
Compiz works, all changes retained.
IceWeasel works, all changes retained.
LibreOffice spreadsheet works, all changes retained.
LXDE works, all changes retained.
Synaptic works, repo (maybe all) changes retained.
...not sure what all to look for.

Infrastructure:
wifi works, all changes retained.
sound works, all changes retained.
mc & locate (two added programs) were retained & work.

System file edits:
Edits to /etc/X11/Xsession retained.
Edits to /etc/syslog-knoppix.conf retained
Edits to /mnt-system/boot/syslinux/syslinux.cfg LOST
...the only item not transferred.

Here's how the two LiveUSB's compare:

Each goes from start-of-boot to on-line-with-wifi in 1m45s.
Each is 244 Mb smart in knoppix-data.img.
The second LiveUSB has 50 Mb more free space;
...this difference is unexpected and unexplained.

[dinosoep]

didn't knew about that, great tip

[utu]

Thanks, dinosoep.

[utu]

Here's how well this experiment went for me:

Note, for perspective here, to reconstruct my favorite LiveCD from my
...Changelog notes 'by hand': 'ALL' manual changes might take hours;
...'All' manual changes: 10's of minutes; 'all' changes: just minutes.

GUI Software manipulations:
IceWeasel works, ALL changes retained; includes
...HomePage def'n, add-ons & deles, mail, passwords, bookmarks, history.
LXDE & PCManFM work, ALL changes retained; includes
...Desktop redefinition & tweaks to PCManFM.
Synaptic works, All changes retained; amounts to
...de->us repos, 2 programs removed, 2 added.
Network Manager & Broadcom wifi, All changes retained; includes
...SSID, WEP key, keyring password, Broadcom wl driver.
Compiz works, all changes retained.
LibreOffice spreadsheet works, all changes retained.

Command line tweaks:
Alsamixer, all edits retained
/etc/X11/Xsession, all edits retained.
/etc/syslog-knoppix.conf, all edits retained
/mnt-system/boot/syslinux/syslinux.cfg,' all edits LOST
...the ONLY item not transferred, since /mnt-system is not in KNOPPIX-DATA.

Here's how the two LiveUSB's compare:

Each goes from start-of-boot to on-line-with-wifi in 1m45s.
Each is 244 Mb smart in knoppix-data.img. Actually most of this 244 Mb
...is for an empty but necessary filesystem, and the bare essential files
...therein which any system needs to operate.

[Forester]

Hi utu,

Figured out what's going on here yet ?

While Knoppix is running, you've got a unionfs that comprises two halves - the read-only /KNOPPIX and the read/write /KNOPPIX-DATA. When you unzip your update tar file, where do you think the files in it go ? Into the empty /KNOPPIX-DATA of course and the backing store for that in your knoppix-data.img. So you end up back where you started. I don't think that is quite what you were trying to do.

You expressed surprise that you could delete you update tar file after first use. Actually you need to do this. Suppose you've tweaked Iceweasel and you've backed those tweaks up in your update tar file. First reboot, Knoppix will unpack the files in your update tar file so restoring your tweaks. Now suppose you tweak Iceweasol some more. You reboot without generating a new update tar file and without deleting the old one. Oops, Knoppix will unpack the files ... most probably obliterating your new tweaks. I imagine this would quickly become tedious.

Potentially worse is that you save to your tar file /KNOPPIX-DATA but Knoppix restores to /UNIONFS. Suppose you delete a package. It's not really gone. The UNIONFS writes a bunch of 'white out' files to the UNIONFS. When you tar up /KNOPPIX-DATA, you tar up all the white out files and any other hidden control files put there by aufs. When Knoppix applies your update tar file, it try to apply the white out files as if they were ordinary hidden file of no significant to aufs. A source of instability in the file system ? I don't think so, I think aufs will refuse to write such files to the UNIONFS and you'd get a bunch of error messages. The result is the deleted package reappears but may well be broken. This too could get tedious. As dinosep will tell you, deleted files should stay deleted.

Anyway, have a look at this post: http://www.knoppix.net/forum/threads...ersistent-data. This is not quite what you've been trying to achieve but it might be good enough for now.