Why would the running knoppix partition be mounted rw?

[Albretch]

# mount | grep mnt
/dev/sda4 on /mnt-system type ext4 (rw,relatime)

# uname -a
Linux Microknoppix 3.16.3-64 #10 SMP PREEMPT Fri Sep 26 02:00:22 CEST 2014 x86_64 GNU/Linux


lbrtchx: http://knoppix.net: Why would the running knoppix partition be mounted rw?

[Capricorny]

Only Klaus Knopper can give you the correct answer, but after some weeks with Debian 8.3 live - which mounts the partition ro when there is no persistence, I think I can provide some motivation. Have you tried Debian live with and without persistence?
In short: While it can be argued that mounting a partition with a compressed image readonly is the "correct" thing to do, in practice we often lose more than we gain that way. There are usually quite a few other files apart from the compressed live image on the partition, and why should it be impossible to modify any of them, or, for example, add more compressed images to the partition while running off it? Personally, I have found ro mounting to be mostly a hassle for me. Over time, partition sizes have grown dramatically, while the Knoppix CD and DVD images have stayed essentially the same size.

As for Knoppix, ro mounting would make it impossible both to change the booting configurations - unless they are placed on another drive, and I think many would find that impractical. Maybe more important, is that we would not be able to do any administration work on any persistent image, like fixing the file system, take a backup on the same partition or resizing it, from any Knoppixes on that partition. To patch this newly created problem, add a cheatcode for "mount rw" - and get back to where we started.

I'm quite sure you could modify init in minirt.gz to add and implement a "ro" cheatcode if you really want it, shouldn't be too hard.

[utu]

(snip)
Personally, I have found ro mounting to be mostly a hassle for me.
(snip)
I'm quite sure you could modify init in minirt.gz to add and implement a "ro" cheatcode if you really want it, shouldn't be too hard.

I'm really interested in this.
I hope you will share some examples of why making persistence ro by default is reasonable,
and enlighten us on how to _easily_ tweak minirt.gz to add a ro option to Knoppix.

As you know, I take some pains to make a cloop or two to protect most of my Knoppix perstence material;
cloops having the nice feature of being not only virtually ro, but also much more compact than regular persistence.

I'd like to see a Knoppix variant which has persistence ro with a chance at shutdown either
to capture what's in the ramdisk, or stick with whatever was in persistence at boot.

atb

[Capricorny]

I haven't even looked into the last couple of versions of init in minirt.gz myself. But from the unchanged cloop mounting, I feel quite sure that making adjustments to the basic mounting procedure should not be too hard. If knoppix-data.img is present and not cheatcode noimage, then mount rw, otherwise mount ro. There is already code for detecting this in init, it just has to be applied somewhat differently.

But I am not going to do it. The principal reason for mounting ro is that whenever what you are going to use is ro, it's best to mount the partition ro. There may have come another reason recently, with linux being less filesystem-safe - but I think this is mostly anecdotal so far.
From my experience with Debian, for me, the drawbacks with mounting ro outweighs the advantages. IMHO, the most important practical advantage is for checking out a live distro version: When you just copy the squashfs to the hard disk to test a bit faster and more thoroughly, it is much safer to run it readonly. I'm not in that situation.

I'm quite with you on the idea mounting persistence ro in the aufs stack, and only update upon shutdown. I think that has become even more useful recently. But for that last update, having the whole partition mounted ro would just require an extra umount/mount step on shutdown, extra complexity with little visible gain.

[utu]

having the whole partition mounted ro would just require an extra umount/mount step on shutdown, extra complexity with little visible gain.
If that were the only consideration, then wouldn't this occur only when I want to keep new changes at shut-down?
If so, a small delay to umount and mount would seem like a reasonable trade-off to me.

Here's my current persistence file
knoppix@Microknoppix:/KNOPPIX-DATA$
sudo du -h --max-depth=2
72K ./etc # half a dozen,or so, one-line tweaks
202M ./var/lib
100M ./var/cache
301M ./var # after reloading Synaptic
5.5M ./usr/share # ttfmscorefonts for LibreOffice
5.5M ./usr
59M ./home/knoppix # my /home stuff
59M ./home
12K ./root
0 ./.wh..wh.orph
365M .
knoppix@Microknoppix:
Refering to this arrangement, one may observe... /var requires r/w capability.
ramdisk provides this until shutdown, but to retain for re-boots, needs r/w storage.
/etc, /usr /home for sure could be ro until I choose to change them.
In that event, I could put up with a small delay on shut-down.

[Capricorny]

Yes, with that kind of use, mounting the whole partition ro might be sensible. I often have a similar situation myself, because the data I work with are on other partitions. So if I had a "commit" option, where changes made temporarily are saved, I would more often work with readonly-ramdisk. Implementing such functionality is much more work than just adding a "ro" cheatcode, but AFAIK, some distros operate that way, so it should be possible to build upon the work done there.

One possible scheme (don't know how smart it is, but apart from space use, it should be fairly efficient) could be to operate with two persistent images, and mount them readonly alternatingly, saving changes to the one that was not mounted. With compressed image c and persistent images p1 and p2, we could quite easily do something like this with two aufs mounts, the compressed image being used in both:
rsync from ramdisk : p1(ro) : c -> p2(rw) : c
On next bootup, p2 will be mounted ro, and p1 will be updated. With changes in the order of tens of megabytes, this should only take a few seconds. There is also a primitive kind of backup in returning to last version.

[utu]

Yes, with that kind of use, mounting the whole partition ro might be sensible. I often have a similar situation myself, because the data I work with are on other partitions. So if I had a "commit" option, where changes made temporarily are saved, I would more often work with readonly-ramdisk. Implementing such functionality is much more work than just adding a "ro" cheatcode, but AFAIK, some distros operate that way, so it should be possible to build upon the work done there.
Here's my suggestion:

Assume persistence is a cloop or squash partition to begin-with,
and normal usage is similar to a LiveDVD rather than LiveUSB
in which case persistence is virtually read-only.
Just add noimage to current boot cheatcodes.

Provide Knoppix with three new menu choices:
1. Unsquash, mount & rsync current persistence with current unsquashed ramdisk
then cloop or squash the union of these two --> produces 'new' squashed persistence file
2. Trade old for new cloop or squash files and vice versa; the un-do/re-do persistence update.
3. Remove unnecesary cloop or squash files as a secondary, 'never mind' option.

This wouldn't require any nagging at shut-down, and would only mean relatively small additional
changes to minirt.gz and/or init.

Another alternative for menu Choice 1 might rather be:
1. Squash current ramdisk and 'merge' this with current squashed persistence, using
overlayfs. There is a Ubuntu precedent for this.

In both cases, persistence would take up much less room than it does now.
Switching from cloops to squashfs and overlayfs would also bring Knoppix closer to
everyday Debian and Ubuntu, in terms of building blocks.

[Capricorny]

Because of trouble with cloop handling, I'm going to make a squashfs version of 7.6.1 and try to run form it.
Old-style cloop remastering doesn't seem to work. Mounting manually and inspecting, there didn't seem to be anything wrong with the cloop remasterings of 7.6.1, but for some reason, Knoppix wouldn't mount them at startup. Have just started looking into 7.6.1 minirt init, though.
Maybe we should try with overlayfs too, while we are at it? But I know absolutely nothing about it, and I am hardly able to handle aufs - it hasn't been necessary with more than the mere basics.

I think you should try to implement your ideas, with some luck, I might be able to make a minirt_sq_ovl.gz that you could start from.

[Capricorny]

Just have to correct myself: Just discovered a missing file stopping findknoppix() in init, remastering works as usual. Just copy all surrounding files over to the directry with the remastered KNOPPIX image - that hasn't been necessary before, and there is just one file needed now, but just to be sure... Still, for more fleixble use, squashfs may be the way to go. I also had a lot of problems with cloop in standard Debian - it is kind of outlawed.

[utu]

.
I agree that Knoppix might be easier to hack if cloops and aufs
were replaced with squashfs and overlayfs. These seem to be where the action is going these days.

You might want to google 'mergeusb' and see how Dav Stott & others are handling remastering.
I had some luck with Mint 17.2 using mergeusb, but my Mint 17.3 has a low-memory corruption problem
I can't solve.

I recommend mergeusb as probably a good example of how to code overlayfs and squashfs
instead of cloops and aufs to combine filesystems. I wouldn't necessarily just
plop all his code in and hope for the best; just refer to his overlayfs code to see how easy it
is to set up.

[Capricorny]

I think this might be tried:

1. On startup, if persistent not empty, copy persistent content to ramdisk, run with ramdisk. Otherwise, swap KNOPPIX.sq and KNOPPIX_b.sq, run from KNOPPIX.sq
2. During running, use two (script) commands: ramdisk_save and ramdisk_commit. ramdisk_save saves to a overlayfs mount of knoppix-data.img and KNOPPIX.sq, unmounting afterwards.
3. ramdisk_commit first performs a ramdisk_save, then compresses the overlayfs mount to KNOPPIX_b.sq, unmounts and clears knoppix-data.img.
4. Unless otherwise requested, KNOPPIX_b.sq will be renamed to KNOPPIX.sq and used at next startup, the old KNOPPIX.sq becoming KNOPPIX_b.sq and usable as a backup until next commit.
5. Cheatcode revert either aborts the swapping after a commit, or works with the oldest squashfs image available.

This way, we never use the persistence for daily work, we just save to it. There is not much complexity added to minirt init either: Copying and possibly image swapping, using noimage setup. With the revert cheatcode, KNOPPIX.sq and KNOPPIX_b.sq are swapped, if KNOPPX.sq is not already the oldest.

With FAT32 limits, this could be done within ca 12GB space if mksquashfs can use a pipe. Otherwise, ca 12 GB extra is needed for filesystem copy. When a 32GB USB3 stick costs about $20, I don't think an extra 16GB is that much to care about. I would guess ca 2GB persistent store would be fine with this setup, and starting from DVD Knoppix slimmed down to ca 3GB (example of purging procedure provided elsewhere), the should be enough workspace for many needs. Even when the least efficient squashfs compression is used.

And using NTFS or ext3 file system, the 4GB limit does of course not apply.

[utu]

@ Capricorny

We are about on the same wavelength, but I think for my purposes, that
I don't need a full re-mastering which I think your last list implies.
I have in mind a 'partial' re-mastering, which would just be updating
just one read-only persistence squash file.

Take Knoppix 7.7 for example. Klaus uses three cloops for his stuff:
KNOPPIX aka KNOPPIX0 is a full 4 Gb cloop of Knoppix DVD material;
KNOPPIX1 is a small 'overflow' cloop which currently has only a few
3d-oriented programs; and
KNOPPIX2, a really small cloop, just for proprietary flash for the CeBIT crowd.

I would co-opt the 7.7 KNOPPIX2 cloop as my read-only persistence receptacle,
doing a re-master with current & future new read-write persistence
whenever I chose to commit newer r/w persistencee stuff to r/o.

Re-mastering just KNOPPIX2 with r/w persistence material would take up
much less real estate to accomplish and the final result would certainly
be more efficient than strictly r/w persistence alone. I would sacrifice
the capability to remaster to delete material, but all-r/w persistence
already does that. I expect my 16 Gb Cruzer LiveUSBs would have room
to spare doing such a 'partial' re-master.

As a second act, I am fascinated by the idea that re-mastering might
be achieved by working with squashed material rather than unsquashed
material which mergeusb purports to do.

atb

[Capricorny]

If you never do any huge upgrade or purge, working with incremental changes in overlays is IMHO perfectly fine. But you'll soon get something that takes up more space than necessary, and with a full 4GB squashing taking just a few minutes (3-10 for example), I don't really understand people being so afraid of it. In particular when it can be performed as a background process, as in this case - you don't need to notice it much. You just issue the command ramdisk_commit & in a terminal and continue with whatever you were doing. Also, the image and persistence handling is taken care of by minirt init.

If mergeusb really works with squashed material all along, it will surely be possible to do the same. But I didn't get that impression. Everything just goes so fast when you are working with small images.

And I'm still curious as to what forces you to use 16GB sticks in 2016.. :-)

[Capricorny]

Resquashing may surely take quite a bit more time with slow sticks - but, then, a small 120GB SSD costs about $50, and yesterday I did full Knoppix remastering in a kvm VM on a 5 year old i5 laptop, using an external SSD drive for copying - still < 30 min, and much of the time and space was used for creating isofs for the new cloop.

[utu]

I don't really understand people being so afraid of...(remastering, in essence).

Looking at mergeusb bash, I interpret that it loop mounts two squash files,
one ro and one rw initially, and uses overlayfs to produce a merged file
which it then squashes as a candidate new ro file, then proceeds to reformat
the now empty rw file. This all takes place on a temporary directory which
is erased on completion. It does not appear to me on closer inspection
that the magic is accomplished at the squashed level after all. The main
difference being that squash files replace cloops and overlayfs replaces aufs.

This sure sounds like plain old re-mastering to me. At some point in the process,
there has to be at least temporary storage for a complete un-squashed
replica of the squashed material, in addition to all the squashed material,
plus a complete ram-disk operating system.

It is not fear which repulses me from this situation, it is the mindlessly
uneconomic use of resources. Bloat is repulsive to me. Two Gb is enough
resource to accomplish a generous allotment of applications and enough
persistence to satisfy my additions and tweaks. My 16 Gb USB3 Cruzers cost
$7. I don't know if you can even buy 2 Gb SanDisk SDHCs anymore, but 2 Gb
is enough for my computing needs. With 14.48 Gb net to start with, GParted
says my 7.7 Cruzer still has 9.74 Gb unused. I'd go for some scheme that
could work some magic with that unused 9.74 Gb.

Having five Gb devoted to Knoppix's read-only part also repels me, since
I know it can be done in CD size, but I we haven't found an alternative
distribution which has Klaus K's magic sauce, so I tolerate its obesity.
Klaus does provide almost half-a-dozen unused cloop assignments, which he
himself is now using with an atrocious lack of economy. One whole cloop
for flash staggers my feeling for economic use of resources.