Results 1 to 3 of 3

Thread: Remastering Knoppix7+: New uses of old overlay techniques

  1. #1
    Senior Member registered user
    Join Date
    Sep 2006
    Posts
    802

    Remastering Knoppix7+: New uses of old overlay techniques

    The 4GB file size limit of FAT32 has been an important premise for KNOPPIX (and similar live distros) in the past, and it still is the main reason why contemporary KNOPPIX comes with two cloop-compressed files: The basic KNOPPIX, and the overlay KNOPPIX1. Both sized to fit on a FAT32 file system.
    When we remaster, we can easily put everything into one file, but it has gotten increasingly harder to purge enough from the DVD package list to make everything fit within 4GB, when own additions are included.

    Experimenting a bit with 7.7.1 DVD remastering, it turns out that while it is fairly easy to purge enough to compress the DVD content into 4GB without any overlay, it is probably not worth it to shoehorn significant added packages into that format.

    And still, there are some good reasons to conform to FAT32. The most important is booting. While it is easy to prepare FAT32 media to boot almost everywhere, too often other kinds of media fail to boot somewhere. So for me, it's not worth the effort to do things otherwise. Also, this seems like an easy basis for running in Vmware Workstation virtual machines.

    This takes us back to the good old multiple overlays technique, which our friend utu has championed on this forum:
    http://knoppix.net/forum/threads/306...x-7-2-LiveUSBs

    The principle is that the persistent store is cloop-compressed into an overlay as KNOPPIX<n>, n=1,2,3..., put into the KNOPPIX directory and automatically aufs-mounted upon next boot. This isn't something I would recommend to do several times. There is, for example, no file deletion involved, old versions of files are not overwritten, but just superseded by newer versions in the aufs mount. In order to have "real" file deletion, remastering is needed.

    But it is possible to make Knoppix into a real power package this way. For example, running on ext3 one could make a 8GB overlay file containing lots of programs, like databases, statistical packages etc. Then compress it into a KNOPPIX1 ca 4GB overlay (assume we have purged and compressed the DVD contents into 4GB KNOPPIX), create 2-4 knoppix-data.img<n> overlay files in addition to knoppix-data.img and stuff everything into a 32 GB USB stick which can be booted "almost everywhere". For my own use, I'm quite sure it will suffice with a ca 2GB overlay. There is, for example, little to be gained by compressing JAVA .jar/.war files, as they are mostly already compressed.

    Here is the overlay creation function utu posted:
    Code:
    #utu's overlay creation
    #http://knoppix.net/forum/threads/30672-Using-Overlay-Updates-in-Knoppix-7-2-LiveUSBs
    
    MakeOverlay() { # Cloop-compress & save /KNOPPIX-DATA as /mnt-system/KNOPPIX$1
        ISO=KNOPPIX$1; ISODIR=/mnt-system/; ISOSRC=KNOPPIX-DATA
        echo -e "Creating "$ISO"; patience, this may take some time..."
        cd /; sudo mkisofs -x *[Cc]ache* -R -U \
               $ISOSRC | create_compressed_fs -B 131072 -m - - > $ISODIR$ISO
        echo ".done; "$ISO" additions from $ISOSRC saved to "$ISODIR"."
    }

  2. #2
    Senior Member registered user
    Join Date
    May 2006
    Location
    Columbia, Maryland USA
    Posts
    1,631
    .
    Nice to hear from you again, Capricorny.
    Thanks for the nice mention. A few notes come to mind on this topic:

    1. I still use this technique on a Knoppix 7.7.1 LiveUSB.

    2. In my case, I usually try to bundle all my own usual tweaks in a single
    first additional compressed overlay and then try to operate with a new modest
    reiserfs read/write persistence file. The compressed overlay is read-only and
    virtually immune to being written-over. The net contents of a new read/write
    persistence file is usually relatively small and it's therefore easy to erase
    (re-format) and to restore its contents from notes or (my own) memory if it
    inadvertently gets written-over and Knoppix operation is obviously corrupted.

    3. I'm sure this same idea will work to some degree with other Linuxes, such
    as Xubuntu, with minor adaptations such as adding a persistence partition and
    replacing cloops with squash files. This provides an avenue to starting with
    a much smaller compressed core Linux file than that of Knoppix's DVD editions.

    4. In addition-to or instead-of "utu's overlay creation",
    the quoted function definition, MakeOverlay, appearing by itself, should include
    the notices: "Copyright TJ Hoye 2013, License GPL Version 2".
    The script is my interpretation of the content of a debian-knoppix mailing list
    post by Klaus K describing how he prepares Knoppix images to include material
    in addition to a working version of Knoppix LiveDVD for special occasions.
    The GPL License notification is typical of Knoppixes of a certain era.

  3. #3
    Senior Member registered user
    Join Date
    May 2006
    Location
    Columbia, Maryland USA
    Posts
    1,631

    Introducing snaps

    .
    In reference to my previous mention of adding squash files in Xubuntu
    in item 3 above...

    Canonical is introducing a new feature for adding programs to Linux systems
    called snaps. This feature brings into being a whole new infrastructure
    which is somewhat reminiscent of stacking compressed knoppix overlays.
    Snaps, in effect stacks squashed file systems representing useful program
    additions which bring their dependencies with them. By stacking, I mean
    combining filesystems by means of a union filesystem. This infrastructure
    includes a means of creating such snap packages, called spapcraft, and a
    concept of 'stores' for obtaining useful snap packages or for offering some
    packages you might create yourself for others to use.

    Canonical introduced this system into Ubuntu, Debian and many of the more
    popular Linux systems in 2016. It is contained in Xubuntu 16.10, for example.

    You can get a feel for snaps with the following video:
    https://www.youtube.com/watch?v=MCrT_AhCOYk

    If you are interested in learning more about snaps, you might begin with
    the following introduction and its references:
    https://tutorials.ubuntu.com/tutoria...c-snap-usage#0

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Supermicro 6028R-E1CR24N 24x LFF- Wholesale Build Your Own Storage Server picture

Supermicro 6028R-E1CR24N 24x LFF- Wholesale Build Your Own Storage Server

$249.99



Supermicro SYS-7048-GR-TR X10DRG-Q 2x E5-2697A v4  512GB 8x 3.5 Trays 2x 2000w picture

Supermicro SYS-7048-GR-TR X10DRG-Q 2x E5-2697A v4 512GB 8x 3.5 Trays 2x 2000w

$969.99



HP ProLiant MicroServer Gen 8 Pentium G2020T 2.5GHz 10GB RAM – No OS, No HDD picture

HP ProLiant MicroServer Gen 8 Pentium G2020T 2.5GHz 10GB RAM – No OS, No HDD

$249.99



HPE ProLiant MicroServer Gen10 Plus v2 Ultra Micro Tower Server - 1 x Intel Pent picture

HPE ProLiant MicroServer Gen10 Plus v2 Ultra Micro Tower Server - 1 x Intel Pent

$551.66



SuperMicro 6018U-TR4T X10DRU-I+ 4-Bay 2x E5-2680v4 32GB RAM 1U Server picture

SuperMicro 6018U-TR4T X10DRU-I+ 4-Bay 2x E5-2680v4 32GB RAM 1U Server

$138.00



SuperMicro Server 505-2 Intel Atom 2.4GHz 8GB (2X4GB) RAM 5018A-FTN4 PF Sense picture

SuperMicro Server 505-2 Intel Atom 2.4GHz 8GB (2X4GB) RAM 5018A-FTN4 PF Sense

$174.98



Supermicro 1U Server with Intel Atom CPU  picture

Supermicro 1U Server with Intel Atom CPU

$65.00



HP ProLiant MicroServer Gen8 Server Xeon E3-1220L @ 2.2 GHZ, 8GB RAM No HDD's picture

HP ProLiant MicroServer Gen8 Server Xeon E3-1220L @ 2.2 GHZ, 8GB RAM No HDD's

$229.99



2U 12 Bay SAS3 SuperMicro Server 6028U-TR4T+ W/ X10DRU-i+ Barebone 12 Caddy RAIL picture

2U 12 Bay SAS3 SuperMicro Server 6028U-TR4T+ W/ X10DRU-i+ Barebone 12 Caddy RAIL

$199.00



Supermicro 1U Server X9DRW-iF 2x E5-2680 v2 2.8ghz / 128gb / 8xTrays / 2x 700w picture

Supermicro 1U Server X9DRW-iF 2x E5-2680 v2 2.8ghz / 128gb / 8xTrays / 2x 700w

$229.99