Re: Knoppix 5.1

[Aleu]

Can somebody explain me the following:

I am trying to remaster Knoppix CD-version. I have removed kde-i18* files and some other stuff emptying around 280 MB of space. I have added only around 20MB of data afterwards. When I try to rebuild the image using the following commands:

mkisofs -R -U -V "KNOPPIX.net filesystem" -P "KNOPPIX www.knoppix.net" -hide-rr-moved -cache-inodes -no-bak -pad /mnt/hda4/knxsource/KNOPPIX | nice -5 /usr/bin/create_compressed_fs - 65536 > /mnt/hda4/knxmaster/KNOPPIX/KNOPPIX

(It informs me that around 1940MB of data was compressed)

cd /mnt/hda4/knxmaster

mkisofs -pad -l -r -J -v -V "KNOPPIX" -no-emul-boot -boot-load-size 4 -boot-info-table -b boot/isolinux/isolinux.bin -c boot/isolinux/boot.cat -hide-rr-moved -o /mnt/hda1/XXX/knoppix.iso /mnt/hda4/knxmaster

The output ISO file is around 730 MB! How the heck is this possible? Is Klaus using some additional parameters with the mkisofs command to achieve better compression or what? Please advise.

Thanks

[JohnnyH]

I've just found the same thing re-mastering the 5.1.1 CD.
mkisofs has been forked (I've not yet found an explanation why ), and mkisofs etch is now a transition package that installs genisoimage; /usr/bin/mkisofs is now a symbolic link to /usr/bin/genisoimage.
DELETION 15:26 hrs GMT : Unless/until we are able to use genisoimage so as to achieve the former levels of compression, I am going to have to boot-up an older version of Knoppix (or any other suitable system) to perform this part of remastering. END-OF-DELETION.

ADDENDUM 15.26 hrs GMT: Having just re-compressed with the version of mkisofs in Knoppix 3.7 ('Old Reliable') exactly the same happens; the new compressed KNOPPIX file is larger than the original, even though the uncompressed system had been reduced in size. (And the compressed file size produced by the old mkisofs command was the same as that produced by genisoimage. Remark: genisoimage was noticeably faster.)

(And yes, I had un-mounted /proc in both cases.)

So, the cause is not with genisoimage, but there is something about the Knoppix 5.1.1 system whereby the commands in the long-established Re-Mastering Howto cannot re-compress it to the same level.

On with the investigation ...

[JohnnyH]

Further investigation; some more figures from my system first to set the scene:-

Size of original KNOPPIX file as reported by Konqueror : 690.4 MB (723 905 561 B)

Size of original, unpacked, KNOPPIX Folder, ~2.1 GB ; as reported by du -s 2157964

After removing and substituting some packages,
size of unpacked KNOPPIX Folder, as reported by du -s 2128592

So, reduction in size of KNOPPIX Folder ~ 29 MB

Re-compressing the Knoppix folder, producing KNOPPIX File (size as reported by Konqueror):-

Knoppix 5.1.1 genisoimage with 'create_compressed_fs' compressed 1945 MB extents to 716.4 MB
(taking less than 1 hour on my Pentium 3 system)

Knoppix 3.7 (old)mkisofs with 'create_compressed_fs' compressed 1945 MB extents to 716.3 MB

Knoppix 5.1.1 genisoimage with 'create_compressed_fs --best' compressed 1945 MB extents to 715.9 MB
(genisoimage running estimate was ~ 6 hours; I left it overnight & went to bed!).

Interesting how Aleu's 260 MB reduction of original size led to '1940 MB' to be compressed,
yet my 29 MB reduction to the original led to '1945 MB' to be compressed,as reported by genisoimage.

This is markedly non-linear and such behaviour did not occur when re-mastering older versions of KNOPPIX.

So it seems not to be related to mkisofs or genisoimage in themselves, but some peculiarity associated with the more recent version of KNOPPIX.

So, there's a bit of 'negative' information, but not yet a solution ...

[tr]

Can somebody explain me the following:
Is Klaus using some additional parameters with the mkisofs command to achieve better compression or what? Please advise.

I'm not sure what Klaus has done (not trying to remaster 5.1 version) but perhaps this http://lists.debian.org/debian-knopp.../msg00009.html gives some ideas what canbe done to reduce the size of remastered Knoppix.

-tapsa-

[JohnnyH]

tr - thank you for that very useful link.

So, as now used by KK I doubled the block size to 131072 . This gave some improvement:-

Knoppix 5.1.1 genisoimage with 'create_compressed_fs' compressed 1945 MB extents to 716.4 MB
(taking less than 1 hour on my Pentium 3 system)

Now:-
Knoppix 5.1.1 genisoimage with 'create_compressed_fs - 131072' compressed 1945 MB extents to 707.9 MB
(taking only ~25 minutes on my Pentium 3 system).

So there is a useful improvement. However, my compressed KNOPPIX file (707.9 MB), generated from a reduced system, is still larger than the KNOPPIX file (690.4 MB) of the original full KNOPPIX 5.1.1.CD.

So there is still something waiting to be explained.

My reduced system is already clean in the apt cache, so to get the ISO image below 700MB I will have to purge some more packages.

By the way, genisoimage output the message :-

Warning, number as file string found. Assuming old command syntax and choosing compatible parameters (-m -B 131072). See the usage info (-h) for better/correct parameters.

[Prometeus]

Installing packages many logs and docs are created

Try this chrooted,copy and paste


If we need the space,

rm -f /var/lib/apt/lists/*Packages
rm -f /var/lib/apt/lists/*Release
rm -f /var/lib/apt/lists/*Release.gpg
rm -f /var/lib/dpkg/*old
rm -f /var/cache/debconf/*old
rm -f /var/cache/apt/*bin

optional

rm -rf /usr/share/doc/*

[bigbrother]

Hi @ ALL

I don't know if my solution could be your solution


I just modified the mkisofs part (genisoimage) as follows:

mkisofs -R -l -U -V "KNOPPIX.net filesystem" -publisher "KNOPPIX www.knoppix.net" -input-charset ISO-8859-15 -allow-leading-dots -hide-rr-moved -cache-inodes -no-bak -pad $KNX_PATH/knxsource/KNOPPIX | create_compressed_fs - 131072 > $KNX_PATH/knxmaster/KNOPPIX/KNOPPIX

I ignore message like:

Assuming old command syntax and choosing compatible parameters (-m -B 131072)


After all now I'm able to generate fully functional knoppix cd's

bye

bigbrother

[JohnnyH]

Thanks - what size was the Knoppix source system that you were compressing to achieve CD size compatibility ?

I measured my system size by :

du -s /<whatever-path>/source/KNOPPIX

After purging several packages source size was :
2027948 (1855 extents reported by compression program)

which was compressed to KNOPPIX file of size :
674.0 MB (706,786,577 B) as reported in Konqueror.

I then repeated the compression using the differing parameters as per your posting:
Added: -l
Added: -input-charset ISO-8859-15 -allow-leading-dots
Removed from just after the | : nice -5

This time the KNOPPIX file was compressed to :
674.0 MB (706,786,388 B) as reported in Konqueror.
... which is almost exactly the same.

From which I infer that some different compression method must be in use for the original generation of Knoppix CDs.

[knopx4me]

Did any body find out how to solve this?

[GrantM]

This has been messing with my head as well. Why couldn't I copy out the contents of /KNOPPIX and then compress it back down to make a file the same size as the one on the CD. No additions and no removals. I started out using the "standard" Knoppix remastering command:

mkisofs -R -U -V "Knoppix FileSystem" -P "Knoppix" -hide-rr-moved -cache-inodes -no-bak -pad sourc
e/KNOPPIX | nice -5 /usr/bin/create_compressed_fs - 65536 > master/KNOPPIX/KNOPPIX

The "-b" or best option for create_compressed_fs definitely will give you added compression. It takes longer, appearently it attempts 10 levels of compression on each file? and then chooses the one that is best for that particular file. And so on, very time consuming. However, as we have all found out, it didn't give me a file the same size as I expected.

I then read in these forums about using a larger block size for the create_compressed_fs command and this also gave me a file smaller than the one generated by the command above.

When I combined the two it gave me something I could live with:

mkisofs -R -U -V "Knoppix FileSystem" -P "Knoppix" -hide-rr-moved -cache-inodes -no-bak -pad sourc
e/KNOPPIX | nice -5 /usr/bin/create_compressed_fs -b - 131072 > master/KNOPPIX/KNOPPIX

Using "ls -l" but just showing the results:
original KNOPPIX file from CD
-r--r--r-- 1 root root 723905561 Jan 3 19:50 /cdrom/KNOPPIX/KNOPPIX

created using "create_compressed_fs -b - 131072"
-rw-r--r-- 1 root root 730236079 Apr 7 02:07 /media/hda3/master/KNOPPIX/KNOPPIX

for comparisons sake this is the ls on the file created with the "standard" remastering command at the top of my post:
-rw-r--r-- 1 root root 761611339 Apr 7 10:47 /media/hda3/master/KNOPPIX/KNOPPIX.65536.no-b

I feel I can now move on and really start with this whole remastering of Knoppix.