create_compressed_fs; how long does it run on your machine?

[Steve F]

This is the story of a guy with a clean house because create_compressed_fs takes so long to run for each test boot, he got bored and did house work instead of watching TV. His girlfriend now loves him and his mother is worried.

I am remastering KNOPPIX V3.6. My dev O.S. is the V3.6 HD install multiuser with HW autodetect. I have used two different computers: a Dell L400 laptop with 733mhz cpu, 256mb ram and 20gig ide HD in three partitions, and a Dell GX300 desktop with dual 733mhz cpu's, 512mb ram and two fast scsi drives in four partitions where the remaster is on a partition on the faster drive and the CD boot partition is on the other drive.

I don't seem to get any additional benefit from with the GX300 except when I run other tasks during a compress I don't get any mouse pointer delays or programs taking 30 seconds to startup like on the L400. The GX300 never uses both cpu's at the same time but switches between them sharing the load. I get the same behavior on MS w2k3 svr. The additional ram and fast scsi drives didn't help much. This seems to be limited by cpu speed. That, or the racing stripes I painted on the L400 really do make a difference in performance.

L400 w/o the -b option takes about 35 minutes to compress the full KNOPPIX distribution.

GX300 w/o the -b option takes about 35 minutes to compress the full KNOPPIX distribution.

GX300 with the -b option takes about 5 hours to compress the full KNOPPIX distribution.

A compression without the -b option resulted in an iso size of 763000832. With the -b option the size is 742182912 for a savings of only 21mb. Doesn't hardly seem worth it unless you are so close to fitting your remaster on your media of choice and need to get that size just a little bit smaller and you can't take away any more packages.

So, what's in YOUR computer?

Steve

[kelmo]

Steve,

I use squashfs, it is ~40% quicker and gives about a 5% benefit in compression over cloop +b.

[Steve F]

If I understand correctly, you patch the knoppix kernel (which version? I am using 2.4.27 on release V3.6), then use the squash compresser on the KNOPPIX file system, create the iso and burn it to disk? That seems like a lot of up-front work and risk to a novice remasterer like me. I'm concerned that the squash docs indicate patching a pure kernel; redistros likely to fail. So far I haven't had to mess with the kernel much and prefer to not have to.

Have you heard any talk of knoppix distro switching from cloop to squash? That would be a good thing.

[mzilikazi]

That seems like a lot of up-front work and risk to a novice remasterer like me.

Why go through all the nonsense? Get Kanotix the kernel is already patched for squashfs. Do an knoppix-type install, boot into it, customize it then boot back into your primary OS.

kelmo is right- it compresses very fast and all it takes is:

mksquashfs <source> <file>
Alot easier eh? Of course you can boot the compressed KNOPPIX before creating the .iso to test it out. You need not use chroot for anything when remastering Kanotix by booting into it.

You can even append a new squashfs to an existing squashfs! Oh yeah!

[UnderScore]

Bootup from offical Knoppix 3.6 CD, using the remastering scripts .16 from Fabian. Decompress CD to HD and then immediately create compressed fs:
On a Dell Optiplex GX1 P3-450MHz 384 MB RAM ~1hour 10minutes
On a AMD Duron 1.6GHz 384 MB RAM ~25 minutes
One a dual proc AMD Opteron 248 1.4GHz 1 GB RAM ~ 15 minutes

[usulix]

Your original numbers seem about right to me.

Coppermine 900 mhz with 1 gig of RAM = 20-30 minutes.

G

[Steve F]

I looked at Kanotix. It isn't SMP and doesn't have the desktop=kiosk boot cheatcode I am using.

When I first started looking at the process of remastering, I followed the suggestion to look at other distributions that may already have the software I need or at least have many packages already removed and could be used as an easier starting point. Where that is generally true, I was stuck because I need to run on many laptops, one of which is the Dell D600. The network card is Broadcom and needs the bmc5700 driver. All the other distros I looked at were based on older knoppix releases and didn't have the driver. Only Knoppix v3.6 was able to run on the D600 with kiosk mode.

Whatever I do, I intend to always be able to start from the latest upstream release. I think that knoppix is always the upstream distribution that all the other distros are based on. Correct me if I am wrong. I set up my scripts to start from a new release of knoppix with little interaction on my part. I'm still sorting all that out. To that end I posted in the Ideas forum that knoppix be released with two iso's. One is the standard full distribution and a second iso that is just the base packages of that release needed to boot and add packages. If you aggree that having a base starting point where we don't have to spend weeks individually figuring out what packages can be removed, go over to Ideas forum and add your comments to the thread. More 'votes' may encourage this idea to become part of their standard release procedures.

Thanks, Steve

[CrashedAgain]

This is the story of a guy with a clean house because create_compressed_fs takes so long to run for each test boot, he got bored and did house work instead of watching TV. His girlfriend now loves him and his mother is worried.

I am remastering KNOPPIX V3.6. My dev O.S. is the V3.6 HD install multiuser with HW autodetect. I have used two different computers: a Dell L400 laptop with 733mhz cpu, 256mb ram and 20gig ide HD in three partitions, and a Dell GX300 desktop with dual 733mhz cpu's, 512mb ram and two fast scsi drives in four partitions where the remaster is on a partition on the faster drive and the CD boot partition is on the other drive.

I don't seem to get any additional benefit from with the GX300 except when I run other tasks during a compress I don't get any mouse pointer delays or programs taking 30 seconds to startup like on the L400. The GX300 never uses both cpu's at the same time but switches between them sharing the load. I get the same behavior on MS w2k3 svr. The additional ram and fast scsi drives didn't help much. This seems to be limited by cpu speed. That, or the racing stripes I painted on the L400 really do make a difference in performance.

L400 w/o the -b option takes about 35 minutes to compress the full KNOPPIX distribution.

GX300 w/o the -b option takes about 35 minutes to compress the full KNOPPIX distribution.

GX300 with the -b option takes about 5 hours to compress the full KNOPPIX distribution.

A compression without the -b option resulted in an iso size of 763000832. With the -b option the size is 742182912 for a savings of only 21mb. Doesn't hardly seem worth it unless you are so close to fitting your remaster on your media of choice and need to get that size just a little bit smaller and you can't take away any more packages.

So, what's in YOUR computer?

Steve

The first time I tried remastering 3.4 it took 8-12 hours with the -b option & about 2 hours without the -b & ended up with an iso of 1.1 gig from an uncompressed size of 1.8 Gig. For several reasons I did a reinstall & redid the remastering, the next time the 1.8 gig system compressed to just under 700 Meg in about an hour. I have no idea why.
System is a 1.2G Hp pavilion.
I have also concluded the -b isn't really worth the bother unless you really need it.

[444]

Dell GX270 P4 2.8 ghz , 512mb ram -> with -b takes 1hr 50 min

[Steve F]

I got my compress time down to 4 minutes, 20 seconds! Basically that's practically all packages removed. The pre compress size is 215 mb and the iso is 79 mb. The bad news is, it takes 20 minutes to reset my remaster directory and remove the packages. I guess for my class machine it will always take 25 to 30 minutes to prepare a remaster.