About cloop and RAM

[Frunktz]

Hi at all.
Little doubt...

The cd of Knoppix contain a compressed cloop file (KNOPPIX), which has the size about 700 MB.
Well, when you boot with cd, the cloop image is mounted and if you type "df -h" you can see the /dev/cloop use about 1.9 GB.
Now my question is: where /dev/cloop can find those memory? I have 256 mb of RAM...No swap with live cd.
And where decompress on the fly the program that I need to use?
For example I open some big program XYZ that uses all my RAM memory...How I can open that program if all memory is used?

Thanks a lot
Sorry for my bad bad English...

[Harry Kuhman]

The cloop files stays on disc unless you use the toram cheat code (and have the memory to support it).

This is further complicated by the unionfs file system, which you may want to look into. This, as I understand it, combines the read-only CD and the Ramdisk into one file system, the end result being that you seem to be able to make changes to the r/o system.

[Frunktz]

The cloop files stays on disc unless you use the toram cheat code (and have the memory to support it).

Ok, the KNOPPIX file (700 mb), is mounted on /dev/cloop, that use about 1.9 GB..
My question is..Where it can find 1.9 GB?




the end result being that you seem to be able to make changes to the r/o system.

For Unionfs I have no question, it's quite simple to understand..
Maybe I don't understand how Unionfs can decide where write a file in this case:
- /dir1 = writable
- /dir2 = writable
- /dir3 = writable
- /home = UNIONFS(/dir1, /dir2, /dir3)
If I type
/home# touch tempfile

Where the file is write? dir1? dir2? dir3? All three directory are writable :?

[Harry Kuhman]

Ok, the KNOPPIX file (700 mb), is mounted on /dev/cloop, that use about 1.9 GB..
My question is..Where it can find 1.9 GB?
It would be nearly 2 gig if uncompressed, but it stays compressed (even if you use the toram option). It just sitting out on the CD in that 700 meg CD-ROM in compressed form. But the cloop software makes it look like a 1.9 gig set of files to the file system. Neat, eh?


For Unionfs I have no question, it's quite simple to understand..
Maybe I don't understand how Unionfs can decide where write a file in this case:
- /dir1 = writable
- /dir2 = writable
- /dir3 = writable
- /home = UNIONFS(/dir1, /dir2, /dir3)
If I type
/home# touch tempfile

Where the file is write? dir1? dir2? dir3? All three directory are writable :?
I'm a long way from understanding union yet. Maybe someone else will join in.

[firnsy]

As far as I understand it, the unionfs directories are unioned with priority, the highest priority is the directy that is first written to.

In the case above if you were to type "mount" and saw for example:

/home unionfs opts=(/dir1, /dir2, /dir3)
Then after the "touch tempfile", the file would exist in the /dir1 directory

[Frunktz]

Ok for the union, thanks firnsy!

@ Harry
Ok, the image it stays compressed.
When a program is executed, a part of that is decompressed..in the ram? And if the ram isn't enought?
Other question, if the image is decompressed on the fly when requested, why "df -h" show 1.9 GB on /dev/cloop?

Thanks a lot

[Harry Kuhman]

When a program is executed, a part of that is decompressed..in the ram? And if the ram isn't enought?
Yea, it decompresses the program and runs it in RAM. If you don't have enough RAM and you have a swap file or swap partition it can use that as virtual memory, otherwise it likely can't run (it could keep swapping in parts of the code on from the CD, since the code doesn't change during the run, but as far as I know that doesn't happen).


Other question, if the image is decompressed on the fly when requested, why "df -h" show 1.9 GB on /dev/cloop?
Magic. Either that or 'cause that is how big the file system thinks it is, since it's being fooled by the cloop code; either answer is as good as the other.

[Frunktz]

Uhm, for the decompression..Knoppix use about, for example, 20% of RAM (128 mb for example), this 20% is used for decompress on the fly the requested program..I think..
But I think again that decompress KDE environment with KDevelop, plus X, and some other apps need more than 30 mb of RAM..Am I wrong?



Magic. Either that or 'cause that is how big the file system thinks it is, since it's being fooled by the cloop code; either answer is as good as the other.

SOrry but my English is very poor :D can you tell me more info about that?
If I understand..df -h show 1.9 gb because the cloop module know exactly how big is the decompressed image..In realty the 1.9 Gb are fake...Maybe...

[DirkS]

Uhm, for the decompression..Knoppix use about, for example, 20% of RAM (128 mb for example), this 20% is used for decompress on the fly the requested program..I think..
But I think again that decompress KDE environment with KDevelop, plus X, and some other apps need more than 30 mb of RAM..Am I wrong?

No, you are (maybe) right. But you don't need this 30MB all the time! The trick is that if something is read from the CD, there is no need to read the whole CD (that contains about 700MB of compressed data, this would be the 1.9GB (uncompressed) you see with 'df') - it just uncompresses a small piece of it, and then, later, the next one.


SOrry but my English is very poor :D can you tell me more info about that?
If I understand..df -h show 1.9 gb because the cloop module know exactly how big is the decompressed image..
That's right. You understand :-)


In reality the 1.9 Gb are fake...Maybe...
Ahem, the 1.9 GB are not "really" fake... ;-) It's just the uncompressed size of the filesystem.

Imagine the following:
- you create a 2.0GB partition on your hard disk, and create a filesystem on it, e.g. ext2.
- you mount it and put lots of data and programs into it until it is nearly full
- you umount it
- now you use a very special program to read the whole partition, compress it and write the result into a file
- after that's finished, you see that the file is about 700MB in size.
- now you mount this file instead of the original partition with the help of cloop.
- cloop decompresses the first few blocks and can now see that the original filesystem was 2.0GB and reports this size to the kernel.
- now let's say, you start a program that has 100kb of size and is located at 1,5GB counted from the beginning of the original filesystem
- so, the kernel asks cloop to get 100KB of data from the 1.5GB position
- cloop "translates" the "uncompressed position" 1.5GB into, for example, a "compressed position" of 550MB
- then cloop reads 60KB of compressed data, uncompresses it to 100kB and tells the kernel "this is the 100KB data from the 1.5GB position" (*)


(*):Cloop must read the data in pieces bigger than 60KB, I think 128kb, and if you are unlucky it needs 2 pieces because the 60KB it wants cross the border between two pieces - so it might need (in the worst case): 128KB+128kb = 256kb of RAM for the compressed data, with the 60KB it wants somewhere in the middle of this 256KB block, plus 100KB for the uncompressed data - altogether it needs = 356KiloByte(!) of RAM. Still much less than 700MB or 1.9GB.

Are you still there? ;-)

Cheers
Dirk