fusecompress on knoppix-data.img?

[Capricorny]

Some comments:

1. I think the "nowear" cheatcode should not always be used. When doing heavy work involving the persistent store, like package upgrading, I think it may be better to skip the ramdisk layer. The ramdisk layer is for "everyday", not "administrative" use.

2. If we often find ourselves doing lots of heavy work with the persistent store - why are we doing it there? Much better to mount extra partitions and use them directly. Myself, I now find that I need about 4GB persistent store for comfort, but not more. Rather remaster, if programs fill it up, or move data out.

3. With modest modifications, how much does it matter if the ramdisk crashes without having been written to persistent store now and then? And with laptops, power outage is normally not a reason for crash.

4. I wonder what is so terribly impractical with an aubrsync-based scheme, where the ramdisk is normally copied at shutdown, but there is also an option to close X, unmount&update persistent store, and remount/restart X. That is safe, and I wonder where the pressing performance/use issues lie that motivate giving up on safety?

[kl522]

Some comments:

1. I think the "nowear" cheatcode should not always be used. When doing heavy work involving the persistent store, like package upgrading, I think it may be better to skip the ramdisk layer. The ramdisk layer is for "everyday", not "administrative" use.
.....

But there is actually an easier way to accomplish the "nowear" thing, is to ask operating system to be lazy on writing to flash. Here is one write up of such thing :-

http://www.cyrius.com/debian/nslu2/linux-on-flash.html

I thought this is a more elegant solution than to mimic the operating system behaviour in userspace.

[Capricorny]

But there is actually an easier way to accomplish the "nowear" thing, is to ask operating system to be lazy on writing to flash. Here is one write up of such thing :-

http://www.cyrius.com/debian/nslu2/linux-on-flash.html

I thought this is a more elegant solution than to mimic the operating system behaviour in userspace.

As far as this mostly is about OS behavior, I fully agree. But I don't think that is the whole story. We have a lot of proper user space tasks resulting in many disk writes, furthermore, use of ramdisk can speed up running from flash sticks a lot. I also don't quite see that a nowear options has to be that much of an ugly hack, and in the actual use situations, we will mostly have more free memory to use in the time to come. (Because of games requirements and cheap RAM.)

At the very least, I think it is worthwile having a few users testing a clean implementation of it. And using a similar solution for things like databases could also be effective: We could, for example, aufs-mount a compressed ro part, a rw part from a disk partition and a ramdisk. Today, aufs won't let us use the compressed part, when that comes from something already aufs-mounted.

[kl522]

We have a lot of proper user space tasks resulting in many disk writes, furthermore, use of ramdisk can speed up running from flash sticks a lot.

The OS caches all disk writes using memory, so in a way, it is like a layer of ramdisk. It becomes slow only when the Ram is flushed to disk or flash. And here we have a choice to ask the OS not to flush it using the kernel parameters. This is the OS behaviour which I am talking about.

At the very least, I think it is worthwile having a few users testing a clean implementation of it. And using a similar solution for things like databases could also be effective: We could, for example, aufs-mount a compressed ro part, a rw part from a disk partition and a ramdisk. Today, aufs won't let us use the compressed part, when that comes from something already aufs-mounted.

Again, you are duplicating what database servers are doing. The database server caches as much thing as memory permits in the memory.

In any case, I shall not stop anyone from doing anything they so prefer. Feel free to implement these. It's useful learning exercise anyway.

[Capricorny]

The OS caches all disk writes using memory, so in a way, it is like a layer of ramdisk. It becomes slow only when the Ram is flushed to disk or flash. And here we have a choice to ask the OS not to flush it using the kernel parameters. This is the OS behaviour which I am talking about.

Again, you are duplicating what database servers are doing. The database server caches as much thing as memory permits in the memory.

In any case, I shall not stop anyone from doing anything they so prefer. Feel free to implement these. It's useful learning exercise anyway.

Of course, I know this.
There are two issues that don't get addressed by your approach:
1. Asking the OS to be as lazy as possible is not necessarily beneficial in all respects.
2. Even with max caching and I/O laziness everywhere, there are in many cases more than enough read/write operations left both to slow down operation and wear out flash.

So, while adding yet another layer of ramdisk may not be optimal in several respects, the issue for many, including me, is whether it severely degrades performance, and whether it goes with a defensive approach to system administration. The issue of performance degradation I think is probably a rather moot one, as all the caching going on will of course reduce the ramdisk involvment too. As for defensiveness, only experience can tell, but there are good reasons to expect at least some benefits.