Printable View
Heheheh, thanks that actually works :D
aubrsync works succesfully but I haven no idea how to clear the ramdisk and safe persistend on a running system, gonna look at puppy linux and see how they do it :p.
It took so long to reply as some creepy update/removal made my keyboard useless after knoppix-autoconfig got run, I hope I'm past that missery now
Great, loot at it and let us know.Quote:
Originally Posted by dinosoep
One way to always have a working version while experimenting is creating several small harddisk partitions and place one testing Knoppix version on each. Normally, I work with at least two or three images, and I also store select versions on external media.Quote:
Originally Posted by dinosoep
Actually I found out how Puppy does it :-Quote:
Originally Posted by dinosoep
1. 'aubrsync' does it the safe way, it locks the file system by putting it in 'readonly' mode before any attempt of copy or move to the lower layer aufs. So 'aubrsync' fails active root file system.
2. Puppy does not do it the safe way, so it does not lock the file system. Puppy uses a self-developed script call 'snapmergepuppy'. If you are backing up a huge file on the top layer of aufs which is the ram file system, while it is being copied to the lower layer of the aufs which is the persistent store, it is possible that the file is deleted during the copy, so you will end up having the deleted file reappearing in the aufs. Puppy developers are fully aware of this. Between the two devils, Puppy developers have chosen this.
3. Puppy does not 'flush' the ram file system after making a copy to the persistent store. Using 'aubrsync' terminology, it is sort of a 'aubrsync copy' without 'lock'. That being the case, does puppy do incremental between the copy or it does full copy each time ? This is something I don't know. If it does incremented, the script must be very sophisticated then. Otherwise if it does full copy each time, if you have written a lot of things into the RAM file system, you will end up repeatedly make a huge amount of data transfer between the layers of aufs.
In any case anyone who is interested to use this Puppy behaviour ( called PUPMODE 13 in Puppy terminology ) can study the script 'snapmergepuppy'. But the sophistication of this has far gone beyond the reach of a typical non-programmer.
Some comments:
1. I think the "nowear" cheatcode should not always be used. When doing heavy work involving the persistent store, like package upgrading, I think it may be better to skip the ramdisk layer. The ramdisk layer is for "everyday", not "administrative" use.
2. If we often find ourselves doing lots of heavy work with the persistent store - why are we doing it there? Much better to mount extra partitions and use them directly. Myself, I now find that I need about 4GB persistent store for comfort, but not more. Rather remaster, if programs fill it up, or move data out.
3. With modest modifications, how much does it matter if the ramdisk crashes without having been written to persistent store now and then? And with laptops, power outage is normally not a reason for crash.
4. I wonder what is so terribly impractical with an aubrsync-based scheme, where the ramdisk is normally copied at shutdown, but there is also an option to close X, unmount&update persistent store, and remount/restart X. That is safe, and I wonder where the pressing performance/use issues lie that motivate giving up on safety?
But there is actually an easier way to accomplish the "nowear" thing, is to ask operating system to be lazy on writing to flash. Here is one write up of such thing :-Quote:
Originally Posted by Capricorny
http://www.cyrius.com/debian/nslu2/linux-on-flash.html
I thought this is a more elegant solution than to mimic the operating system behaviour in userspace.
As far as this mostly is about OS behavior, I fully agree. But I don't think that is the whole story. We have a lot of proper user space tasks resulting in many disk writes, furthermore, use of ramdisk can speed up running from flash sticks a lot. I also don't quite see that a nowear options has to be that much of an ugly hack, and in the actual use situations, we will mostly have more free memory to use in the time to come. (Because of games requirements and cheap RAM.)Quote:
Originally Posted by kl522
At the very least, I think it is worthwile having a few users testing a clean implementation of it. And using a similar solution for things like databases could also be effective: We could, for example, aufs-mount a compressed ro part, a rw part from a disk partition and a ramdisk. Today, aufs won't let us use the compressed part, when that comes from something already aufs-mounted.
The OS caches all disk writes using memory, so in a way, it is like a layer of ramdisk. It becomes slow only when the Ram is flushed to disk or flash. And here we have a choice to ask the OS not to flush it using the kernel parameters. This is the OS behaviour which I am talking about.Quote:
Originally Posted by Capricorny
Again, you are duplicating what database servers are doing. The database server caches as much thing as memory permits in the memory.Quote:
At the very least, I think it is worthwile having a few users testing a clean implementation of it. And using a similar solution for things like databases could also be effective: We could, for example, aufs-mount a compressed ro part, a rw part from a disk partition and a ramdisk. Today, aufs won't let us use the compressed part, when that comes from something already aufs-mounted.
In any case, I shall not stop anyone from doing anything they so prefer. Feel free to implement these. It's useful learning exercise anyway.
Of course, I know this.Quote:
Originally Posted by kl522
There are two issues that don't get addressed by your approach:
1. Asking the OS to be as lazy as possible is not necessarily beneficial in all respects.
2. Even with max caching and I/O laziness everywhere, there are in many cases more than enough read/write operations left both to slow down operation and wear out flash.
So, while adding yet another layer of ramdisk may not be optimal in several respects, the issue for many, including me, is whether it severely degrades performance, and whether it goes with a defensive approach to system administration. The issue of performance degradation I think is probably a rather moot one, as all the caching going on will of course reduce the ramdisk involvment too. As for defensiveness, only experience can tell, but there are good reasons to expect at least some benefits.
kl522, I have been studying that specific script a little bit back and as you predicted, I don't understand it :p. I'm used to object oriënted java but I should consider learning bash xd.
What I wanted to do was check for every file in the ramdisk if a process was using it and if not copy it to the knoppix-data, remove from ramdisk and sync.
This should happen when the ramdisk is over 100 mb ever 2 minutes a check.
That link you posted is indeed a way more elegant solution and I'm going to aply it too :D