PDA

View Full Version : Place Knoppix into RAM when RAM >= 750MB? Now possible!



cyrano
03-27-2003, 08:46 PM
I don't know if this is possible or not, but I think it would be cool to have knoppix take a look at how much RAM the system has, and give the option to load itself into RAM if the user has more than, say 750 or 800MB. My thinking is that it could exist compressed in RAM, in the same way that it is compressed on the CDROM right now. The ram that is not used up to host the file system could be used to run applications, as normal.

If this is possible, I would imagine that this approach would give knoppix a major edge over everything else available. The hard drive is typically the slowest accessing part of any computer system, right? Well, actually the CDROM is slower, Knoppix runs off of CD, and therefore it takes a performance hit vs other distros. If Knoppix ran from RAM, though, it would be faster than HD based distros! It could just use the HD to store user data files [documents, bookmarks, etc.] persistantly between sessions. Additionally, this would free up the CDROM for normal operations again, since once the OS is loaded into RAM, the CD could be removed. Just think, it would be possible to load Knoppix, download an updated ISO to your /home directory (which is stores it on your hard drive), burn a new CD while Knoppix is running, and boot the new CD next time. Instant (clean) OS upgrade without having to worry about backing up your data files first.

I realize that most end users do not have that much RAM, but those who do typically have it just sitting around not doing anything. Plus I've heard it said that when designing a program, you should design for the system of tomorrow in mind, not the system of today. In just another year to two, most systems will contain gobs of RAM, and Knoppix could be ruling the world if it is prepared!

So what do you think? Is this a simple as copying everything into a giant ramdisk? Do I just not have any idea of what I'm talking about and the technical issues involved? I'd love to hear feedback on this idea.

Fabianx
03-28-2003, 08:52 PM
Hi,

I'm somewhat familiar with knoppix and its associated technologies and I believe what you want to do is possible. :D

Btw. KNOPPIX / LiveCDs (especially with cloop) can be and are faster on my computer from cdrom then from harddisk.

I've looked a fairly long time at the code and think I have found a "solution" (it is no real problem).

You only need to alter the boot.img/miniroot.gz/linuxrc

Just mount the image, gunzip miniroot.gz, mount miniroot, change linuxrc according to idea below, unmount miniroot, gzip miniroot again, copy it back to boot.img, unmount boot.img and then most important step: syslinux boot.img to make it boot again.

Ok, so much for the instructions here is the idea: (Just search for cloop in linuxrc, and you'll find soon, what you seek)

if test -n "$FOUND_KNOPPIX" -a -f /cdrom/KNOPPIX/KNOPPIX; then
# DEBUG
# echo "6" > /proc/sys/kernel/printk
insmod -f /modules/cloop.o file=/cdrom/KNOPPIX/KNOPPIX
mountit /dev/cloop /KNOPPIX "-o ro" || FOUND_KNOPPIX=""
fi

ok, before the insmod we need to insert our code to copy it to Ramdisk (perhaps check if some parameter is missing, but thats just future, when it could be available as cheat code)

but, we don't have all necessary commands to do what we want to do cp, mv, mkdir ... all missing) .. so we let it mountit first from cdrom ...

Not tested (don't have so much ram, just the idea)



insmod -f /modules/cloop.o file=/cdrom/KNOPPIX/KNOPPIX
mountit /dev/cloop /KNOPPIX "-o ro" || FOUND_KNOPPIX=""
fi

COPYTORAM="yes" # yes we want to copy it

# CopyToRam ?
if test -n "$COPYTORAM";
then
# Pre-test if everything succeeded.
if test -n "$FOUND_KNOPPIX"
then
# copy library cache
cat /KNOPPIX/etc/ld.so.cache > /etc/ld.so.cache
echo ""

# Enable kernel messages
echo "6" > /proc/sys/kernel/printk

# Set paths
echo -n "${CRE}${BLUE}Setting paths...${NORMAL}"
PATH="/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:."
export PATH

# Debian weirdness
/bin/cp -a /KNOPPIX/etc/alternatives /etc/ 2>/dev/null

# From here, we should have all essential commands available.
hash -r

echo "Copying /cdrom to RamDisk, this will take a while ..." # Hey, we could use some fancy dialog skript here :-))
mkdir /cdrom2
mount -t tmpfs -o size=800M /dev/shm /cdrom2
cp -a /cdrom/* /cdrom2 # Copy everything to /cdrom2; yes we need more things then only the KNOPPIX image, we don't want to produce problems we hadn't before, do we ?
export PATH= # reset path
umount /KNOPPIX # unmount it
umount /dev/cloop # release CD
umount /cdrom # unmount CD
insmod -f /modules/cloop.o file=/cdrom2/KNOPPIX/KNOPPIX
mountit /dev/cloop /KNOPPIX "-o ro" || FOUND_KNOPPIX="" # if everything else did suceeded so far, this will hopefully succeed too
# lets again do the initial stuff :-))
fi
fi

# Final test if everything succeeded.
if test -n "$FOUND_KNOPPIX"
then
# copy library cache
cat /KNOPPIX/etc/ld.so.cache > /etc/ld.so.cache
echo ""

# Enable kernel messages
echo "6" > /proc/sys/kernel/printk

# Set paths
echo -n "${CRE}${BLUE}Setting paths...${NORMAL}"
PATH="/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:."
export PATH

# Debian weirdness
/bin/cp -a /KNOPPIX/etc/alternatives /etc/ 2>/dev/null

# From here, we should have all essential commands available.
hash -r

# Did we copy from ram ?
if test -n "$COPYTORAM";
then
rmdir /cdrom
ln -s /cdrom /cdrom2 # make a symlink and go on to normal boot
fi


Thats it! Only thing, that could make problems now is that there is another ramdisk for /home and /var mounted and there is not enough memory left to do so effiecently ...

Perhaps also in syslinux.cfg the size of the initial ramdisk has to be made bigger!

Then you can rewrite cdrom or boot from floppy!

And it "should" work!

Best Regards,

Fabian Franz

PS: Hope to hear from your efforts soon. If it wokrs successful maybe I can talk to Klaus to include it in official distribution (in 2005 ;-) )

aay
03-28-2003, 10:07 PM
Hehe. Yeah by 2005 we will all have plenty of RAM for this!

cyrano
04-04-2003, 04:00 PM
Wow, thanks for your answer, I would have never figured all that out on my own. As soon as I pay off my taxes, I hope to spend the $50 needed to buy another 512MB of RAM. Then I'll have enough of a system to try out this approach and report back on how well it works.

Thanks again for the info! :D

mithrill
04-05-2003, 03:34 PM
I think this sounds like a great idea, perhaps Klaus should try to add a function like this to Knoppix.
After all it isnt that uncommon to have 1gb of ram today.
Couldnt this be added without to much trouble as a boot option / cheatcode?

true1ever
04-05-2003, 05:26 PM
This would be a great idea for systems that have a lot of RAM but only one CDROM drive.

mithrill
04-05-2003, 05:42 PM
Yeah, and if you're running it from RAM you don't have to hear the extremly loud and annoying noise from the spinning cd-rom.

true1ever
04-05-2003, 05:55 PM
true that :)

cyrano
04-10-2003, 06:05 PM
I just tried installing from CD to HD on my test system, and noticed that I was tapping my foot for a good 15 minutes while the contents of the CD were transferred over. I was thinking that it would take this long to boot up every time if we actually set the sytem up to pull from CD into RAM each time. It got me thinking of how to get around that, and I had another cool idea. Picture this:

Your hard drive contains the following parititions:
- a 300 MB swap partition
- a ?? GB /home partition
- a 700 MB /fastload partition

The /home partition would contain all your user files that you want to be persistant between sessions. (Mozilla bookmarks, OpenOffice documents, movies, songs, etc.) The /fastload partition would contain a copy of the ISO image from the CD. Then we could boot off the CD, load the kernel and some copy commands into RAM, and copy the contents of the CD from the HD image to create our filesystem in RAM, like we did before. Perhaps it would make sense to have a small /boot partition which would contain a kernel and some copy commands to put everything into memory. Then we wouldn't need to boot from CD each time.

OK, so now by pulling the filesystem from the HD instead of from the CD, we should have cut our boot time dramatically. Then I started thinking about keeping the system up to date. Perhaps if we downloaded a new package into our /home directory, we might be able to run a new command. "apt-get upgrade -kpx-double" or something similar. It would work the way you expect it to: it would update the filesystem in RAM with the contents of the new package -- but then it would mount the ISO image in /fastload, chroot over there, and update the package there too. That way the next time you reboot your system, you won't lose the changes that you made to the packages. This would have two, rather nice effects:

1) Knoppix could set up a system for distributing patches instead of users downloading a new ISO image every 3 days. That would cut down on their bandwidth, thus cutting their expenses.

2) At any time while running Knoppix, you could burn a new CD from the ISO that's in /fastboot, which would have all your updates applied to it. Not only would this be convenient for sharing with friends, but it would probably spawn a bunch of special purpose Knoppix clones. One might focus on Gnome, another might focus on Multimedia apps, etc.

I've been rambling much too long now. Obviously I'm talking about things that I only understand a little about, but I had such a nice response to my first post, that I thought I would throw these ideas out there and see what happened. That's the purpose of this forum right? :lol:

Ambrose
04-15-2003, 12:50 AM
Wont you loose all of the software that's in the ram everytime you reboot???

ie, /ramdisk on KNOPPIX

cyrano
04-15-2003, 06:30 PM
Yes you would. Each time you boot your system, you'd have to load the contents of the CD into RAM from the HD. The way that knoppix works now, we load just the necessary parts of the CD into RAM using the CDROM. Maybe we only load 100MB worth of stuff, but using the CDROM to do it is slow. If we load the full 700MB of the CD into RAM using the HD, we will be loading more data, but HD access times are much faster than CD access times. Once your OS is fully loaded into RAM, launching any program would be lightning fast! Obviously, this approach is not meant for people that reboot their machines 10 times a day -- but if you boot once in the morning, or if you boot your machine and keep it up for several days, I think the benefit of the lightning fast program response times would outweigh the longer boot up.

LinuxSam
04-17-2003, 12:20 AM
If you are to use the harddrive to store the files from the CD... why not make a real linux installation? No need to boot from the CD at all.

I don't really see your point in this...


/LinuxSam

Henk Poley
04-17-2003, 08:53 PM
If you are to use the harddrive to store the files from the CD... why not make a real linux installation? No need to boot from the CD at all.

I don't really see your point in this...
I think there isn't one, unless you qualify " fun to thinker with".

btw, couldn't you boot Knoppix, load an fs into RAM, with everything symlinked to the CD. And then 'slowly' replace all symlinks with the 'real stuff'?

I do know this is impossible under Windows, apart from symlinking (which isn't that impossible btw). That's the reason why you need to reboot when you install software. Which reminds me of my Windows98 from RAM project ;-)

Lex
07-25-2003, 10:34 AM
Now back to the topic: I have changed the linuxrc as Fabianx suggested: http://www.mathematik.unni-bielefeld.de/~num0312/linuxrc
The file has been put back on the CD as he described.

BUT Knoppix boots as always, without loading the CD to the RAM (I have 1GB, Knoppix's limit is at 900MB).

Please take a look at the file and tell me what is wrong about it.

Lex
07-25-2003, 04:58 PM
the correct URL is of course:

http://www.mathematik.uni-bielefeld.de/~num0312/linuxrc

The problem I have mentioned above is solved (boot.img was not set as the booting image).

So here is the next one:

error messages while booting:


Getting paths...Copying /cdrom to RamDisk, this will take a while ...
mount: Unknown option 'size=800M' ignored.
cp: writing `/cdrom2/Demos/Audio/opensource.ogg' : No space left on device
... [no space left on device error messages]
cp: writing `/cdrom2/index.html : No space left on device
umount: Invalid argument
umount: Device or resource busy
insmod: a module named cloop already exists
/linuxrc: 442: Syntax error : end of file unexpected (expecting "fi")
FAT: bogus logical sector size 0
FAT: bogus logical sector size 0
Kernel panic: VFS: Unable to mount root fs on 01:01


I ran it this time in VMware virtual machine with 880 MB RAM with "knoppix 2" cheat code.

I will burn the image and try to boot the real PC and report here.

Lex
07-25-2003, 05:19 PM
the same errorrs on the real PC

It seems like mount does not want to create a RamDisk :?

Fabianx
07-26-2003, 11:46 AM
the correct URL is of course:

http://www.mathematik.uni-bielefeld.de/~num0312/linuxrc

The problem I have mentioned above is solved (boot.img was not set as the booting image).

So here is the next one:

error messages while booting:


Getting paths...Copying /cdrom to RamDisk, this will take a while ...
mount: Unknown option 'size=800M' ignored.
cp: writing `/cdrom2/Demos/Audio/opensource.ogg' : No space left on device
... [no space left on device error messages]
cp: writing `/cdrom2/index.html : No space left on device
umount: Invalid argument
umount: Device or resource busy
insmod: a module named cloop already exists
/linuxrc: 442: Syntax error : end of file unexpected (expecting "fi")
FAT: bogus logical sector size 0
FAT: bogus logical sector size 0
Kernel panic: VFS: Unable to mount root fs on 01:01


I ran it this time in VMware virtual machine with 880 MB RAM with "knoppix 2" cheat code.

I will burn the image and try to boot the real PC and report here.

Heh! Thats great that you actually try it!

Sorry, it was a shot in the air. I will try to fix the remaining issues and see form you nice error report, what did go wrong :-))

And I'll do it asap. (as soon as possible, which will mean monday or tuesday; at least its a nice idea and nice that someone works on this :-)) )

cu

Fabian

Fabianx
07-26-2003, 12:58 PM
the same errorrs on the real PC

It seems like mount does not want to create a RamDisk :?

Here is a new diff:



--- linuxrc.orig 2003-07-26 13:10:09.000000000 +0200
+++ linuxrc 2003-07-26 13:48:21.000000000 +0200
@@ -235,7 +235,31 @@
insmod -f /modules/cloop.o file=/cdrom/KNOPPIX/KNOPPIX
mountit /dev/cloop /KNOPPIX "-o ro" || FOUND_KNOPPIX=""
fi
-
+
+ COPYTORAM="yes" # yes we want to copy it
+
+ # CopyToRam ?
+if test -n "$COPYTORAM";
+then
+ # Pre-test if everything succeeded.
+ if test -n "$FOUND_KNOPPIX"
+ then
+ # copy library cache
+ cat /KNOPPIX/etc/ld.so.cache > /etc/ld.so.cache
+ echo ""
+
+ echo "Copying /cdrom to RamDisk, this will take a while ..." # Hey, we could use some fancy dialog skript here :-))
+ /bin/mkdir /cdrom2
+ /bin/mount -t tmpfs -o size=800M /dev/shm /cdrom2
+ /bin/cp -a /cdrom/* /cdrom2 # Copy everything to /cdrom2; yes we need more things then only the KNOPPIX image, we don't want to produce problems we hadn't before, do we ?
+ umount /KNOPPIX # unmount it
+ rmmod cloop # release CD
+ umount /cdrom # unmount CD
+ insmod -f /modules/cloop.o file=/cdrom2/KNOPPIX/KNOPPIX
+ mountit /dev/cloop /KNOPPIX "-o ro" || FOUND_KNOPPIX="" # if everything else did suceeded so far, this will hopefully succeed too
+ fi
+fi
+
# Final test if everything succeeded.
if test -n "$FOUND_KNOPPIX"
then
@@ -257,6 +281,13 @@
# From here, we should have all essential commands available.
hash -r

+# Did we copy from ram ?
+if test -n "$COPYTORAM";
+then
+ rmdir /cdrom
+ ln -s /cdrom /cdrom2 # make a symlink and go on to normal boot
+fi
+
# Clean up /
rm -rf /modules /static


I also uploaded necessary files and an boot.img for 06-06 and earlier (I know its deprecated, but I don't have newer version atm) to:

http://debian.tu-bs.de/knoppix/RamPatch/

You can get linuxrc also form there.

I tested everything except the copying, as I don't have that much RAM.

cu

Fabian

PS: I'm awaiting your results. VMWare test is fine :-)[/code]

Lex
07-26-2003, 02:21 PM
I have just tested the modified Knoppix on a 1GB-RAM-PC and it works perfectly (really fast and silent, too).

Great job, Fabian!

EDIT: you may have to use the "toram" cheat code if you have used the boot.img for modifying Knoppix

ahfoo
07-28-2003, 02:38 PM
I didn't see that anywhere? Might there be a shortcut to a pure RAMDisk based install with this magic cheat code? That would sure be nice. 750 megs of RAM is pretty easy to come up with these days.

Fabianx
07-29-2003, 08:27 AM
I didn't see that anywhere? Might there be a shortcut to a pure RAMDisk based install with this magic cheat code? That would sure be nice. 750 megs of RAM is pretty easy to come up with these days.

It can't be yet in the docs. Its not even in Knoppix, yet. As it was "just" :-) developed.

Yeah, it does copy the whole cdrom to RAM, so you can eject it afterwards. :-)

cu

Fabian

ahfoo
07-29-2003, 08:29 PM
I was looking for it in the cheat code sheet. Okay, I'm on the same page now. I'm so excited to try this, but sadly I have to wait to get some more RAM.
But being impatient, I'm wondering about using a smaller knoppix distro as I could definitely afford another 250megs by the end of the week. I looked at tiny distro, but I'm thinking more along the lines of one of the smaller Morphix packages.
By the way, you folks might be interested in an article over at EETimes today. It announces the Dragon processor which is a native Chinese CPU. The interesting thing is that the company that is marketing it intends it to run a distro from Transmeta that seems to also run in RAMDisk and is loaded from Flash memory. At least that's how it looked to me.

Stephen
07-29-2003, 08:57 PM
I was looking for it in the cheat code sheet. Okay, I'm on the same page now. I'm so excited to try this, but sadly I have to wait to get some more RAM.
But being impatient, I'm wondering about using a smaller knoppix distro as I could definitely afford another 250megs by the end of the week. I looked at tiny distro, but I'm thinking more along the lines of one of the smaller Morphix packages.
By the way, you folks might be interested in an article over at EETimes today. It announces the Dragon processor which is a native Chinese CPU. The interesting thing is that the company that is marketing it intends it to run a distro from Transmeta that seems to also run in RAMDisk and is loaded from Flash memory. At least that's how it looked to me.

You may want to look at knoppix kde (ftp://ftp.kde.org/pub/kde/stable/3.1/contrib/cdimages/) it's only 457mb for the iso so that saves at least 250mb right there and now that I'm thinking I got enough ram I may just give it a try.

aay
07-29-2003, 09:12 PM
Wow looks pretty neat. I hope this makes its way into a future Knoppix release.