Ability to boot either Knoppix/Ubuntu/DSL from a USB thumb

[smhall05]

All,

I am looking for advice on how to configure a boot loader that will allow me to choose which LiveCD distro I want loaded from a USB thumb drive. What should I search for in order to start doing this? I am not sure what file is read on boot from knoppix, I'm sure that would give me insight into this.

I like Knoppix alot, just want the ability to use ubuntu for its wireless detection or DSL, well because it is so damn small! Any help here would be great. Thanks.

[Harry Kuhman]

I am looking for advice on how to configure a boot loader that will allow me to choose which LiveCD distro I want loaded.
Here's an idea: The selection of booting of a Live CD distro can be controlled by labeling the various LiveCDs and putting the proper one in the CD drive when booting the system.

If you're talking about installing Knoppix to hard disk and then booting it, I will not be a part of that blasphemy. And any discussion of it really belongs in the hard disk install forum. However, for legitimate Linux hard disk installs, Grub should be able to do this quite easily (and if you can't get it to do this then you really really shouldn't even be thinking about "installing" Knoppix onto a hard disk). Another neat tool for multi-boot operation is free software called XOSL (http://en.wikipedia.org/wiki/XOSL), available here (http://www.ranish.com/part/). I don't think it has been updated in years, but it does some nice tricks for you when booting multiple OSs that the Linux boot loaders will not do.

[smhall05]

Here's an idea: The selection of booting of a Live CD distro can be controlled by labeling the various LiveCDs and putting the proper one in the CD drive when booting the system.

Harry,

Touche! I sent you a PM re-explaining what I want to do. My CDs are already nicely labeled with Sharpie Permanent Marker "Think out loud" style.

[smhall05]

I may have asked my question poorly. What I want to do is using an USB thumb drive store all my distros. I want when I insert my USB drive, a menu appears that will allow me to pick which distro I will execute. I don't claim to be a programming/linux expert, but I find that tackling practical projects like this helps me get a better understanding of linux overall. If you have any ideas please let me know. If this already exists please point me to the code.

[Harry Kuhman]

I may have asked my question poorly. What I want to do is using an USB thumb drive store all my distros. I want when I insert my USB drive, a menu appears....
Unfortunately, while I've used Grub and XOSL a lot to multi-boot from hard drives, there are still some mysteries about USB booting for me. Some others in these forums do have a better grasp on it than I do and I'm hoping that they will join in on this discussion.

To that end, yea, your original post wasn't clear (at least to me) that you were talking about a USB flash device boot. I would suggest that for the best responses you go back and use the edit option on your original post and change the subject to one that reflects the use of USB flash devices and multi-booting.

Of course, it's worth mentioning that USB flash booting is not available on all systems. I've tried to get a DSL flash drive to boot on several systems with "boot from USB" settings in the BIOS with no luck (both an HP Pavilion zx4240 and an IBM Thinkpad), but the same USB flash drive booted just fine on a newer HP notebook (a Pavilion DV6000). So don't be too surprised if any solution posted here works on some systems but not others.

From what I remember of prior discussion on this issue, there are multiple ways that USB flash devices can be formatted, and some are more hard disk like than others. I can't, at the moment, locate those prior posts though (a good reason why, if you want to post helpful information of that nature, it would be handy to put it in the wiki and just post a link to it in a forum response.) It would be ideal if we could run something like XOSL from a usb flash device and multi-boot that way.

Meanwhile, you might want to look at Puppy Linux. In my experience it has good wifi support (much better than Knoppix). It's almost as small as DSL, and it has a nice package system that should let you include whatever applications you want with it right on the flash device. Overall it might be much more efficient to add what you want to Puppy and carry around one OS with all the applications that you want than have a flash drive with 3 OSs that have a lot of redundant programs between them. But I do see benefits for multibooting a flash drive and hope that people knowledgeable in this area join in the conversation.

[Harry Kuhman]

.... (a good reason why, if you want to post helpful information of that nature, it would be handy to put it in the wiki and just post a link to it in a forum response.)....
I did manage, after several searches, to find that someone has put some of the information that I remember seeing in the forums into a wiki article, which is here (http://www.knoppix.net/wiki/USB_Based_FAQ). It has some information that may be helpful, including talking about running grub from a usb flash device. However, I still cannot locate some of the discussion about the different low level formatting options available on a flash device that affect how it behaves like a hard disk. I'm still hoping that the knowledgeable will join in on this conversation.

[johnrw]

smhall05,

Here is the discussion of what you are trying to do.
http://www.knoppix.net/forum/viewtopic.php?t=11796

Gilles Van Ruymbeke is probably the most experienced person here at doing what you are trying to. The notion of booting from a usb thumbdrive... works better on newer pc's then older pc's. Thumbdrives have limititations, they can't be written to more than a million or so times... so even using a filesystem that records a file's last access time can cause a shorter lifespan of the media, for example.

Unlike Harry, I prefer to run linux from the iso file. So does Gilles. He routinely makes custom minirts after each Knoppix release that empower Knoppix to boot from an iso, which is stored on ntfs filesystems. I prefer to run from iso, because you can't break it.
Once you get the iso booting... you can create a persistent home. The persistent home is a second file that holds any changes you make, that is overlaid onto the readonly filesystem. Brilliant stuff. Not all live cd's can do these other things like boot from ntfs etc.

In general, if you change the drive geomtry to a 64 head, and 32 sectors per track... with the cylinders being adjusted to give the right overall size... that will be the most compatible to older buggy (as far as usb booting is concerned) bioses. If you don't care about older bios booting... then the existing geometry will do fine on a newer bios.

I recently explored the possibilities and finally decided to go for broke and have a 8GB usb thumbdrive formatted with an ntfs boot partition... and an ext2 partition for my persistent home. Sda1 is to hold the whole Knoppix 5.3.1 DVD and is booting grub4dos's master stroke called GRLDR... though I decided to name it NTLDR(lol) and use a ntfs master boot sector left by Knoppix's mkfs.ntfs and sfdisk... and the first 446 bytes of MS code found on every Windows initialized disk.

Maybe I will document my experience for others somewhere. The topic I referenced above is long. Too long.
Maybe I'll try and write a script to do it.

To change geometry you must first zero out the first 512 bytes of the drive. I then reboot because the kernel still has an idea of what the drive is. Then you can use sfdisk to force a change... with sfdisk -H64 -S32 -uM -D /dev/sda

Then it prompts you for the first partition which you can respond with
,,7,* to make the it use
Starting at the first sector,
and all space available,
ntfs partition type,
and make it bootable.

Then pressing return for the next partition gets you to where it will ask if it okay to write the new partition... and viola... you have changed the geometry reported by the drive.

If you won't be booting from a dvd iso image... but just cd images... then you can use the HP utility "out there" to make it bootable for newer computers using fat32. It won't hold dvd iso images though. Otoh, there are ways to get an ntfs bootable partition made from that utility.

Using a Knoppix persistent home, it is very possible to get your wireless working. The problem with wireless is you would need to know the access point's name at bootup when Knoppix sets up networking. Another set of cheatcodes would cover that need I guess... but when I used ndiswrapper for a broadcom wireless... I used the knoppix.sh script to make the changes needed to get my wireless working. You can unload modules... and load other ones... copy needed firmware all inside knoppix.sh... which is activated with the myconfig= cheatcode.

But if you want multiple os'es from a grub4dos menu... that's easy. Make a small partition as a boot partition which can be vfat(fat32) and load grub4dos in that. When you boot grub4dos on usb thumbdrive... it will show up thinking it is rooted on hd0,0. (It doesn't understand the sda stuff, but the bios tricks it into thinking usb is hd) but when you load the kernel you use sda2 sda3 for the bootfrom= stuff depending on your chosen layout.

So grub4dos's menu.lst... winds up looking like:

root (hd0,0)
kernel linux knoppix bootfrom=/dev/sda2/knoppix.iso other kernel stuff
initrd minirt_531.gz

So the first thing you need to decide is if you want older bios compatibility.

[Harry Kuhman]

In general, if you change the drive geomtry to a 64 head, and 32 sectors per track... with the cylinders being adjusted to give the right overall size... that will be the most compatible to older buggy (as far as usb booting is concerned) bioses. If you don't care about older bios booting... then the existing geometry will do fine on a newer bios.
Thanks for clearing up the reason for the geometry change. I knew it was an issue, but didn't fully know why.


Unlike Harry, I prefer to run linux from the iso file. So does Gilles......
I do prefer to just boot from a CD. Or from an ISO, using that ISO mounted as a virtual disk in VMware. Gilles has done some great work also making tiny VMXs that will mount and run the ISO and include the VMware tools, but apparently there is a conflict with the tools and the kernel that the newest Knoppix uses. Running in a virtual machine has some limitations (the graphics performance may not let you run some of the 3D wiz-bang new UIs for example), but overall it is a great way to run the Knoppix ISO.

I suspect that one of the reasons that I don't use the Gilles miniroot technique is that I really don't know what he's doing on a technical level and so I tend to gloss over it. I've seen and read the very long thread on the subject, but it sometimes assumes Linux knowledge that I lack. Can you explain in technical but non-Linux terms just what is going on with his miniroot, what it really does, why different versions are needed for each release and so on? I think that part of the reason for the changes each release is that it includes a Linux kernel and that must match the kernel that Knoppix uses, but there seems to be more to it than just that and I'm not clear on what else he's doing.

[johnrw]

In general, if you change the drive geomtry to a 64 head, and 32 sectors per track... with the cylinders being adjusted to give the right overall size... that will be the most compatible to older buggy (as far as usb booting is concerned) bioses. If you don't care about older bios booting... then the existing geometry will do fine on a newer bios.
Thanks for clearing up the reason for the geometry change. I knew it was an issue, but didn't fully know why.


Unlike Harry, I prefer to run linux from the iso file. So does Gilles......
I do prefer to just boot from a CD. Or from an ISO, using that ISO mounted as a virtual disk in VMware. Gilles has done some great work also making tiny VMXs that will mount and run the ISO and include the VMware tools, but apparently there is a conflict with the tools and the kernel that the newest Knoppix uses. Running in a virtual machine has some limitations (the graphics performance may not let you run some of the 3D wiz-bang new UIs for example), but overall it is a great way to run the Knoppix ISO.

I suspect that one of the reasons that I don't use the Gilles miniroot technique is that I really don't know what he's doing on a technical level and so I tend to gloss over it. I've seen and read the very long thread on the subject, but it sometimes assumes Linux knowledge that I lack. Can you explain in technical but non-Linux terms just what is going on with his miniroot, what it really does, why different versions are needed for each release and so on? I think that part of the reason for the changes each release is that it includes a Linux kernel and that must match the kernel that Knoppix uses, but there seems to be more to it than just that and I'm not clear on what else he's doing.

The reason for the geometry change lies in the fact that those bioses could boot usb zip drives, and so we are making the bios think it is a usb zip drive, which has a fixed geometry layout. I have seen some articles expounding the need for the partition to be located in the fourth primary partition slot of the mbr even.

As for a non linux technical description of what Gilles does in his miniroots... I am not entirely sure. I know he puts the needed kernel drivers in for filesystems like ntfs and reiserfs and then he would need to edit the linuxrc file to add tests for which filesystem is there. The ntfs_rw cheatcode he added to overcome an annoying problem when the bootfrom= cheatcode is used to boot from a iso file. If you put the knoppix.iso file on drive C:,.. and then use bootfrom=/dev/hda1/knoppix.iso 2 things happen.

First the whole drive C: is mounted read only. Second the iso file is mounted.

The first mounting stays... and prevents any further writing to the drive... even though by now ntfs writing is working well. (I copied a 15GB knoppix.img file to my ntfs drive... and then md5sumed it... compared to original md5sum etc.. then deleted it and windows had no complaints about the drive later on in a drive test.) I have proved to myself that linux considers a file mounted such as a knoppix.img as a separate filesystem because when I mount a knoppix.img(ext2) that is stored on a journaled filesystem like ext3... the ext2 file is not protected by journaling... ie if you lose the kernel for some reason... you will have errors on reboot in the knoppix.img. Since they are separate, there is no reason not to have them governed with separate permissions, and allowing a readwrite drive holding a readonly file and it's separate readonly filesystem. Gilles understood this problem, and after asking Klaus to do something about it... took it upon himself to implement a brute force hack. I feel like I am probably one of a handful of people that even noticed the cheatcode... in the last version. I recommend it whenever I get the chance as it changes the possibilities. If drive C: is readonly, you cannot even change the boot menu fille, C:\menu.lst, from within Knoppix... which is pretty frustrating when you are using grub4dos. But Klaus thinks Gilles miniroots are just too big for inclusion into the distro I suspect.

So, in short, Gilles' miniroots address specific usability problems. He (and I) wants it to be easy for windows users to dump windows without doing any more than downloading a iso file... downloading grub4dos's GRLDR, and putting an appropriate c:\menu.lst together... and adding C:\GRLDR="Knoppix 5.3.1" to the [operating systems] group of C:\boot.ini. This leaves windows intact... is unbreakable...(remember we want people to learn, and having a real installation, with all the things superuser can do to hurt it is the best way to make sure they never do.) and as confidence is gained with more frequent use... they could use the new 0wn installer... which btw works great. Then when that's not able to do something they want to do... they can opt for a real installation and all that that entails.

That is my preferred entry path to using linux... for windows users wanting to do so. My own experiences before I found (unbreakable) Knoppix, consisted of getting an install, doing rpmfind searches for a missing dependancy to some program I wanted to have... and installing something wrong and in so doing... breaking it. It is a viscous loop. If you can't keep it running... you can't learn it. If you can't learn it, you can't keep it running. There is absolutely no value in having a broken linux system, and not knowing how you broke it, requiring a linux user group master to come and fix it... except maybe increasing the need for linux user groups.

I recently posted a build kde4 howto... that builds kde4 inside a persistent knoppix.img file... which should be at least 10GB+ and since I did so with a 15GB file... should probably be that large. This can be done on a computer that has just one big ole C: drive with windows, if they have the empty space for the dvd iso file and a knoppix.img file. People have their own reason for keeping windows, and whether they love it or not don't need to be forced to resize a partition using this method of running Knoppix. Resizing a partition can be scary enough. It just takes running one program that knows about absolute sectoring, to punch holes in the value of that scheme. And since many people get windows without a cd, they must make a big decision to mess with their only copy. Most don't know that they can get the cd from the maker, as long as they can prove they are entitled to it, for a few bucks. Hope this helps.

What the heck... it's just bytes. Here is my C:\menu.lst which doesn't mention Win 98SE on C:\Windows or XP on E:\Windows, but first is my C:\boot.ini which does. Note... how easy it is to change the default to Knoppix 5.1.1! No point in fighting MS over the mbr.)


[boot loader]
timeout=30
default=C:\GRLDR

[operating systems]
C:\GRLDR="Knoppix 5.1.1"
multi(0)disk(0)rdisk(0)partition(6)\WINDOWS="Microsoft Windows XP Professional" /fastdetect /noexecute=alwaysoff
C:\="Microsoft Windows"

[comment]
default=multi(0)disk(0)rdisk(0)partition(6)\WINDOW S
multi(0)disk(0)rdisk(0)partition(6)\WINDOWS.old="Microsoft Windows XP Professional" /fastdetect /noexecute=alwaysoff


C:\menu.lst... (Hmm, I think Sabayon Linux is no longer there. Maybe a spring cleanup is due)



timeout 30
default /default

foreground = ffffff
background = 29447B
splashimage=(hd0,0)/splash.xpm

title Knoppix stock (hdb7/KNOPPIX_V5.1.1DVD-2007-01-04-EN.iso) home=/media/hdb9
root (hd1,6)
kernel /boot/grub/linux init=/etc/init ramdisk_size=100000 knoppix lang=us apm=power-off nomce quiet dma bootfrom=/dev/hdb7/KNOPPIX_V5.1.1DVD-2007-01-04-EN.iso home=/media/hdb9 myconfig=/media/hdb9 vga=791 BOOT_IMAGE=knoppix 2
initrd /boot/grub/minirt_511a.gz

title Knoppix 5.3.1 DVD Gilles Ruymbeke's (hdb9/KNOPPIX_V5.3.1DVD-2008-03-26-EN.iso) home=/media/hdb10
root (hd1,8)
kernel /boot/grub/linux init=/etc/init ramdisk_size=100000 knoppix lang=us apm=power-off nomce ide1=reset quiet dma libata.atapi=1 SELINUX_INIT=NO nmi_watchdog=0 bootfrom=/dev/hdb9/KNOPPIX_V5.3.1DVD-2008-03-26-EN.iso home=/media/hdb10/knoppix.img myconfig=/media/hdb10 ro vga=791 BOOT_IMAGE=knoppix 2
initrd /boot/grub/minirt_531.gz

title Knoppix 5.3.1 0wn
root (hd1,7)
kernel /vmlinuz init=/etc/init root=/dev/hdb8 ramdisk_size=100000 knoppix lang=us apm=power-off nomce ide1=reset quiet dma libata.atapi=1 SELINUX_INIT=NO nmi_watchdog=0 vga=791 BOOT_IMAGE=knoppix 2

title Backtrack 2
root (hd0,7)
kernel /boot/vmlinuz ro root=/dev/hda8 vga=791
initrd /boot/splash.initrd

title=Gentoo Linux 2.6.22-r2
# splashimage=(hd1,0)/grub/splash.xpm.gz
root (hd1,0)
kernel /boot/kernel-genkernel-x86-2.6.22-gentoo-r2 root=/dev/ram0 init=/linuxrc real_root=/dev/hdb5 ramdisk=8192 agpart video=vesafb:vga=794,mtrr:2,ywrap,kdgraphics splash=silent,theme:livecd-2007.0 quiet CONSOLE=/dev/tty1
initrd /boot/initramfs-genkernel-x86-2.6.22-gentoo-r2

title=Gentoo Linux 2.6.22-r2
# splashimage=(hd1,0)/grub/splash.xpm.gz
root (hd1,0)
kernel /boot/kernel-genkernel-x86-2.6.22--gentoo-r2 root=/dev/ram0 init=/linuxrc real_root=/dev/hdb5 ramdisk=8192 agpart video=vesafb:vga=794,mtrr:2,ywrap,kdgraphics splash=verbose,theme:livecd-2007.0 console=/dev/tty1
initrd /boot/initramfs-genkernel-x86-2.6.22-gentoo-r2

title Slackware 12 silent
root (hd0,5)
kernel /boot/vmlinuz-generic-smp-2.6.21.5-smp rw ramdisk_size=10000 agpart video=vesafb:vga=794,mtrr,ywrap,kdgraphics splash=silent,theme:SlackwareLinux CONSOLE=/dev/tty1 quiet
# earlyprintk=vga, keep
initrd /boot/initrd.gz

title Slackware 12 silent logged
root (hd0,5)
kernel /boot/vmlinuz-generic-smp-2.6.21.5-smp rw root=/dev/ram rdinit=/init ramdisk_size=8192 video=vesafb:vga=794,mtrr,ywrap,kdgraphics splash=silent,theme:SlackwareLinux CONSOLE=/dev/tty1 earlyprintk=vga, keep loglevel=7
initrd /boot/initrd.gz

title Slackware 12 verbose
root (hd0,5)
kernel /boot/vmlinuz-generic-smp-2.6.21.5-smp ro root=/dev/hda6 agpart video=vesafb:vga=794,mtrr,ywrap,kdgraphics,fadein splash=verbose,theme:SlackwareLinux console=/dev/tty1
# ,theme:livecd-2007.0
initrd /boot/initrd.gz

title Sabayon Linux x86 3.4
root (hd0,5)
kernel /boot/kernel-genkernel-x86-2.6.22-sabayon root=/dev/ram0 ramdisk=8192 real_root=/dev/sda6 quiet init=/linuxrc splash=silent,theme:sabayon CONSOLE=/dev/tty1 agpart vga=794
initrd /boot/initramfs-genkernel-x86-2.6.22-sabayon

title Sabayon Linux x86 3.4 console
root (hd0,5)
kernel /boot/kernel-genkernel-x86-2.6.22-sabayon root=/dev/ram0 ramdisk=8192 real_root=/dev/sda6 quiet init=/linuxrc splash=silent,theme:sabayon CONSOLE=/dev/tty1 agpart nox
initrd /boot/initramfs-genkernel-x86-2.6.22-sabayon

title Ooops go back to Windows
rootnoverify (hd0,0)
makeactive
chainloader +1


title find and load NTLDR of Windows NT/2K/XP
fallback 1
find --set-root /ntldr
chainloader /ntldr
savedefault --wait=2

title memdrive duplicated from floppy image file (hd0,0)/sbm.bin
map --mem (hd0,0)/sbm.bin (fd0)
map --hook
chainloader (fd0)+1
rootnoverify (fd0)
savedefault --wait=2

title GRUB4DOS commandline
savedefault --wait=2
commandline

[Harry Kuhman]

btw... My name is John. John Woods. You can use my nick johnrw... but johnnies is unwelcome.
Thanks for the information. I apologize for the name issue. It was quite unintentional, and I'm not sure how I even did it. Didn't know that I had until you pointed it out. I generally quote a posting and then edit it down to the key point or points that I'm responding to. In this case I know copied the post to the clipboard so that I could respond to two different sections, and I honestly have no idea how I managed to replace the rw with nies (and a search of the original text for nies finds no match). I'm very sorry.

[Krellan2]

I did manage, after several searches, to find that someone has put some of the information that I remember seeing in the forums into a wiki article, which is here (http://www.knoppix.net/wiki/USB_Based_FAQ). It has some information that may be helpful, including talking about running grub from a usb flash device.

Hi there. I also have a Wiki page that I wrote a while back, and has been edited by several others over time:

http://knoppix.net/wiki/Bootable_USB_Key


However, I still cannot locate some of the discussion about the different low level formatting options available on a flash device that affect how it behaves like a hard disk. I'm still hoping that the knowledgeable will join in on this conversation.

I have found that there are at least 3 "booting standards" when booting USB mass storage devices:

* USB-ZIP = The most compatible with old BIOS's, but has the most weird restrictions: data must be on partition 4 (not 1), geometry must be 64 heads 32 sectors. Cylinders maximum 1023, thus USB keys booted with USB-ZIP must be 1GB or less in size.

* USB-HDD = Basically relaxes the restrictions of USB-ZIP. Partition can be 1 through 4, geometry can be anything that makes sense (most likely 255 heads 63 sectors). Uses LBA so geometry really doesn't matter and there should be no size limit.

* USB-Superfloppy = I haven't seen a name for this standard, most BIOS's just call it "removable". No partitions used at all. Just format the entire USB mass storage drive as a big single filesystem, without a partition table. This used to be called "superfloppy" format, since floppies didn't have partitions. This is what Windows does by default, when inserting an unformatted USB key. LBA is used, of course, as there is no geometry at all. This is the easiest to make, as you don't need to worry about partitions or geometry at all, but relatively few BIOS's support it.

If trying to make a USB key that's as compatible with as many BIOS's as possible, try USB-ZIP first, and then go down the list.

If trying to make a USB key that is easy to make and able to use all available space (no space lost to uneven geometry), try USB-Superfloppy first, and then go up the list.

BIOS's are very weird when it comes to USB devices. I've seen USB devices appear in entirely different menus, depending on how they are formatted. USB booting is not yet a selling point when it comes to motherboard makers, so they do not advertise or test this feature, and so it's really random whether your motherboard can do it correctly or not. I've seen expensive motherboards fail and cheap motherboards succeed, so it doesn't have to do with price or name brand.

Hope this helps!