Bootable USB Key


From Knoppix Documentation Wiki
Jump to: navigation, search

Admin Notice: You can create your own copy of Knoppix on a USB memory stick using the instructions here and at the USB Based FAQ. You could also buy it (5.3.1 and 6.0.1) from $14.95 delivered.

Intro

These instructions were originally written for Knoppix 5. I tried early versions of Knoppix 6 and it was so stripped down that I did not want it on my USB key, as many of the useful tools were no longer there. However, Knoppix 6.2.1 is once again useful, and these instructions also work (with a minor change described below) for Knoppix 6 and 7.

Knoppix 6 and 7 also have their own built-in utility to create a USB key after booting from the CD, so using that might work easier. Also, the UNetbootin package can perform this installation automatically for you. However, if you want to learn how these tools work their magic, read on.

Even though Knoppix 5 is rather out of date these days, I still believe it set the high water mark for being packed full of useful software, both for system repair/recovery purposes, and for general productive use. Later Knoppix versions stripped out much of this functionality, unfortunately. However, they have updated Linux kernel and drivers, so will work better with newer hardware. Choose whichever is best for your needs. The beauty of having your operating system boot from a USB key is that it becomes easily interchangeable, unlike a permanent installation on your hard drive!

Knoppix 5 and newer boot rather nicely from a USB key. Knoppix 5 will work better on older hardware, and has a larger selection of preinstalled software. Knoppix 6 has better support for newer hardware, and does a better job of handling your encrypted home directory on USB. Knoppix 7 is the latest and greatest. Choose whichever you prefer.

This is a "recipe" for creating a Knoppix bootable USB key. It was originally posted to the forums: http://knoppix.net/forum/viewtopic.php?t=25402

Preparation

  • Download a Knoppix image and burn it to CD or DVD.

For Knoppix 5, I recommend version 5.3.1 for DVD, or version 5.1.1 for CD. For Knoppix 6, I recommend version 6.2.1 for both DVD and CD. For Knoppix 7, use the latest version (I have tested only 7.2.0). Choosing DVD or CD is up to you. It's a function of how large your USB key is, and how much free space you want to have remaining on the USB key after completing installation of Knoppix onto it.

  • Back up all data on your USB key, because we will be erasing and reformatting it!
  • Make sure SanDisk U3 is not installed on the USB key

Unfortunately, many newer USB keys come with "U3" Windows software (badware?) preinstalled. Removing U3 is highly recommended, and will free up additional space on your USB key, as well as eliminate a source of potential conflicts.

Find a Windows machine, and try these programs to uninstall U3. Unfortunately, they are a bit flaky. If one does not work, try the other:

http://www.sandisk.com/Assets/u3/launchpadremoval.exe

http://www.u3.com/uninstall/

To my knowledge, U3 has not yet been reverse-engineered, and the U3 vendors are secretive, so there's no way to manipulate it from Linux.

The U3 removal software searches for a valid U3 installation before removing it! There seems to be no way to just wipe U3 cleanly away, unfortunately. You have to infect your computer with U3, in order to use their software to clean your USB key. About the only workaround for this Catch-22 situation is to avoid purchasing a USB key with U3 in the first place.

Newer SanDisk USB keys come with a built in uninstall application. Try that first, before trying the above solutions.

Fortunately, SanDisk has listened to customer outcry, and has stopped infecting their USB keys with U3. Newer USB keys should be U3 free, even from SanDisk.

Instructions

  • Boot Knoppix from CD (or DVD), and make sure that it works.

As soon as the title screen of Knoppix appears (the cheatcode entry screen), put in the USB key.

Do this before Knoppix starts to load, so that your USB key is sure to be found by Knoppix's autodetection.

Do not use the "toram" cheatcode when booting Knoppix. The RAM copy of the CD omits the /boot directory, but we require this directory, so we can't use the RAM copy.

  • Once Knoppix is up and running, open a root shell (from the penguin icon).

Now, we need to find the drive letter of your USB key. It will be something like /dev/sda, /dev/sdb, /dev/sdc, and so on. Be very careful and make absolutely sure you have the drive letter correct, because you will be erasing the USB key, and you don't want to erase your hard drive by mistake!

Examples of possible commands to try:

dmesg | grep -i scsi
dmesg | grep -i usb
cat /proc/partitions
fdisk -l
lsscsi -k

Look at the size of each drive and match it to the size of your USB key. If you confuse your hard drive with the USB key, you will destroy your hard drive installation with the instructions that follow. So, here is a little more info about identifying the USB key:

USB keys are identified by the Linux kernel as a SCSI device (as implied by the preceding info). If your system has a SCSI hard drive, chances are you know about it already. If you only see one SCSI device (i.e. /dev/sda, /dev/sda1 or some other digit located where there is a "1", here) and you are certain the USB key is not damaged, the odds favor the possibility that you just found the USB key. If you do SCSI drives in your machine, the chances are that the kernel recognized them during boot and assigned the first (then second, and so forth) available identifier to these drives, which means the very last one that you see is probably the USB key. The most certain method of identifying the key is by using "fdisk -l". Any SCSI devices will appear in this output, as the last devices to be shown.

For the rest of this example, I'm going to be using /dev/sdz, a fictional drive letter that nobody has, just to prevent bad mistakes from typing in commands blindly.

Note 
If your USB stick is FAT-formatted and has enough free capacity, you can skip the following repartitioning phase and jump to installing SYSLINUX section. Just make sure your stick's partition has it's boot flag activated so the boot loader works properly. This can be done with the command fdisk /dev/sdz and subsequent commands (example: a 1 p w )
Note
If you're repeating your installation and have already zeroed your USB key at least once, then you can add bs=1k count=16k to the line below. It becomes dd if=/dev/zero of=/dev/sdz bs=1k count=16k which will just zero out the first few sectors of your USB key. This avoids wear and tear on your USB key, but still writes to enough sectors to clean out any existing formatting on your USB key.
Note
Newer USB keys larger than 8GB often work best with "superfloppy" format, see below.
  • Erase your entire USB key.
dd if=/dev/zero of=/dev/sdz

Did you use the right drive letter? Did you? If not, you'll be crying, right about now....

This command will take a while to finish. By writing to every block on the USB key, we are also testing it for bad blocks.

You should get the "no space left on device" error message when it finishes erasing and reaches the end of the drive. Any other error messages might mean bad blocks: if you notice any, your key might be worn out and you'll need to buy another one (cheap keys are notorious for wearing out very quickly).

  • Run the mkdiskimage script, to automatically partition and format your USB key.

Choose only one of these sections to try, based on the size of your USB key.

If your USB key is exactly 1GB in size:

mkdiskimage -4 /dev/sdz 0 64 32

Here are some alternatives you can try, for larger USB keys. If the above command worked for your USB key, then you do not need to try these! Simply proceed to the SYSLINUX step below.

If your USB key is 2GB or less in size, but larger than 1GB:

mkdiskimage -4 /dev/sdz 0 128 32

If your USB key is 8GB or less in size, but larger than 2GB:

mkdiskimage -F -4 /dev/sdz 0 255 63

If your USB key is larger than 8GB in size, first, try the same command as above:

mkdiskimage -F -4 /dev/sdz 0 255 63

If your USB key is larger than 8GB in size, and the above command didn't work (you got "divide by zero" or similar error), next, try this sequence of commands (see cautions below):

mkdiskimage -F -4 /dev/sdz 1 255 63
dd if=/dev/zero of=/dev/sdz bs=1 seek=446 count=64
echo -e ',0\n,0\n,0\n,,C,*' | sfdisk /dev/sdz
mkdosfs /dev/sdz4

The final section above, for keys larger than 8GB, is rather tricky. The mkdiskimage script can't handle partitions larger than 8GB, so I have to hack around it a bit. (This bug might be fixed in Knoppix 7, as it seems to happen less often, but I've still seen it happen.) What we are doing here is making a dummy partition of only 1MB(!) in size, so that mkdiskimage can prepare the MBR properly. Then, we manually delete that partition, by doing some surgery in the partition table. We then use sfdisk to re-create the partition properly, using the full size of your USB key as FAT32, and finally format it with mkdosfs.

Please be very careful typing in the command(s) in your section above. If using the final section for keys larger than 8GB, the 0's in the echo command above are zeroes, not the letter O, and please be sure to use single quotes instead of double quotes. This should work for all USB storage devices, up to 2TB in size. I tested this using a 320GB USB hard drive, and it worked. Beyond 2TB, you're on your own....

This lays down a new MBR and partition table on the drive, and partitions and formats the entire drive as one big partition, either FAT16 or FAT32. Even though I prefer FAT32, I'm still recommending FAT16 for smaller keys, for increased compatibility with some strange BIOS's.

The mkdiskimage script is USB-ZIP compliant, which may help it boot on more systems than USB-HDD. It's nice that Knoppix now includes this script by default. Here's the webpage for mkdiskimage, for additional reference:

http://syslinux.zytor.com/usbkey.php

  • Close KDE's popup window.

KDE may throw a stupid popup window in your face, once it recognizes your new partition. If this happens, close the window.

  • Run the SYSLINUX installer on your new partition.
syslinux -s /dev/sdz4

This installs the SYSLINUX bootloader onto the newly-created partition, and includes a special file, ldlinux.sys. The -s option is for safety and additional compatibility, although if you want your USB key to boot a little faster, you can omit the -s option. Notice the partition is the fourth partition on the drive, not the first. This is for USB-ZIP compliance.

  • Mount your new partition.
mount /dev/sdz4 /media/sdz4

Although Knoppix should have autodetected the new partition and set up a fstab entry for it, Knoppix's fstab options are incorrect. So, by giving both the device and the mountpoint directory on the command line, we mount it manually, with no options. The /media/sdz4 directory should now correspond to the /dev/sdz4 partition.

cd /media/sdz4
ls

You should notice one, and only one, file in this directory now: ldlinux.sys

If you don't see this, then something probably went wrong earlier. Go back and try again before continuing.

  • Copy the contents of the Knoppix CD's boot directory, onto your USB key.

You should already be in the /media/sdz4 directory, from the cd command above. The rsync command is very fussy about spaces, dots, and slashes, so be careful you've typed in the command correctly.

If you're running Knoppix 5:

rsync -av --progress /cdrom/boot/isolinux/ ./

If you're running Knoppix 6 or 7:

rsync -av --progress /mnt-system/boot/isolinux/ ./

Note the different location where the files on the CD appear. In Knoppix 5, it's the /cdrom directory, but in Knoppix 6 and 7, it's the /mnt-system directory.

This copies all files from that directory to the current directory (the rsync command is an improvement over the cp command for handling large amounts of files, as it is better at handling errors). Ignore any warnings about "failed to preserve ownership". This is normal, because we're using the FAT filesystem, which does not support file ownership. This is not a problem for us.

Finish up:

mv isolinux.cfg syslinux.cfg
rm -f isolinux.bin

What we're doing here is copying the Knoppix CD's /boot directory first. This increases reliability, by getting it as close to the beginning of the drive as possible. This should help on systems with buggy CHS handling of USB keys, which may become an issue as new USB keys get larger.

We are renaming Knoppix's isolinux.cfg file to syslinux.cfg, and then removing the unnecessary isolinux.bin file. ISOLINUX (used to boot the CD) and SYSLINUX (what we just installed to your USB key) are really two different variations of the same bootloader program. They accept the same config file format, so we don't need to change anything at all, just rename.

BTW, since ISOLINUX has isolinux.bin as its main program file, can you guess what is the equivalent file for SYSLINUX? If you guessed ldlinux.sys, the file you saw earlier, good job!

  • Finish copying the rest of the Knoppix CD.

Again, you should still be in the /media/sdz4 directory. Again, don't forget to doublecheck your typing of the spaces, dots, and slashes in the rsync command.

If you're running Knoppix 5:

rsync -av --progress /cdrom/ ./ --exclude boot

If you're running Knoppix 6 or 7:

rsync -av --progress /mnt-system/ ./ --exclude boot

As with the earlier rsync command, select one of the above, depending on your version of Knoppix.

This should take a while -- we are filling up the USB key with the contents of the Knoppix CD. The excluded "boot" directory was already copied earlier.

You will see a speed readout of how fast the files can be installed to your USB key. Some keys are much faster than others. Getting about 2MB per second is reasonable. If your speed is extremely slow, such as getting KB per second instead of MB, check your USB ports, your hardware might be USB 1 instead of USB 2. Avoid USB 1, it is frustratingly slow.

  • You are now ready to boot!

Cleanly unmount the USB key.

sync
cd /
umount /media/sdz4
sync

Shut down Knoppix, and remove the CD when it tells you to.

  • Reset your computer and fiddle with the BIOS until it sees the USB key.

After unmounting above, the USB key can be removed. Some BIOS's require a removal and re-insertion before they will recognize it.

Other things to try, if you're having trouble getting the BIOS to see your USB key and let you boot from it:

  1. Try a different USB port. Some BIOS's only search a few ports, not all of them. Avoid using a USB hub, if possible.
  2. Hit a special key to bring up a boot menu. Some BIOS's can only boot from USB when manually selected.
  3. Go into your BIOS's setup menu, and walk through the boot settings. Look under all menus, especially "hard drives" and "removable devices" and other things like that. Some BIOS's will not search for newly attached devices until you manually go through the menus again.
  4. Enable "USB Keyboard and Legacy Support" if it is disabled.
  5. Turn your computer completely off and on again. Some BIOS's require you to physically turn the computer off and on again, before they will see newly attached "hard drives".
  6. If all else fails, try a different PC. It's still fairly rare these days to find a computer that has a good ability to boot from USB devices, and it's not something that motherboard makers advertise, so it takes a bit of luck to find a computer that can do this.
  • Boot into Knoppix from your USB key.

The USB key will be slow to load at first, especially during the cheatcode screen, because most BIOS's will default to running USB devices in a very slow mode. Once Knoppix detects and autoconfigures the USB port during booting, though, it will return to normal speed.

If it works, that's great. Notice how much faster it is. There's zero seek time on a USB key, so even though it isn't very fast, it feels a lot faster than the CD. Programs are surprisingly quick to load.

Find the drive letter for your USB key again. It should not change, but it might have.

  • Make the USB key writeable, if running Knoppix 5. Knoppix 6 and 7 autodetect the usage of USB keys, so this step is not necessary.

Knoppix 5 assumes that the "CD-ROM" that it booted from is impossible to write to, but we have a USB key now, not a CD-ROM. So, we must fool Knoppix 5 into letting you write to it.

mount -o remount,rw /cdrom
mount --bind /cdrom /media/sdz4
  • Optional step: Modify the Knoppix boot menu

This is an optional step, for customization. We want Knoppix to autofind the persistent home directory (once we make it), and fix a few other things.

vi /media/sdz4/syslinux.cfg

On line 3, the default timeout is 300. Lower this, to around 50 or so (5 seconds, since this counter is in tenths of seconds).

It's nice to have it boot faster, without waiting for you to hit Enter, so you can put in the USB key and walk away, coming back in a few minutes to a fully booted system.

On line 2, add a few more options to the end of the line.

noswap noeject noprompt dma home=scan

Here's a little explanation of what each option does.

noswap 
Security fix: assuming you're going to be using the USB key on a bunch of different computers you don't own, you don't want to be swapping your memory onto their hard drives.
noeject 
You can't physically eject a USB key.
noprompt 
Gets rid of the "remove CD and close drive door" prompt when shutting down, as this prompt is rather misleading when it's really a USB key instead.
dma 
Gets you a significant speed increase when dealing with many IDE drives, a good thing to have.
home=scan 
This will cause Knoppix to pick up your persistent home directory automatically the next time you boot up, asking you for the password as needed. You will get an extra menu during bootup, asking you if you want to mount your home directory or not.

Save the file and quit the editor.

  • Optional step: Create a Knoppix persistent home directory, if running Knoppix 5. Knoppix 6 and 7 will automatically prompt you for this information when you boot it from your USB key for the first time, so this step is not necessary.

Now, we can run the Knoppix 5 "Create Persistent" utility, to create something that will save your settings across a reboot.

(It's under the penguin menu, Configure, Create Persistent.)

Choose your drive from the menu that appears, and follow the instructions.

When making your encrypted persistent home directory, choose a size that is a good balance. I recommend not using the entire remaining space on the USB key. It's handy to have some extra free space, for quick file copying and such. Also, if you have a large USB key with roughly 260MB of free space or more, and need to also work under Windows, you can install PortableApps (see below).

The program should complete without errors. If the program suddenly disappears midway through, check the kernel logs with "dmesg" again. If you see pagefuls of SCSI errors and such, beware, you might have a worn-out USB key!

You're done!

Now, you should be all set. Reboot again, to make sure it all works.

Enjoy your Knoppix-on-a-keychain!

Josh

--Krellan 00:43, 10 Sep 2006 (GMT)

Large USB keys

The mkdiskimage command is very sensitive to the size of your USB key. I have included several variations to try, in the above article. Here are some more alternatives, for completeness.

These will break USB-ZIP compatibility, as it is a USB-ZIP requirement to have 64 heads and 32 sectors. Unfortunately, there is no other way, as a valid cylinder count (less than 1024) is also a requirement. So, try USB-HDD instead of USB-ZIP. The tradeoff is that it won't be as compatible with as many BIOS's as USB-ZIP.

mkdiskimage -4 /dev/sdz 0 255 63

This will allow you to partition a USB key up to 8GB in size. You probably could also use 256 heads instead of 255, for a little more space, but 255 heads is recommended, as it's slightly more compatible.

You may then run into the 2GB limit of the FAT16 partition. Beyond 2GB, it is probably best to use FAT32 instead. Supply the -F option to mkdiskimage.

mkdiskimage -F -4 /dev/sdz 0 255 63

At first, I did recommend FAT32 for all users: http://www.knoppix.net/forum/viewtopic.php?t=25402

However, it did not work on as many BIOS's, so I now recommend FAT16, unless you have a USB key that is larger than the 2GB limit of FAT16.

The other instructions on the page above should still be valid. Only the mkdiskimage command line needs to be changed, for use with large keys.

--Krellan 20:30, 19 Oct 2006 (GMT)

PortableApps

Here's also something great I found. If you have roughly 260MB or more of free space remaining on your USB key, you might want to drop PortableApps on there.

http://portableapps.com/suite

http://portableapps.com/

This is a suite of freeware Windows utilities, and will help you if you need to do work within Windows without being able to reboot into Knoppix. Much better and cleaner than U3!

When installing PortableApps to a USB key that already has Knoppix on it, PortableApps will complain about overwriting the autorun.inf file. This is fine. Say Yes to the overwrite. This will allow the USB key to bring up the PortableApps menu, instead of the Knoppix homepage, when clicking on it from within Windows.

With a 1GB key formatted as FAT16, as per the above instructions, there is, unfortunately, just barely not enough room for both Knoppix and the full PortableApps suite to coexist. The workaround is to "install" PortableApps to a spare empty folder on your hard drive first. Make edits to that folder as necessary, deleting some apps you can live without. Then, copy the entire folder manually to your USB key from there.

Formatting as FAT32 will work, as FAT32 is more efficient at cramming in the thousands of small files PortableApps creates. A larger USB key will provide much more room for both Knoppix and PortableApps, and enough extra left over for a decently-sized Knoppix home directory as well. See above for alternate mkdiskimage command lines you can try, when formatting larger USB keys or using FAT32.

--Krellan 19:23, 27 Nov 2006 (GMT)

Superfloppy format

These days, USB keys larger than 1GB are common, and newer computers have improved BIOS support for booting from USB keys. Therefore, USB-ZIP partition layout is not as necessary as it used to be, and in fact, can become a hindrance! I am really starting to prefer the simplest partition layout of all: superfloppy format.

As the name implies, this works by just formatting the entire USB key device as one big filesystem, without a partition table at all. It's easy to do this.

Clear the first 16MB of the USB key first (see the next section below for why):

dd if=/dev/zero of=/dev/sdz bs=1k count=16k

The mkdosfs command will create your filesystem. When using the mkdosfs command, just use the "I" option, and give the device name containing the drive letter, without a partition number at all. That causes the entire device to be formatted directly, as if it were one big partition, and you get a little more space as well, because you don't need to reserve any space for the partition table. The command is this:

mkdosfs -I /dev/sdz

Now, run the sfdisk command, to inform the kernel that you have just changed the filesystem, and that it should rescan the disk for changed partitions/filesystems. The -R option to this command doesn't change what's on the disk, instead, it just forces the kernel to do a rescan of the disk so that it can detect your new filesystem.

sfdisk -R /dev/sdz

You can then proceed with the rest of the instructions above, starting at the syslinux command. The mkdiskimage sequence of steps is no longer necessary!

Whenever the instructions mention a drive letter with a partition number, for example /dev/sdz1, you no longer need to supply the partition number, because there isn't one. Just use the drive letter instead, /dev/sdz, in all of the commands. As an example, here's the SYSLINUX command to use:

syslinux /dev/sdz

As you can see, I'm also experimenting with leaving out the -s option to syslinux. It is much faster, but might be less compatible with some BIOS's.

--Krellan 00:11, 19 March 2009 (MDT)

Letting Windows format it

Some BIOS's are cranky about superfloppy format, unfortunately. Even on brand new hardware, I've seen really bad behavior from the BIOS supplied with "Shuttle" brand computers. It's pathetic, really, how a company can spend so much money developing hardware and then put essentially zero effort into writing a decent BIOS for it. Anyway, this BIOS seems to work best with USB keys that have been completely formatted by Windows (most likely, that's all they bothered to test with).

To let Windows format a USB key, the key must be zeroed, otherwise Windows will attempt to use what's already there and not let you reformat it. Ironically, zeroing the key is something that can't be done under Windows. It's not necessary to erase the entire key. I've observed that Windows only writes data to about 16MB when formatting, so I'm guessing that 16MB is sufficient to get a clean reformat from Windows. In Knoppix, the command to zero the first 16MB of the key is this:

dd if=/dev/zero of=/dev/sdz bs=1k count=16k

Then, remove the key from Knoppix and put it in a Windows machine, and format it under Windows. You can then return to Knoppix and make the key bootable for Knoppix, just don't change anything about the partition or filesystem that Windows has applied to the key.

Inspect the partition that Windows has applied to the USB key (most likely /dev/sdz1), and then go back to the syslinux command above. Follow the steps from there. For example, to apply SYSLINUX to the /dev/sdz1 partition, the command is this:

syslinux -s /dev/sdz1

Sometimes, ironically, Windows chooses to format the USB key in superfloppy format! *Shrug*

--Krellan 00:05, 19 March 2009 (MDT)


There is an another project based on PortableApps, with the addition of TOR-networking enabled by default. This makes it possible to anonymize web surfing at the cost of speed.

More info at http://www.democrakey.com/

--Neva 18:54, 20 Jun 2007 (GMT)


mkdiskimage won't set up a 2 gb USB device with the USB-ZIP format using the example given. According to other FAQ's this is an issue with too many cylinders. A 1 gb drive is supposed to work fine. You may be able to manually create two drives on the device and manually format it in order to set it up as USB-ZIP.

Depending on the USB device, you can use fdisk to figure out the cylinders, heads and sectors and get mkdiskimage to work with these values. The one I used had 1015 cylinders, 64 heads and 62 sectors per track -- these values worked fine and I was able to create a bootable Knoppix USB key.


I was able to use this guide to setup and boot from Knoppix on a Creative Zen Vision:M MP3 player. Details here: http://www.francisshanahan.com/detail.aspx?cid=509


For the Cylinders/Head/Sectors issue, an easy way to find the information is simply type: hdparm /dev/sd? Where '?' is the drive letter. It will output the "Geometry: x/y/z" where X=Cylinders, Y=Heads, Z=Sectors. Enter those in for the last three variables on the mkdiskimage command and you are all set.

--Geekner 18:53, 7 Oct 2007 (UTC)

When running Knoppix 5.1.1, and booting from your USB key, sometimes udev hardware detection will fail, or lock up entirely. This is strange, and I don't know why this happens. This doesn't happen when booting from the original CD.

The workaround is to use the "noudev" cheatcode. However, when using this cheatcode, fewer of your peripherals will be detected correctly. It's better than having the system lock up, though. If this cheatcode is necessary for you, consider editing it into your syslinux.cfg file as described above, so that you won't need to enter it every time.

--Krellan 10:55, 18 Oct 2007 (UTC)