PDA

View Full Version : Using kvm to install Knoppix from ISO image to flash and HD, 7.6.1 example.



Capricorny
02-28-2016, 12:22 AM
This is well known, but there are a few refinements that can be useful.
First, connect a USB stick, here it is identified, e.g.by fdisk -l as /dev/sdc. Go to the download directory and boot the ISO image with kvm, connecting the stick. (Here 1024KB memory is used, Knoppix can run with less.)


sudo kvm -cdrom KNOPPIX_V7.6.1DVD-2016-01-16-EN.iso -hdb /dev/sdc -m 1024 -boot d &
Boot Knoppix as a virtual machine, at the boot prompt write knoppix or knoppix64 + cheatcodes (keyboard=... for instance)
Inside the virtual machine, check that everything works fine. You should have internet connection, for example.
Use the 'Install KNOPPIX to flash disk' to install to the USB stick. Inside the VM, it is considered a hard drive and called /dev/sda, but from the size you can see that the correct drive is referenced. You may also create at persistent store at this time.
It takes some time, but after a while, you will get the message that the installation is complete.
Then it is time for powering down the ISO VM.
The preparation of the new Knoppix version can be continued by running the flash in kvm:


sudo kvm -hda /dev/sdc -m 1000 -boot d &
With persistent store in place, you can use synaptic, aptitude or apt-get to update or install packages. Don't forget to update the package listings!
It takes a bit more time to do installs in the kvm VM, but you can continue with your usual tasks on your pc in the meantime, so at least for me, this is a timesaver.
After having updated/installed the most important packages, you can shut down the VM, and copy the knoppix main directory from the USB stick to the pc. If there are already Knoppix directories there, give it a suitable name, like KNOPPIX761_0 (The _0 suffix is in case you, like me, will do remasterings.)
To boot the new version, you will have to modify the bootloader.
I use old, legacy GRUB, where this is very easy - but enough for now :-)

Capricorny
02-28-2016, 12:43 PM
(Continued) Now I have 7.6.1 on a USB stick, with persistent store and a few programs updated/installed. I have one directory on the pc HD where live versions are preferentially stored and run from (/dev/sda1), and if that had been mounted rw, I could have performed all the steps of a poor man's install without rebooting. But some 'live' systems, like the 64 bit Debian I was routinely running here, insist on the directory with the compressed live image being mounted readonly. So I had to boot from the USB stick. (Not too bad, really, that was also a check of that install.)

When booted from the stick, the hard disk partitions for Knoppix installs and grub booting showed up as /dev/sda1 and /dev/sda2. If possible, I never use a partition with booting info for ordinary running - if that gets goofed up, you lose the booting setup for all your installs. I have installed old legacy grub on /dev/sda2, and in the boot directory I have subdirectories for all the OS versions installed.

For Knoppix, I have knx705, knx720, knx742, and now knx761.
In this knx subdirectory, I simply copy the content of the boot/syslinux directory from the stick, then minirt.gz and the kernel images linux and linux64 are in place.

mount /media/sda2
cd /media/sda2/boot
sudo mkdir knx761
sudo cp -ar /mnt/system/boot/syslinux/* knx761
cat knx761/syslinux.cfg
The boot info from syslinux.cfg is used to create a correct booting stanza for grub. The 64-bits version I use as kind of template:

LABEL knoppix64
KERNEL linux64
APPEND lang=en apm=power-off initrd=minirt.gz nomce libata.force=noncq hpsa.hpsa_allow_any=1 loglevel=1

Adding the new Knoppix to the boot menu is as easy as editing old grub's menu.lst:

sudo emacs grub/menu.lst &
This is the entry I added for 64-bits 7.6.1:

title Knoppix 7.6.1 64 bits sda1
kernel (hd0,1)/boot/knx761/linux64 ramdisk_size=100000 fromhd=/dev/sda1 knoppix_dir=KNOPPIX761_0 lang=en keyboard=no apm=power-off initrd=minirt.gz nomce nosound libata.force=noncq hpsa.hpsa_allow_any=1 loglevel=1 tz=localtime
initrd (hd0,1)/boot/knx761/minirt.gz

It tells grub that the kernel is found on (hd0,1)=/dev/sda2, as boot/knx761/linux64, that the initrd image is minirt.gz in the same directory, that it should use Knoppix from /dev/sda1 and the directory KNOPPIX761_0 etc. For 32 bits kernel boot, the same is used, only with linux instad of linux64 as kernel image.
It is a little bit problematic to copy a live system with itself, but, I'll take that chance now:

mount /media/sda1 && cd /media/sda1
mkdir KNOPPIX761_0
sync
cp -ar /mnt/system/KNOPPIX/* KNOPPIX761_0 &
cp -ar /mnt/system/boot KNOPPIX761_0/boot &
The last command moves the original boot directory "out of place", but that way, all relevant information about this version is retained within the KNOPPIX761_0 directory. And it does no harm there.

Now it's time for rebooting and try the new poor man's install. It worked right out of the box, and next step for me was to continue the update/install process. It turned out that quite a few basic programs, like iceweasel and chrome, already had upgrades. Using synaptic (main menu, preferences), reloading package info and looking for packages to update is good practice IMHO - then eventual problems to report will not be because of outdated versions.

This time, quite a few packages could not be upgraded because of broken dependencies - has not experienced so much of that before. After a round of install/upgrade, it is always a good idea to clean up:

sudo apt-get clean

The next step for me now, is to do a few rounds of purging, then install some (for me) important things like vmware, citrix clients, rstudio etc. When the persistent store starts to fill up, it will be time for first remastering.

Capricorny
03-06-2016, 12:38 AM
Just want to report a small surprise: I could run 64 bits Debian Live 8.3 ISO with kvm on Knoppix 7.6.1.

That ISO was not copied into the persistent store, which is almost exclusively used for programs and data connected with them.

I automatically mount a data storage volume (here /dev/sda5) common for all OS versions by adding the following to /etc/rc.local


mkdir store
mount /dev/sda5 /store

joselb
03-06-2016, 09:28 AM
Are you using many GB,s of RAM?

Capricorny
03-06-2016, 10:24 AM
Are you using many GB,s of RAM?

As I commented on above, the -m 1024 parameter specifies 1GB RAM for the VM, but Knoppix can run with less. I'm currently using machines with 8 or 16 GB RAM for such system administration, but I think most operations could be done with 4GB. (And I can't really see the sense in using machines with less as the main workstation today, but YMMV.)

Capricorny
03-06-2016, 02:34 PM
There may be problems with kvm kernel modules not being loaded. There should be two modules loaded, kvm and either kvm-intel or kvm-amd. If you don't know the processor type, and load status just look at the contents of /proc/cpuinfo and output of lsmod, e.g. by using

less /proc/cpuinfo
lsmod | grep 'kvm'

If there are no kvm modules loaded, use modprobe to insert them, in the Intel case:

sudo modprobe kvm && modprobe kvm-intel

A useful feature of kvm is that it can be used to run different OSes installed on the computer in a VM. This can be handy for lots of uses. Using GRUB, this command will give me GRUB's boot menu in a window:

sudo kvm -hda /dev/sda -m 1024 &
And from the boot screen, pick the OS version to boot. If you are running 32 bits kernel, 64 bits OS versions of course won't work.
The problem can be if different installs mount the same partitions rw, and insist on writing to them. But running Knoppix from a separate partition from what you are using can work just fine. You can do all kinds of administrative work in such a window, I even ran a DVD remastering there once. (The objection that DVD remastering takes a lot of time - if it turns out to be correct - becomes kind of moot when you can du it as a background process in a VM.)

Capricorny
03-07-2016, 12:18 AM
I have used the same procedure for getting the newest Debian 8.3.0 live installed with persistence. It took some time getting the details right, as the persistence setup has changed from Wheezy to Jessie.
Now, Debian live has been knoppified quite a bit, so I can specify bootfrom (fromhd), live-media-path (knoppix_dir) and persistence-path (which I don't think has a parallel in Knoppix).
I place the compressed image and the persistence volume in deb830-directories on two different volumes, on the current machine /dev/sda8 and /dev/sda9. (Grub is installed on /dev/sda6.)
As the volume with the compressed image isn't mounted ro any longer, the same volume might be used - I think they might as well be kept separate.
The persistent image is now called persistence, and it must contain a file persistence.conf to be union mounted with the compressed (squashfs) image.
Setting up a persistent image can be done this way (among others)


mkdir deb830 && cd deb830
dd if=/dev/zero of=persistence bs=1M count=3800 # Create the image file
sudo mkfs.ext4 -L persistence persistence # Setup a file system on it, label 'persistence' (may not be necessary, but the principle is that such volumes should be labeled
mkdir /tmp/persistence # Temporary mount point for loop image
mount -o loop persistence /mnt/persistence # Loop-mount the image
cd /mnt/persistence
sudo chmod a+w .
echo '/ union' > persistence.conf # Create a persistence.conf file
cat persistence.conf # Check it
sync
cd -
sudo umount /tmp/persistence # Now the image should be ready for use

The Debian live kernel is called vmlinuz (linux/linux64) and the initrd (minirt.gz) is initrd.img. Those files are copied to a deb830 directory under boot.
My legacy GRUB stanza in boot/grub/menu.lst for booting then becomes


title Debian 8.3.0 64 bits live sda8+9 1
root(hd0,8)
kernel (hd0,5)/boot/deb830/vmlinuz boot=live config quiet splash initrd=(hd0,5)/boot/deb830/initrd.img persistence bootfrom=/dev/sda8 live-media-path=deb830 persistence-path=deb830 keyboard-layouts=no
initrd (hd0,5)/boot/deb830/initrd.img


Then I can use kvm for the rest of testing and setup:

sudo kvm -dev/sda -m 1024 &
Choosing Debian 8.3.0 from the boot menu will give me a functional Debian live with persistence running in a window. In particular when there is a lot of testing and troubleshooting to do, this can be very helpful. There is also a "sandbox" element to this: The risk for doing damage during the development and testing is minimized. Creating several deb830_X directories each with a persistent image, and creating boot stanzas for them allows working with several versions in parallel - which may be very time-saving, for example when some installation projects end up being rather futile.

Capricorny
03-13-2016, 04:49 PM
Note about Debian 8.3.0 - in standard Jessie, both kernel (vmlinuz) and initrd.img may be patched upon packages install, e.g. firmware-linux-nonfree. The boot entries will therefore have to be updated with those patched files. Otherwise, the package system will report normal install, but system will work as before.

Capricorny
12-22-2016, 03:21 AM
Procedure works for 7.7.1, I ran it from Debian 8.3. Mounting a USB stick which was recognized as /dev/sdb for creating a bootable USB version with persistent store.


sudo kvm -cdrom KNOPPIX_V7.7.1DVD-2016-10-22-EN.iso -hdb /dev/sdb -m 1024 -boot d &


Everything went just fine, though the creation of 4GB persistent store was slow on a modest 6 year old i5 processor. It was probably not the fault of the USB3 stick, for system transfer was quite fast.

I encountered a small problem on a new HP ProBook 450 G3 PC: Virtualization was disabled in the BIOS, so I had to activate it. Anybody encountering problems with probing/inserting the kvm_intel or kvm_amd modules should check for such disabling. Some CPU types don't support virtualization at all, but most do.

I tested the procedure also for a Devuan-based refracta 8-ISO. While I could partition and format the USB-stick from inside the refracta-VM runnng under kvm om Knoppix 7.6.1, copying did seemingly not work right out of the box in that version - looked like it did not get what to copy :-)

Capricorny
09-10-2017, 12:14 PM
Installing a fully functioning live Debian 9.1.0 on the HP ProBook 450 G3 was a bit of a challenge, as the Intel 3165 wireless adapter required non-free + drivers + firmware. Using Debian 8.3.0 can not be recommended on this hardware without an upgrade from kernel 3.16 - Debian 9.X stretch uses 4.9 kernels.
So what I did, was placing the squashfs live file in one directory, creating a persistence file as described above on another, making av boot stanza for legacy grub, and then running the whole system under kvm on Knoppix 7.7.1 for setup, using
sudo kvm /dev/sda -m 2046 &. But in this case, wired network did not work right out of the box under kvm (first time ever I have experienced this). The simple workaround I used, was that on every start I had to patch /etc/network/interfaces with the correct "eth0" version, in this case ens3 which I found from the output of
ifconfig -a. I created a small script for this:

#!/bin/bash
echo 'auto ens3' >> /etc/network/interfaces
echo 'iface ens3 inet dhcp' >> /etc/network/interfaces
/etc/init.d/networking restart

Then wired ethernet worked, and I could do all the installs online in the virtual machine. Afterwards, I booted Debian 9.1 natively, and from the
iwconfigoutput I found out what they had invented as a name for wlan0. Telling wicd about this, I got wireless up and running.

There are a few reasons I have to run "pure" Debian, but Knoppix has, again, turned out to be an almost indispensable tool to get a "poor man's install" of Debian up and running.

I still use old, legacy grub for booting - gives me very few problems. In Debian 9.1, kernel and initrd does not seem to be patched to accomodate non-free firmware and modules.

Capricorny
09-10-2017, 06:38 PM
For the record, here is the script used to remaster Debian Live 9.1.0. Basically, it performs the aufs union mount, copies the whole thing to a directory with rsync, and then compresses the result. Squashfs-tools are needed. The whole operation takes only a few minutes on a contemporary i7 machine with SSD disk.


#!/bin/bash
#
[ -d sqfs ] || mkdir sqfs ;
[ -d persfs ] || mkdir persfs
[ -d aufs ] || mkdir aufs
[ -d copyfs ] || mkdir copyfs
#

# Running under Knoppix 7.7.1 w/Debian 9.1 image on same partition as Knoppix, persistent store at /dev/sda9
[ -d /media/sda9/deb91_1p ] || mount /media/sda9
fsck.ext3 -y /media/sda9/deb91_1p/persistence
mount -o ro,loop /mnt-system/deb910/live/filesystem.squashfs sqfs
mount -o loop /media/sda9/deb91_1p/persistence persfs

mount -t aufs -o "br=persfs/rw=rw:sqfs=ro,noplink" none aufs
#
#
# Clean up, in case
rm -rf copyfs/*
# Copy union file system
rsync -axu aufs/* copyfs
# Clean up mounts
umount aufs
umount sqfs
umount persfs

#
# Compress file system to new live file
mksquashfs copyfs filesystem_rem_1.squashfs

Afterwards, the new live file is copied to a new directory, a new persistence file is created in a new directory, and a new grub entry is created with the new locations. This way, it is possible to go back to an earlier version on the fly if somethings goes very wrong. At irregular intervals, depending on how much is changed, the new persistence file is backed up.

Capricorny
11-11-2017, 11:46 AM
Have just repeated the procedure with Knoppix 8.1 under Debian 8.3 live. A bit more glitches than earlier, but it has worked.
Downloaded the ISO+md5sums and went to the directory. Kvm had no problems running the ISO, and after checking that things seemed to work, and that the flash was recognized as /dev/sdb, I used flash-knoppix to create a bootable flash install.
Used the old setup with persistence file, no encryption.



user@debian:/store/download/iso$ md5sum -c KNOPPIX_V8.1-2017-09-05-EN.iso.md5
KNOPPIX_V8.1-2017-09-05-EN.iso: OK
user@debian:/store/download/iso$ sudo kvm -cdrom KNOPPIX_V8.1-2017-09-05-EN.iso -hdb /dev/sdb -m 1024 -boot d &


I had some problems with the choices in the flash-knoppix setup. We must choose fixed disk, for that's what the flash drive looks like from kvm Knoppix. But I had to double click to choose that.
Also, it complained that it couldn't get the geometry for the persistent image right. The file was created, but file system creation somehow went wrong.
So, I had to do a mkfs.ext2 on the knoppix-data.img file afterwards. From there, things seem to work fine.

But, to get the stick to boot properly and consistently on a Windows PC, I had to use a USB2 port - the recognition in the USB3 ports seemed a bit random. Maybe it has to do with the drive, a 32 GB HyperX Fury.

There is an interesting new option for remastering in flash-knoppix. Will try it out from a Poor Man's Install. Will also try this whole procedure under Knoppix, and under Debian 9.1.

Capricorny
11-11-2017, 02:34 PM
Posting this from 8.1 Poor Man's Install. For the first time, I have had to manually set screen resolution - wrong detection left me with an unusable Knoppix.
But, as so often, the knoppix-cheatcodes.txt file helped me out. What I needed, was a specification screen=1920x1200. One might think the norandr alternative would do, but it didn't.

Here are the Knoppix 8.1.0 boot stanzas from legacy grub's menu.lst file. I have added a 1280x1024 frame buffer alternative to have a fallback.



title Knoppix 8.1.0 64 bits sda1 cloop no persistence fixed screen
kernel (hd0,5)/boot/knx810/linux64 screen=1920x1200 fromhd=/dev/sda1 noimage knoppix_dir=KNOPPIX810_0 nosound keyboard=no ramdisk_size=100000 lang=no apm=power-off initrd=minirt.gz nomce hpsa.hpsa_allow_any=1 loglevel=1 tz=localtime
initrd (hd0,5)/boot/knx810/minirt.gz

title Knoppix 8.1.0 64 bits sda1 cloop fixed screen
kernel (hd0,5)/boot/knx810/linux64 screen=1920x1200 fromhd=/dev/sda1 knoppix_dir=KNOPPIX810_0 nosound keyboard=no ramdisk_size=100000 lang=no apm=power-off initrd=minirt.gz nomce hpsa.hpsa_allow_any=1 loglevel=1 tz=localtime
initrd (hd0,5)/boot/knx810/minirt.gz


title Knoppix 8.1.0 64 bits sda1 cloop fbdev
kernel (hd0,5)/boot/knx810/linux64 fromhd=/dev/sda1 knoppix_dir=KNOPPIX810_0 vga=794 video=1280x1024 xmodule=fbdev nosound keyboard=no ramdisk_size=100000 lang=no apm=power-off initrd=minirt.gz nomce hpsa.hpsa_allow_any=1 loglevel=1 tz=localtime
initrd (hd0,5)/boot/knx810/minirt.gz

By specifying knoppix_dir, and eventually fromhd, I can multi-boot as many Knoppixes I want. I place the kernel and initrd files in separate directories under boot on the partition where grub is installed.
One important reason for having noimage ie. no persistence, boot alternatives, is that I can then backup or copy the persistence file "from within".

Right now, I have 5 live OS alternatives for booting on this old PC: Knoppix 7.05, 7.4.2, 6.7.1 and 8.1.0, and Debian 8.3.0. The old stuff is for running program versions or hardware drivers that can no longer be installed under newer OS versions.

Capricorny
11-11-2017, 06:10 PM
The problem with flash-knoppix not recognizing drives seems to depend on the type of drive. An old, quite slow 16GB USB2 stick (Kingston DataTraveler G2) was handled straight away. But the problem with creating file system on the persistent image correctly, persists. Not that hard to fix, though - boot the stick, go to /mnt-system/KNOPPIX and do a mkfs.ext2 on knoppix-data.img, reboot and the new persistent store is used.
Flash-knoppix isn't all that bright - looks like it won't transfer the system unless Knoppix resides in a KNOPPIX directory - which it should not do if we are doing multi-booting. Which in turn is necessary with each upgrade at least until we have verified that all important things work well in the new version.

Capricorny
07-03-2018, 01:18 PM
Update for Knoppix 8.2: The same procedure also works with Knoppix 8.2, but there were a few complications I didn't really understand. I also tried the "hybrid" direct dump option, but didn't get a working system from that. Running the ISO image under kvm and copying to flash disk worked fine, but going back to hard disk, several things did not work properly. When i did a lot of upgrading/installing (ca 2GB worth) and wrote a "remastered" image to flash, leaving me with KNOPPIX, KNOPPIX1 and KNOPPIX2 cloop files, the final system worked perfectly, with an almost empty persistent store. Seemingly no problems with hardware detection or graphics - it may have to do with X and other components having been upgraded.
Installing a running system to flash with the "remastering" option can be a great timesaver - in return for a couple of gigabytes extra space used, we don't have to do remastering ourselves, and the whole system is still FAT32-compatible. Which my standard squashfs Debian 9.1 image will not be after the third remastering. A 16GB flash stick will be more than enough for all the space requirements of a Knoppix release.