Successfully remastered 3.9 with rebuilt kernel and unionfs

[Pallbearer]

I finally remastered 3.9 with a corrected unionfs module! This was an arduous journey. I surfaced from this mile-deep Linux rabbit hole with the following notes and observations:

• Remastering was easier from a 3.9 HDD installation. I initially tried from the booted CD using a mounted drive for the source and master folders; however I had problems due to my lack of experience with chroot. I later learned to mount the proc file system with “mount –t proc /proc /proc” to gain network access and updated /etc/resolv.conf. My long term goal is to remaster from a CD with chroot and contain the remaster scripts on the CD. Even better, not have to reboot by releasing the CD-ROM and burning the updated image. Perform the burn after testing with qemu first, of course.

• I retrieved the unionfs-source and rebuilt it. First I tried with “m-a a-i unionfs-module”, then directly from the source in /usr/src/unionfs. I later installed the 2.6.11 kernel source and rebuilt manually using old .config file. Finally, I discovered the Makefile in /usr/src and ran “make” from there. It covers everything including the sources in /usr/src/modules. Note I tweaked the command line parameters to 512. Finally I used “dpkg –i” to install the new unionfs-module deb file.

• The kernel patch in /usr/src was EXTREMELY outdated. The only thing you need to update is the parameter length define in i386 path include file. Or directly in the setup.c file.

• The resulting unionfs.ko module is HUGE and will not fit in the mounted uncompressed minirt file system. You will need to strip the module, BUT not completely. Run “strip --strip-unneeded unionfs.ko”. You also have to rename the file to unionfs.o.

• Update /etc/X11/Xsession.d/45xession per UnderScore’s help to populate the /home/knoppix folder with everything.

• Turn on DMA to speed up the remastering process. Use “dma” option at boot or the following command line below. Use hdparm to observe the before and after performance results!
  

  hdparm –tT /dev/hdX
  echo “using_dma:1” >/proc/ide/hdX/settings
  hdparm -tT /dev/hdX

• I encountered problems with cloop rebuild. I received “-1 Invalid module option” error message with insmod command in linuxrc. I guess it didn’t like the “file=” option. Running modinfo revealed the vermagic data matched between my rebuilt cloop.ko and the original cloop.o. Note that I ran “strip --strip-unneeded cloop.ko; mv cloop.ko cloop.o”. I finally copied the old cloop.o, which worked since this was the same version of the kernel. Note that the latest source for cloop is NOT available AND if you use “apt-get install cloop-source” you will get a cloop.tar.bz in /usr/src that unzips into the wrong directory structure! Go get the source tar.gz and unzip in /usr/src/modules/. Also rename the cloop-2.0 to cloop. Then run make from /usr/src. Also note that some unpatched versions of the source have problems with 2.6 kernel with regard to the buf structure. You need to change the offending two lines to arg.buf. Google for help on this one. I have seen complaints for release of the 2.02 version of the source. The 2.01 version is about one year old. Hopefully ffeirtag’s request will be heard. I think the new source would have allowed me to use the new cloop.ko that I built. Part of me still wants to know why my cloop didn’t work. Oh well, enough about cloop!

• Use the “file” command to determine file types.

• KNOPPIX boot sequence revealed (need to elaborate here):
  1. BIOS boot to CD-ROM device via boot.cat and boot image
  2. Loading of isolinux.bin and isolinux.cfg to present the user boot options
  3. Decompression of linux (bzImage) kernel and minirt.gz via initrd
  4. Execution of linuxrc in minimal root partition
  5. Linuxrc uses insmod to load cloop to access KNOPPIX image
  6. Switch to KNOPPIX image and init process runs /etc/init.d/knoppix-autoconfig, among other things.
  7. Merging of clooped CD-ROM KNOPPIX image and ramdisk via unionfs module
  8. Execution of /etc/init.d/xsession
  9. Execution of /etc/X11/Xsession.d/45xsession, which constructs the /home/knoppix folder from /etc/skel and starts kde.

• Use qemu to check the validity of an ISO. For example: “qemu –m 128 –cdrom knoppix.iso –boot d –user-net”. I run from init level 3 to create compressed KNOPPIX file and create final ISO. Use <ALT><Fx> or <ALT> left-or-right arrow keys to locate the session where you launched the command. Default in init 3 is <ALT><F1>. Use control-C to terminate a stuck session, say if you have a german keyboard map and cannot recover.

• I still have problems with German keyboard mapping in remaster. At least in qemu, haven't tried the real CD yet. I have created a new us.kbd with keytab-lilo and set KBDMAP us.kbd in isolinux.cfg. I also have the lang=us and keyboard=us kernel options for the APPEND line. What gives?

• The linuxrc in miniroot is OUTDATED! Why are the 2.6 modules still referenced with “.o” extensions instead of “.ko”? Why are all modules (except cloop and unionfs) grouped under /modules/scsi? Some of these modules are not scsi related. Perhaps move everything into /etc/modules? Cleanup unneeded modules. I have created a Python script to copy over and strip the new modules from /usr/src/kernel-source/ and/or /lib/modules/2.6.x/. I need to expand this script to cover the entire kernel/module rebuild process for Knoppix.

• BASH scripting is painful! Period! I prefer Python for serious scripting.

• Never give up on remastering KNOPPIX! Patience is the key. Compare against the working CD, reread the forums, search on Google, and then attack from a different angle. Take a break and grab a cold one. Sleep also helps.

I just wanted to share this info for others who are lost down a remastering rabbit hole contemplating their own sanity. Thanks to all who have posted on the forum. The HOWTOs were very helpful!

Happy Remastering!

[cluzterix]

thanks, I'll save your thread anyway.