Simple remastering procdure for 7.0.2

[Capricorny]

I am writing this from a remastered 7.0.2, using the usual Poor Man's HD install, running from an external USB3 HD (1000GB WD) with 3.5 GB persistent storage, and a 3.8GB cloop image with Oracle XE, Bibble, R, Octave and some other programs added.

The remastering script is virtually unchanged from earlier versions. I haven't tried to make it very complete. It may rather easily be integrated into a larger script, for "real" remastering, i.e. creating an ISO image for DVD.

Times for 7.0.2 on Toshiba 830R (i5-2410 processor, 8GB RAM), running 32-bits kernel and copying 7.0.2 from Corsair Flash Voyager 32GB USB3, using an ext3 partition on the internal HD for creating a new compressed image. About 65 minutes.

I haven't changed the init script, as it works well for my use right now. But this can be done afterwards.


Times from the output of the script:
Set up workspace...............Sat Jun 2 18:27:50 UTC 2012
Copy live system............... Sat Jun 2 18:28:09 UTC 2012
Make isofs.............. Sat Jun 2 19:07:22 UTC 2012
Compress isofs............... Sat Jun 2 19:22:39 UTC 2012
Loopcreate persistent....... Sat Jun 2 19:31:12 UTC 2012
Purge workspace............... Sat Jun 2 19:32:56 UTC 2012

#!/bin/bash
# Based loosely on Foresters script on Knoppix-forum modified by tay 20110511-20110810,20120602


function to_exist() {
    [ -d "$1" ] || sudo mkdir -p $1 ;
}

function purge_or_create() {
    [ -d "$1" ] && sudo rm -rf $1 
    sudo mkdir -p $1 ;
}

function remaster_knoppix() {
   command=$1; shift;
   operand=$1; shift;

    case "${command} ${operand}" in
 
    "create workspace")         # Setup workspace as loop image 
        workdir=$1; shift;
        psize=$1; shift; 
       # sudo dd if=/dev/zero of=${workdir}/knoppix-remaster-data.img bs=1M count=$psize
        sudo dd if=/dev/zero of=${workdir}/knoppix-remaster-data.img bs=1 count=0 seek=${psize}
        sudo losetup /dev/loop7 ${workdir}/knoppix-remaster-data.img
        sudo mkfs.ext3 /dev/loop7 2>create_workspace.log ; 
        sudo losetup -d /dev/loop7
        purge_or_create /tmp/knx-remaster-data;
        sudo mount  ${workdir}/knoppix-remaster-data.img /tmp/knx-remaster-data -o loop=/dev/loop7 ;
        ;;


     "copy live-system")      # This is the simplified copy
        to_exist /tmp/knx-remaster-data/knx_source ;
           # Copy main /UNIONFS
        sudo rsync -ax --exclude=home --exclude=lost+found --exclude=var /UNIONFS/ /tmp/knx-remaster-data/knx_source;
           # Use a couple of directories/files from KNOPPIX as stubs
        sudo rsync -ax /KNOPPIX/home /KNOPPIX/var  /tmp/knx-remaster-data/knx_source;
        sudo rsync -ax /KNOPPIX/etc/fstab /tmp/knx-remaster-data/knx_source/etc;
        ;;


    "make isofs")  # We don't use pipe here
        purge_or_create /tmp/knx-remaster-data/knx_tmpiso ;
        sudo chmod a+rwx /tmp/knx-remaster-data/knx_tmpiso;
        sudo mkisofs -R -U -V "KNOPPIX.net filesystem" -publisher "KNOPPIX www.knoppix.net" -hide-rr-moved -cache-inodes -pad /tmp/knx-remaster-data/knx_source > /tmp/knx-remaster-data/knx_tmpiso/knoppix.iso 2>make_isofs.log ;
        ;;

    "compress isofs") # Not optimized cloop compression, also copies "surrounding" system files into target directory
        newknoppix_dir=$1; shift;
        to_exist ${newknoppix_dir}
        sudo create_compressed_fs -B 65536 /tmp/knx-remaster-data/knx_tmpiso/knoppix.iso $newknoppix_dir/KNOPPIX 2>compress_isofs.log;
        sudo rsync -ax --exclude=KNOPPIX --exclude=knoppix-data.img /mnt-system/KNOPPIX/ $newknoppix_dir ;
 
        ;;
    
    "make squashfs")
        newknoppix_dir=$1; shift;
        to_exist ${newknoppix_dir}
        sudo mksquashfs /tmp/knx-remaster-data/knx_source  $newknoppix_dir/KNOPPIX.sq -b 262144 -noappend ;
        ;;   

    "loopcreate persistent") # Create new persistent image, size in MB must be given.  
        newknoppix_dir=$1; shift;
        to_exist ${newknoppix_dir}
        psize=$1; shift;  
        sudo dd if=/dev/zero of=${newknoppix_dir}/knoppix-data.img bs=1 count=0 seek=${psize}
        sudo losetup /dev/loop6  ${newknoppix_dir}/knoppix-data.img
        sudo mkfs.ext3 /dev/loop6 2>loopcreate_persistent.log
        purge_or_create /tmp/knx-data
        sudo mount -t ext3 -rw -o loop /dev/loop6 /tmp/knx-data  ;
        sudo rsync -ax /UNIONFS/home /UNIONFS/var /tmp/knx-data ;
        sudo umount /tmp/knx-data;
        # sudo losetup -d /dev/loop6
        ;;

   "purge workspace")
        sudo umount /tmp/knx-remaster-data
        # sudo losetup -d /dev/loop7
        workdir=$1; shift;
        sudo rm -f ${workdir}/knoppix-remaster-data.img 
        ;;

    *)
        echo oops;
        ;;

    esac


}

# Calling examples:

# ./rem_02.sh /store/local 20G /store/local/KNOPPIX671 3.5G
# ./rem_02.sh /store/local 20G /store/local/KNOPPIX702 3.5G

  wrkspc_dir=$1 ;   wrkspc_sz=$2 ; remaster_dir=$3 ; persist_sz=$4 ; 
echo -e 'Set up workspace...............\c' ; date ; 
  remaster_knoppix create workspace ${wrkspc_dir} ${wrkspc_sz}
echo -e 'Copy live system...............\c' ; date ; 
  remaster_knoppix copy live-system
echo -e 'Make isofs...............\c' ; date ; 
  remaster_knoppix make isofs
echo -e 'Compress isofs...............\c' ; date ; 
  remaster_knoppix compress isofs ${remaster_dir}
 # remaster_knoppix make squashfs ${remaster_dir}
echo -e 'Loopcreate persistent...............\c' ; date ; 
  remaster_knoppix loopcreate persistent  ${remaster_dir} ${persist_sz} 
echo -e 'Purge workspace...............\c' ; date ; 
  remaster_knoppix purge workspace ${wrkspc_dir}

A short overview of the whole process

0. Install Knoppix

0.1 Download and burn DVD
0.2 Run from DVD, use flash-knoppix to write to USBdrive (usb3 if possible)
0.3 Boot from USB, set up persistent storage.
0.4 Do some customizations
0.5 Install some packages:
* flashplayer nonfree plugin
* R, basic package selection
* Some octave packages
* bison and flex
* bibble (last vs to get, it has been acquired by Corel)
* oracle xe server&client
* squashfs-tools
* vmware workstation 32-bits

1. Look at packages and do some purging

1.1 First, original package list

  dpkg-query -W --showformat='${Section} ${Package} ${Installed-Size}\n' | sort -n > knx702_DVDpack_sect2.txt

1.2 Purging done here:
Games, ca 975 MB freed
bacula, 6M, lyx, 50M, abiword etc 29M, gnucash 30M, kmail 30M, icedove 40M, amanda 8M, tuxpaint 75M, scribus 62M, gnome-games 11M, kdewallpapers 95M, kde localiz 166M, libreoffice-help-de 25M, mingw 135M, texmacs 34M

Commands used:

#Purging games 
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^games | cut -d' ' -f2`

  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep bacula | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^editors | grep lyx | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^editors | grep abiword | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^gnome | grep gnucash | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^mail | grep kmail | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^mail | grep icedove | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^utils | grep amanda | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^graphics | grep tuxpaint | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep scribus | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^gnome | grep gnome-games | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^kde | grep kdewallpapers | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^kde | grep kde-l10n | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^doc | grep libreoffice-help-de | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^devel | grep mingw | cut -d' ' -f2`
  sudo aptitude purge `dpkg-query -W --showformat='${Section} ${Package}\n' | grep ^editors | grep texmacs | cut -d' ' -f2`

Now, the extra packages will fit comfortably into 4GB cloop - using squashfs we would get even more space.

2. Run first remastering
2.1 Run the script

   ./rem_02.sh /store/local 20G /store/local/KNOPPIX702 3500M

2.2 Copy over the remaining Knoppix files

sudo rsync -ax --exclude=KNOPPIX --exclude=knoppix-data.img /mnt-system/KNOPPIX/* KNOPPIX702

2.3 The /boot directory and ldlinux.sys I copy manually over to the new structure for now.

Personally, I don't feel much need for developing this further right now, but there may be some bugs involved - although I have been running all day long from such remasterings since last spring.

I will see if this release can be used for creating a 64-bits Knoppix 7, so that I can run 64-bits versions of the most important applications. With 16GB RAM, (even with 8 ) I can run several virtual machines simultaneously on a 64-bits system. It worked with 6.4.4, but a bit rough.