-
Senior Member
registered user
Re-mastering Proof-of-Principle Effort
.
Re-mastering will require many gigabytes to accomplish.
Interestingly, ntsf may offer some unique capabilites for re-mastering.
I chose to use an external ntsf-formatted hard drive for this purpose.
Mine is a one-terrabyte-USB-2, which matches the USB capability of my laptop.
This allows me not to hazard any interference with the Windows 7
capability of the laptop, even though its gigabytes were tempting.
I've added ntfs-3g via apt to my Knoppix 6.7 LiveUSB.
I mount my external ntfs drive 'by hand' with the command
sudo mount -t ntfs-3g /dev/sdc1 /media/sdc1 when it's on sdc1.
/etc/fstab should be modified to do this automatically, but
I've not discovered how to do that just yet.
Knoppix forum contributors Forester and Capricorny have provided us with
a programming framework on which further development may proceed.
Capricorny has provided us with a working program script which has the
following functional sequence of operations:
Preparing the workspace
Copying the current Live system
Combining the Compressed and un-compressed Elements
and Preparing an isofs representation of their re-combination
Compressing the re-combination
Re-establishing an un-compressed persistence file
Purging or cleaning-up the workspace
I would like to add the following functions as well:
Re-establish the group of programs which define
both LiveCD and LiveUSB end products and
Provide both production and and testing menu choices as
part of these end products
Current status Aug 18, 2011:
The current executable script is only about 4700 bytes.
I envision the final script may be less than 10,000 bytes.
If so, it is quite likely this may be made to fit on a CD version of Knoppix,
said version having somewhat the same generous selection of other programs
to which we have become accustomed.
I think this forum may, as a group, refine this initial script successively
until it constitutes, at a minimum, a proof of principle adequate for Knoppix's
author to endorse and consider for incorporating into a later version of
Knoppix. Likely, it may also provide useful programming that Klaus may
co-opt, as is, into his final product.
I have critiqued the original Forester/Capricorny script to have some rough edges,
in my estimation, along the following lines:
1. I initially was concerned that half the time in re-mastering
following the initial script was spent in preparing the workspace. Luckily,
kl522 proposed using a sparse-file approach to minimize this effort.
There may yet be pitfalls here, but the time saving is incredible.
2. The rsynch and subsequent print-out is excessive and should be trimmed
down to whatever might serve some useful purposes. I was able to quiet-down
the rsync effort but not the compress_isofs phase. The latter needs work.
3. The current choices of what and how to keep are way to restrictive and
too unclear for my uses. I suggest a redo that is, first of all, more
transparent. Secondly, let's make sure we can allocate what's-in and what's-
not-in the compressed file.
4. I suggest we leave it to Knoppix-itself to replace the persistence file,
and not do its work for it. Unless of course, the sparse-file approach changes
the name of the game.
5. In my case /dev/loop7 isn't found and deleted as it should be in the
purge process.
6. I suspect that the remastered product may often be larger than where
it started even if programs are removed. This is not a tolerable end solution.
Its ok to get things sorted out, but that's got to come under control at some
point.
7. One has to make sure the KNOPPIX670 file from a previous
remaster is indeed 'gone', not just hidden as PCManFM does when
you think it's deleting files.
We may need to look for hidden files and delete them.
8. Construct an architecture which efficiently provides both production
and testing of both LiveCD and LiveUSB products as menu choices for the
Live products.
9. Address a few collateral issues like the fstab problem & whether there are
some hidden minirt.gz compatibility difficulties.
10. An overall wire-brush for bloat; e.g. are all those sudos necessary.
Here's my current rem_10.sh version of the Forester/Capricorny/kl522 script:
Code:
#!/bin/bash
# Based loosely on Foresters script on Knoppix-forum modified by tay 20110511-20110810
# with touches by kl522 & utu.
function to_exist() {
[ -d "$1" ] || sudo mkdir -p $1 ;
}
function purge_or_create() {
[ -d "$1" ] && sudo rm -rf $1
sudo mkdir -p $1 ;
}
function remaster_knoppix() {
command=$1; shift;
operand=$1; shift;
case "${command} ${operand}" in
"create workspace") # Setup workspace as loop image
#sudo mount -t ntfs /dev/sdd1 /media/sdd1 (mounted prior to using this script)
workdir=$1; shift;
psize=$1; shift;
# sudo dd if=/dev/zero of=${workdir}/knoppix-remaster-data.img bs=1M count=$psize # replace with sparse-file approach
dd if=/dev/zero of=${workdir}/knoppix-remaster-data.img bs=1 count=0 seek=${wrkspc_sz}
sudo losetup /dev/loop7 ${workdir}/knoppix-remaster-data.img
sudo mkfs.ext3 /dev/loop7
sudo losetup -d /dev/loop7
purge_or_create /tmp/knx-remaster-data;
sudo mount ${workdir}/knoppix-remaster-data.img /tmp/knx-remaster-data -o loop=/dev/loop7 ;
;;
"copy live-system") # This is the simplified copy
to_exist /tmp/knx-remaster-data/knx_source ;
# Copy main /UNIONFS
sudo rsync -ax --exclude=home --exclude=lost+found --exclude=var /UNIONFS/ /tmp/knx-remaster-data/knx_source;
# Use a couple of directories/files from KNOPPIX as stubs
sudo rsync -ax /KNOPPIX/home /KNOPPIX/var /tmp/knx-remaster-data/knx_source;
sudo rsync -ax /KNOPPIX/etc/fstab /tmp/knx-remaster-data/knx_source/etc;
;;
"make isofs") # We don't use pipe here
purge_or_create /tmp/knx-remaster-data/knx_tmpiso ;
sudo chmod a+rwx /tmp/knx-remaster-data/knx_tmpiso;
sudo mkisofs -R -U -V "KNOPPIX.net filesystem" -publisher "KNOPPIX www.knoppix.net" -quiet -hide-rr-moved \
-cache-inodes -pad /tmp/knx-remaster-data/knx_source > /tmp/knx-remaster-data/knx_tmpiso/knoppix.iso ;
;;
"compress isofs") # Not optimized cloop compression
newknoppix_dir=$1; shift;
to_exist ${newknoppix_dir}
sudo create_compressed_fs -B 65536 /tmp/knx-remaster-data/knx_tmpiso/knoppix.iso $newknoppix_dir/KNOPPIX;
;;
"make squashfs")
newknoppix_dir=$1; shift;
to_exist ${newknoppix_dir}
sudo mksquashfs /tmp/knx-remaster-data/knx_source $newknoppix_dir/KNOPPIX.sq -b 262144 -noappend ;
;;
"loopcreate persistent") # Create new persistent image, size in MB must be given.
newknoppix_dir=$1; shift;
to_exist ${newknoppix_dir}
psize=$1; shift;
sudo dd if=/dev/zero of=${newknoppix_dir}/knoppix-data.img bs=1M count=$psize
sudo losetup /dev/loop6 ${newknoppix_dir}/knoppix-data.img
sudo mkfs.ext3 /dev/loop6
purge_or_create /tmp/knx-data
sudo mount -t ext3 -rw -o loop /dev/loop6 /tmp/knx-data ;
sudo rsync -ax /UNIONFS/home /UNIONFS/var /tmp/knx-data ;
sudo umount /tmp/knx-data;
sudo losetup -d /dev/loop6
;;
"purge workspace")
sudo umount /tmp/knx-remaster-data
sudo losetup -d /dev/loop7
workdir=$1; shift;
sudo rm -f ${workdir}/knoppix-remaster-data.img
;;
*)
echo oops;
;;
esac
}
echo '...Starting the re-mastering process...'
# Calling examples:
#./rem_03.sh /media/sdc1 15000 /media/sdc1/KNOPPIX670&
#./rem_04.sh /media/sdc1 15000 /media/sdc1/KNOPPIX670&
#./rem_05.sh /media/sdc1 15G /media/sdc1/KNOPPIX670&
#./rem_10.sh /media/sdc1 15G /media/sdc1/KNOPPIX670&
wrkspc_dir=$1 ; wrkspc_sz=$2 ; remaster_dir=$3 ; persist_sz=$4 ;
echo -e 'Set-up workspace ******************************** \c'; date
remaster_knoppix create workspace ${wrkspc_dir} ${wrkspc_sz}
echo -e 'Copy live-system ******************************** \c'; date
remaster_knoppix copy live-system
echo -e 'Make isofs ************************************** \c'; date
remaster_knoppix make isofs
echo -e 'Compress isofs ********************************** \c'; date
remaster_knoppix compress isofs ${remaster_dir}
# echo -e 'Make squashfs ********************************** \c'; date
# remaster_knoppix make squashfs ${remaster_dir}
# echo -e 'Create persistent image ************************ \c'; date
# remaster_knoppix loopcreate persistent ${remaster_dir} ${persist_sz}
echo -e 'Purge workspace ********************************* \c'; date
remaster_knoppix purge workspace ${wrkspc_dir}
echo -e 'All done **************************************** \c'; date
exit 0
Sample to follow output on next post.
.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
![Dell SNPTP9W1C/16G 16 GB 2666MHz (PC4-2666) Memory Ram picture](/store/img/g/JKYAAOSwpONmaNJP/s-l225/Dell-SNPTP9W1C-16G-16-GB-2666MHz-PC4-2666-Memory-R.jpg)
Dell SNPTP9W1C/16G 16 GB 2666MHz (PC4-2666) Memory Ram
$34.95
![Micron 8GB DDR4 1Rx16 PC4-3200AA Laptop RAM Memory MTA4ATF1G64HZ-3G2E2 picture](/store/img/g/PxMAAOSw~T1mYfdq/s-l225/Micron-8GB-DDR4-1Rx16-PC4-3200AA-Laptop-RAM-Memory.jpg)
Micron 8GB DDR4 1Rx16 PC4-3200AA Laptop RAM Memory MTA4ATF1G64HZ-3G2E2
$10.00
![Adata XPG Gammix D10 16GB (8GBx2) DDR4 3200MHz RAM (AX4U320038G16A-BB10) picture](/store/img/g/2NIAAOSwwiBmeapc/s-l225/Adata-XPG-Gammix-D10-16GB-8GBx2-DDR4-3200MHz-RAM-A.jpg)
Adata XPG Gammix D10 16GB (8GBx2) DDR4 3200MHz RAM (AX4U320038G16A-BB10)
$23.99
![A-Tech 8GB DDR3 1600 PC3-12800 Laptop SODIMM 204-Pin Memory RAM PC3L DDR3L 1x 8G picture](/store/img/g/RCAAAOSweqdeWaui/s-l225/A-Tech-8GB-DDR3-1600-PC3-12800-Laptop-SODIMM-204-P.jpg)
A-Tech 8GB DDR3 1600 PC3-12800 Laptop SODIMM 204-Pin Memory RAM PC3L DDR3L 1x 8G
$13.99
![NEW OLOy 16GB 2x8GB KIT DDR4-3000 PC4-24000 Desktop Gaming RAM MD4U0830160BB1ST picture](/store/img/g/k~8AAOSwIBdmZRfq/s-l225/NEW-OLOy-16GB-2x8GB-KIT-DDR4-3000-PC4-24000-Deskto.jpg)
NEW OLOy 16GB 2x8GB KIT DDR4-3000 PC4-24000 Desktop Gaming RAM MD4U0830160BB1ST
$19.99
![Crucial DDR3L 16GB 1600 2x 8GB PC3-12800 Laptop SODIMM Memory RAM PC3 16G DDR3 picture](/store/img/g/mecAAOSw2Vlkv6Ot/s-l225/Crucial-DDR3L-16GB-1600-2x-8GB-PC3-12800-Laptop-SO.jpg)
Crucial DDR3L 16GB 1600 2x 8GB PC3-12800 Laptop SODIMM Memory RAM PC3 16G DDR3
$22.45
![Team T-FORCE VULCAN Z 32GB (2 x 16GB) PC RAM DDR4 3200 (PC4 25600) Memory picture](/store/img/g/~u4AAOSwK-liKn7t/s-l225/Team-T-FORCE-VULCAN-Z-32GB-2-x-16GB-PC-RAM-DDR4-32.jpg)
Team T-FORCE VULCAN Z 32GB (2 x 16GB) PC RAM DDR4 3200 (PC4 25600) Memory
$54.99
![RAMAXEL 16 GB ( 4 GB x 4 sticks ) DDR3L-1600 PC3L-12800U 12800 Desktop RAM - HVD picture](/store/img/g/6W4AAOSwlB1l~Iv1/s-l225/RAMAXEL-16-GB-4-GB-x-4-sticks-DDR3L-1600-PC3L-1280.jpg)
RAMAXEL 16 GB ( 4 GB x 4 sticks ) DDR3L-1600 PC3L-12800U 12800 Desktop RAM - HVD
$11.42
![HyperX FURY DDR4 16GB 3200 MHz PC4-25600 Desktop RAM Memory DIMM 288pin 2x 16GB picture](/store/img/g/hWAAAOSwikdkv6jx/s-l225/HyperX-FURY-DDR4-16GB-3200-MHz-PC4-25600-Desktop-R.jpg)
HyperX FURY DDR4 16GB 3200 MHz PC4-25600 Desktop RAM Memory DIMM 288pin 2x 16GB
$128.20
![(4 Piece) Micron MTA36ASF4G72PZ-2G6 DDR4-2666v 128GB (4x32GB) Workstation RAM picture](/store/img/g/D6gAAOSw825l66Eh/s-l225/-4-Piece-Micron-MTA36ASF4G72PZ-2G6-DDR4-2666v-128G.jpg)
(4 Piece) Micron MTA36ASF4G72PZ-2G6 DDR4-2666v 128GB (4x32GB) Workstation RAM
$135.99