-
Senior Member
registered user
@ Capricorny
I would like to encourage you to take Klaus up on his offer in his post #29
to present him with 'the automation' to make 'poor-man's re-mastering'
a menu item in some later Knoppix release.
One way to do this might be first to lay out a proof of principle that others
on this forum might emulate and/or suggest improvements to. You've probably
done this already, and only need to collect all your notes, and Forester's
as well if they are relevant, in one place.
It may be more likely to succeed earlier, as a Knoppix refinement, if squashfs
vs cloop differences were left to be suggested as a subsequent refinement
of the basic idea.
I know you also have an avenue to address this on the debian-knoppix mailing list.
I wish you good luck in this however you choose to proceed.
-
Senior Member
registered user
Think that is a very good suggestion
![Quote](images/misc/quote_icon.png)
Originally Posted by
utu
@ Capricorny
I would like to encourage you to take Klaus up on his offer in his post #29
to present him with 'the automation' to make 'poor-man's re-mastering'
a menu item in some later Knoppix release.
One way to do this might be first to lay out a proof of principle that others
on this forum might emulate and/or suggest improvements to. You've probably
done this already, and only need to collect all your notes, and Forester's
as well if they are relevant, in one place.
It may be more likely to succeed earlier, as a Knoppix refinement, if squashfs
vs cloop differences were left to be suggested as a subsequent refinement
of the basic idea.
I know you also have an avenue to address this on the debian-knoppix mailing list.
I wish you good luck in this however you choose to proceed.
I will try to carry out something in the direction of this very good suggestion.
While the use of cloop vs squashfs is not peripheral in principle, it surely is peripheral in this context, as ordinary users don't need to be aware of such issues at all.
Furthermore, here on this forum we can see that there is a certain demand for remastering - the reasons may not always be that good, but if it can be done easily, then why not?
One important reason to make it simple, is that it will be easier to ask people to upgrade to newer Knoppix versions - older versions are very often the reasons for trouble, and having made a customization will often be an important reason not to upgrade.
There are some compromizes to be made, I think. Just updating the compressed and persistent images should really be a separate step, for testing before a new ISO image is made, but for the utmost ease for the user, a new ISO image is the best end point. That can be very easily tested, and handled with minimum knowledge of file systems etc.
Making an ISO image means, among other things, that the package database gets written to the ISO, and that extra content in /home may get lost if the users don't take the newly generated persistent image along. For several reasons, among them privacy, I will NOT write the whole of user's /home to the ISO! ![Wink](images/smilies/icon_wink.gif)
Also, even if it is somewhat less efficient, I think making a separate loop-mounted image for the interim file handling during remastering is best. We can foresee many users running off NTFS, and for them, a new ISO file being the only thing left on the NTFS file system afterwards would be the safest and least intrusive option.
Last edited by Capricorny; 08-09-2011 at 07:15 AM.
-
Senior Member
registered user
![Quote](images/misc/quote_icon.png)
Originally Posted by
Capricorny
While the use of cloop vs squashfs is not peripheral in principle, it surely is peripheral in this context, as ordinary users don't need to be aware of such issues at all.
My suggestion is to avoid the possibility that KK may have his own preferences re squashfs vs cloop.
If so, it might be better to handle that issue later. I'm not offering any judgment as to that choice.
If that issue is truly essential, then disregard my caution, and by all means, bring it on.
Best Regards
Last edited by utu; 08-09-2011 at 04:15 PM.
-
Senior Member
registered user
![Quote](images/misc/quote_icon.png)
Originally Posted by
utu
My suggestion is to avoid the possibility that KK may have his own preferences re squashfs vs cloop.
If so, it might be better to handle that issue later. I'm not offering any judgment as to that choice.
If that issue is truly essential, then disregard my caution, and by all means, bring it on.
It's surely not truly essential today, and even with pure 64 bits remastering, it turned out I might as well use cloop. The main reason we keep suggesting squashfs, is that there are some advantages, in particular when several modifications are done, and that support for squashfs can be added truly transparently to end users: Using cloop, you won't notice anything about squashfs until you extract minirt.gz and look at the init script.
And for end user tools like the remastering script we discuss, it is, as noted, of some value to deviate as little as possible from the starting point. When people create a new ISO, it should resemble the original one as much as possible.
-
Senior Member
registered user
@ Capricorny
I'm looking forward to your 'proof of priinciple' enterprise.
I addressed a few notes to you in that regard on Werner's other (Re-Mastering) thread.
Good Luck.
Last edited by utu; 08-10-2011 at 02:09 PM.
-
Senior Member
registered user
Example of simple remastering script
OK, here's a quick and dirty, and not-that-well-debugged version of the basic poor man's remastering, which I just used for remastering 6.7.0 DVD (cloop version). I can guarantee that each single step works, but haven't had time to test the whole thing.
It takes 4 parameters:
- Directory of remastering workspace
- Size of workspace
- Directory of remastered version
- Size of new persistent store
All those may of course be skipped, and sensible defaults used.
Example: ./rem_02.sh /store/local 20000 /store/local/KNOPPIX670 4000
I'm running the remastered version under qemu right now, and it surely works. I see that I will get space problems with max 4GB compressed image, as the cloop is 3.6 GB after purging the obvious things and installing a few programs. Therefore, squashfs may be to prefer here.
Using cloop, space requirement is ca 2x uncompressed KNOPPIX image size, i.e. ca 18-20 GB for DVD version. Using squashfs, it's only 1x. Images are written directly to new KNOPPIX location, so space is needed for that too. I would not recommend writing directly to slow flash memory. Use some hard disk space if you can.
Code:
#!/bin/bash
# Based loosely on Foresters script on Knoppix-forum modified by tay 20110511-20110810
function to_exist() {
[ -d "$1" ] || sudo mkdir -p $1 ;
}
function purge_or_create() {
[ -d "$1" ] && sudo rm -rf $1
sudo mkdir -p $1 ;
}
function remaster_knoppix() {
command=$1; shift;
operand=$1; shift;
case "${command} ${operand}" in
"create workspace") # Setup workspace as loop image
workdir=$1; shift;
psize=$1; shift;
sudo dd if=/dev/zero of=${workdir}/knoppix-remaster-data.img bs=1M count=$psize
sudo losetup /dev/loop7 ${workdir}/knoppix-remaster-data.img
sudo mkfs.ext3 /dev/loop7
sudo losetup -d /dev/loop7
purge_or_create /tmp/knx-remaster-data;
sudo mount ${workdir}/knoppix-remaster-data.img /tmp/knx-remaster-data -o loop=/dev/loop7 ;
;;
"copy live-system") # This is the simplified copy
to_exist /tmp/knx-remaster-data/knx_source ;
# Copy main /UNIONFS
sudo rsync -ax --delete --exclude=home --exclude=lost+found --exclude=var /UNIONFS/ /tmp/knx-remaster-data/knx_source;
# Use a couple of directories/files from KNOPPIX as stubs
sudo rsync -ax /KNOPPIX/home /KNOPPIX/var /tmp/knx-remaster-data/knx_source;
sudo rsync -ax /KNOPPIX/etc/fstab /tmp/knx-remaster-data/knx_source/etc;
;;
"make isofs") # We don't use pipe here
purge_or_create /tmp/knx-remaster-data/knx_tmpiso ;
sudo chmod a+rwx /tmp/knx-remaster-data/knx_tmpiso;
sudo mkisofs -R -U -V "KNOPPIX.net filesystem" -publisher "KNOPPIX www.knoppix.net" -hide-rr-moved -cache-inodes -pad /tmp/knx-remaster-data/knx_source > /tmp/knx-remaster-data/knx_tmpiso/knoppix.iso ;
;;
"compress isofs") # Not optimized cloop compression
newknoppix_dir=$1; shift;
to_exist ${newknoppix_dir}
sudo create_compressed_fs -B 65536 /tmp/knx-remaster-data/knx_tmpiso/knoppix.iso $newknoppix_dir/KNOPPIX;
;;
"make squashfs")
newknoppix_dir=$1; shift;
to_exist ${newknoppix_dir}
sudo mksquashfs /tmp/knx-remaster-data/knx_source $newknoppix_dir/KNOPPIX.sq -b 262144 -noappend ;
;;
"loopcreate persistent") # Create new persistent image, size in MB must be given.
newknoppix_dir=$1; shift;
to_exist ${newknoppix_dir}
psize=$1; shift;
sudo dd if=/dev/zero of=${newknoppix_dir}/knoppix-data.img bs=1M count=$psize
sudo losetup /dev/loop6 ${newknoppix_dir}/knoppix-data.img
sudo mkfs.ext3 /dev/loop6
sudo losetup -d /dev/loop6
purge_or_create /tmp/knx-data
sudo mount ${newknoppix_dir}/knoppix-data.img /tmp/knx-data -o loop=/dev/loop6 ;
sudo rsync -ax --delete /UNIONFS/home /UNIONFS/var /tmp/knx-data ;
sudo umount /tmp/knx-data;
;;
"purge workspace")
sudo umount /tmp/knx-remaster-data
sudo losetup -d /dev/loop7
workdir=$1; shift;
sudo rm -f ${workdir}/knoppix-remaster-data.img
;;
*)
echo oops;
;;
esac
}
# example: ./rem_02.sh /store/local 20000 /store/local/KNOPPIX670 4000
wrkspc_dir=$1 ; wrkspc_sz=$2 ; remaster_dir=$3 ; persist_sz=$4 ;
remaster_knoppix create workspace ${wrkspc_dir} ${wrkspc_sz}
remaster_knoppix copy live-system
# remaster_knoppix make isofs
# remaster_knoppix compress isofs ${remaster_dir}
remaster_knoppix make squashfs ${remaster_dir}
remaster_knoppix loopcreate persistent ${remaster_dir} ${persist_sz}
remaster_knoppix purge workspace
Last edited by Capricorny; 08-10-2011 at 10:38 PM.
-
Senior Member
registered user
@ Capricorny
Excellent effort.
I think I can recall dinosoep correcting Forester's 'make isofs' chmod with chown.
Wouldn't that still apply here?
Also, seemed like Forester had some useful notes regarding the filesystems used.
Hope you don't mind the comments; trying to help.
Cheers
Last edited by utu; 08-10-2011 at 10:58 PM.
-
Senior Member
registered user
![Quote](images/misc/quote_icon.png)
Originally Posted by
utu
@ Capricorny
Excellent effort.
I think I can recall dinosoep correcting Forester's 'make isofs' chmod with chown.
Wouldn't that still apply here?
Also, seemed like Forester had some useful notes regarding the filesystems used.
Hope you don't mind the comments; trying to help.
Cheers
That temporary iso-directory may have whatever properties it will for me, as long as I can use it for making a compressed image, and I can.
File systems: You need something allowing the necessary file system sizes for remastering. But as only one (temporary) file is created on the native file system, it can be NTFS or one of the Linux file systems. You can use FAT32 for the directory (drive) to write the remastered versions to. Like Knoppix in general, this is rather file system agnostic.
IMHO, not using a poor man's install on a hard drive (internal or external) for such enterprises is rather masochistic
-
Senior Member
registered user
@ Capricorny:
Let's see if I have this right:
0. What I shall call my working LiveUSB is one with persistent store which
has accumulated additions and 'deletions' of program material. I wish to modify
its original LiveUSB compressed image to embody all these accumulated
changes into a new LiveUSB compressed image and to provide a new 'empty'
persistence file for future temporary changes.
1. For these purposes, I need to add to my currently working LiveUSB a small
remastering script; I don't need to re-do the minirt.gz of the LiveUSB for this
version of remastering.
2. My LiveUSB is on /dev/sdb1; I have some external storage on /dev/sdc1,
exceeding 25 Gb or so which I mount as /store/local. It may be ext, fat, or ntsf:
it doesn't matter as far as the remastering goes.
This model of re-mastering separates (1) the development of useful modifications
by adding and removing program material from (2) the process of revising the
compressed image and the persistence file. With suitable safeguards, it is
hoped this might serve as a menu choice item in some future version of Knoppix.
Corrections or modifications welcome.
Last edited by utu; 08-11-2011 at 01:01 AM.
-
Senior Member
registered user
![Quote](images/misc/quote_icon.png)
Originally Posted by
utu
@ Capricorny:
Let's see if I have this right:
0. What I shall call my working LiveUSB is one with persistent store which
has accumulated additions and 'deletions' of program material. I wish to modify
its original LiveUSB compressed image to embody all these accumulated
changes into a new LiveUSB compressed image and to provide a new 'empty'
persistence file for future temporary changes.
1. For these purposes, I need to add to my currently working LiveUSB a small
remastering script; I don't need to re-do the minirt.gz of the LiveUSB for this
version of remastering.
2. My LiveUSB is on /dev/sdb1; I have some external storage on /dev/sdc1,
exceeding 25 Gb or so which I mount as /store/local. It may be ext, fat, or ntsf:
it doesn't matter as far as the remastering goes.
This model of re-mastering separates (1) the development of useful modifications
by adding and removing program material from (2) the process of revising the
compressed image and the persistence file. With suitable safeguards, it is
hoped this might serve as a menu choice item in some future version of Knoppix.
Corrections or modifications welcome.
Re 2: The remastering storage must be able to accomodate files >4GB for realistic remastering work..
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
![ZipLink ZL-2CBL2-2P picture](/store/img/g/pBEAAOSwlDtirMsF/s-l225/ZipLink-ZL-2CBL2-2P.jpg)
ZipLink ZL-2CBL2-2P
$19.50
![Intel - Core i9-12900K Desktop Processor 16 (8P+8E) Cores up to 5.2 GHz Unloc... picture](/store/img/g/snwAAOSwhWZhsOrI/s-l225/Intel-Core-i9-12900K-Desktop-Processor-16-8P-8E-Co.jpg)
Intel - Core i9-12900K Desktop Processor 16 (8P+8E) Cores up to 5.2 GHz Unloc...
$347.99
![Intel - Core i7-12700K Desktop Processor 12 (8P+4E) Cores up to 5.0 GHz Unloc... picture](/store/img/g/7-YAAOSwHaFjXQ2k/s-l225/Intel-Core-i7-12700K-Desktop-Processor-12-8P-4E-Co.jpg)
Intel - Core i7-12700K Desktop Processor 12 (8P+4E) Cores up to 5.0 GHz Unloc...
$242.99
![Intel - Core i9-14900K 14th Gen 24-Core 32-Thread - 4.4GHz (6.0GHz Turbo) Soc... picture](/store/img/g/YFUAAOSwTDplMDa3/s-l225/Intel-Core-i9-14900K-14th-Gen-24-Core-32-Thread-4-.jpg)
Intel - Core i9-14900K 14th Gen 24-Core 32-Thread - 4.4GHz (6.0GHz Turbo) Soc...
$544.99
![Intel Core I7-7700K Processor (4.2 GHz, Quad-Core, LGA 1151) - SR33A picture](/store/img/g/gugAAOSwQqRmBtDc/s-l225/Intel-Core-I7-7700K-Processor-4-2-GHz-Quad-Core-LG.jpg)
Intel Core I7-7700K Processor (4.2 GHz, Quad-Core, LGA 1151) - SR33A
$84.00
![Intel 6 Core i5-8600 3.1GHZ Desktop Processor SR3X0 picture](/store/img/g/G3sAAOSwKXxk-6A7/s-l225/Intel-6-Core-i5-8600-3-1GHZ-Desktop-Processor-SR3X.jpg)
Intel 6 Core i5-8600 3.1GHZ Desktop Processor SR3X0
$50.00
![Intel - Core i7-14700K 14th Gen 20-Core 28-Thread - 4.3GHz (5.6GHz Turbo) Soc... picture](/store/img/g/0eYAAOSwcHZlMDWu/s-l225/Intel-Core-i7-14700K-14th-Gen-20-Core-28-Thread-4-.jpg)
Intel - Core i7-14700K 14th Gen 20-Core 28-Thread - 4.3GHz (5.6GHz Turbo) Soc...
$399.99
![INTEL XEON GOLD 6148 20 Core SR3B6 2.4GHZ 27.5MB Processor @24 picture](/store/img/g/T4UAAOSwjXlmBZfi/s-l225/INTEL-XEON-GOLD-6148-20-Core-SR3B6-2-4GHZ-27-5MB-P.jpg)
INTEL XEON GOLD 6148 20 Core SR3B6 2.4GHZ 27.5MB Processor @24
$99.99
![AMD Ryzen 7 2700X CPU Processor 3.7GHz AM4 picture](/store/img/g/jsoAAOSwKd5l80o2/s-l225/AMD-Ryzen-7-2700X-CPU-Processor-3-7GHz-AM4.jpg)
AMD Ryzen 7 2700X CPU Processor 3.7GHz AM4
$74.99
![Intel Core i9-13900K Raptor Lake 24-Core (8P+16E) Desktop Processor CPU picture](/store/img/g/5T8AAOSwqEZl165w/s-l225/Intel-Core-i9-13900K-Raptor-Lake-24-Core-8P-16E-De.jpg)
Intel Core i9-13900K Raptor Lake 24-Core (8P+16E) Desktop Processor CPU
$499.00