Remaster from HD install:a metohd that works!

[nelsongs]

Hi Guys,
Greetings from Brazil!
I'd like to submit you a script I developed to automate the procedure of remastering the HD installed and customized knoppix sucessfully. I published the first version of this script in the Kurumin Forum ( a brazilian distro based on knoppix) for about one month ago and now I've made some litlle revisions in it and translatede it to english in order to make it availabel for the international community. You can download it at http://www.ngsys.eti.br/remasters/knphdremaster_en-1.0.sh. It's necessary to set its permissions like this: chmod +x knphdremaster_en-1.0.sh and chown root.root knphdremaster_en-1.0.sh. You have to run it as a normal user, not root. The script will ask you the root password in order to execute correctly. It can be put on the menu. I usually leave this script on /usr/local/sbin and call it, from a menu item, like this: konsole -e /usr/local/sbin/knphdremaster_en-1.0.sh. The script does most of its work automatically. It just ask you to insert the knoppix CD at the first time you use it. From the second time on everything is there.
Please, fell free to improve the script or adapt it at your own discretion. I am not a skilfull shell script programmer, actually has never has never programmed in shell script before, for this maybe you will realize my code is very simplistic, but it works well! Another observation, it was made to the 3.7 version of knoppix, using KDE. It was not tested against others versions or with other window managers.
Suggestions and contributions are very wellcomed.
Cheers,
nelsongs

[knopx4me]

nelsongs

Thanks alot. I will try it and let you know the results. Do you think it works Knoppix 3.8?

[nelsongs]

knopx4me,
Nice talking to you,
I really don´t know if it would work with knoppix 3.8, cause I still don´t have it. I´ll try to download it tonight and let you know if it fits ok. For the version 3.7 it works very fine, with all your customizations preserved.
Please, let me know if everything was ok or not.
Cheers!

[nelsongs]

knopx4me,
I just have made my first remater using the new knoppix 3.8.1 and it woked perfecly. So the script also fits the 3.8.1 version of knoppix. If you test it, please let us know the results and any suggestion to improve it.
Cheers.

[knopx4me]

nelsongs

Thanks for the good news. I just finished the job in my hand and will give this script a try today and let you know. I tried it with Kanotix 2005 and gave me some problems. I hope by tomorrow I can give you full details.

[knopx4me]

nelsongs

It is a great job. Yes it works with Knoppix 3.8. I was able to remaster it with your scripts. The remastring went smooth and fast. However, I got the following:

1- Mozilla Firefox could not start the extensions. I got the following message:

knoppix@Knoppix:/ramdisk/home/knoppix$ firefox -UILocale ar-JO -contentLocale JO
INTERNAL ERROR on Browser End: JavaPluginFactory5 init - no agent?

System error?:: No such file or directory

When I remastered it manually, it did work. I checked the /home/knoppix/.mozilla folder and found the extensions there but Fire fox could not start them. I also cheched Firefox extension list under tools > extensions and found the extensions.

2- I noticed that it shows that the partition in which I remastered it in as mounted. May be it copied every thing from /mnt. I will try some thing to avoid it and come back to you but I wanted to let you know that you are genius.

Thanks.

[flay.s]

Passei só para dar um alô (y) e dizer que fiz um remaster ontem e ficou P E R F E I T O :lol: .

Um Abraço pessoal

Bye

[sakiZ]

I downloaded and ran the script.

I told it to use hda6, a windows partition. Is it normal to get a lot of messages saying permissions could not be changed? This proceedure didn't go smoothly. Konsole crashed while running the script. So I ran it again. It seemed to go better that time.

Anyway, it looks like the script created two directories:

master, where the boot files are

and source directory where the knoppix install files were copied to.

To create the CD do I now just need to compress the source directory into a file to merge it into the master directory and then create the iso? (I have the Knoppix Hacks books I've been studying on this.)

I guess, having never done this before, I am wondering if I have done things right so far and exactly to what degree this script automates the whole process and did the Konsole crashes screw things up.

sakiZ

[nelsongs]

sakiZ,
You can´t use windows partition to do the remastering process! If you read carefully all the screens that appears in the beginning of the script you´ll notice that one of them states that. And that´s because windows partitions don´t allow symbolic links, that´s why you have had several error messages. Please, try it again with an ext2, ext3 or, preferably, reiserfs partition in order to do it correctly.
Cheers!

[sakiZ]

Wearing a sheepish grin. Feeling like a real noob.

OK. Thanks. I'll retry this when I get home.

sakiZ

[nelsongs]

sakiZ,
Since you´re going to try it again I would like you to try the new version I produced to fix some minor problems the previous one had and to better address the minor differences between the 3.7 and the 3.8.1 versions of knoppix. This new script can be downloaded at http://www.ngsys.eti.br/remasters/knphdremaster_en-1.2.sh. Please feel free to contact me at any time reporting bugs or suggestions for improvement.

knopix4me, thank you very much for your words. I really appreciated you´ve tested it and had some kind of sucess. That problem with the firefox browser and its extensions I´ve never faced before because I didn´t use them. Now I´m using them for trying to figure out what happened. I have fixed some minor bugs in the previous version as I stated here above and if you consider to run a new testing, please download the new version.
One thing that I noticed after remastering with this new version is that firefox shows itself as if it is in a "read only" state, not allowing you to do anything with it while working on the CD. On the other hand when you happen to install it on HD (now via an icon created by the script) it´s there fully, with all the extensions running smoothly. I´m still investigating this and any help from you or from whoever wants to help is welcome.
Hope to hear from you soon.
Best regards.

PS: I´ve changed something in terms of the place where to keep the "important" files and directories that are preserved, so it´s advisable to start a new remastering from a new installed system instead of an already remastered one, at list for the first time.

[knopx4me]

nelsongs

Thanks alot for making the time to update the scripts. I will run it again and report back.

[sakiZ]

Ok,

I switched the process to a Linux formatted drive on /mnt/hdb2.

I ran both the 1.0 and 1.2 scripts. Did not get the complete iso file yet.

In both scripts, at a certain point when it is generating the compressed file I get an error message saying "No Such File or Directory".

In the 1.2 version the line number is 480. I think it is referring to the /master/KNOPPIX/KNOPPIX file.

After the error message there is a blizzard of messages that scroll by.

The 1.0 version generated an ISO but it was only 5.5 megs. :-(

sakiZ

[nelsongs]

sakiZ,
It seems like you don't have enough RAM+SWAP combination or the create_compressed_fs command is missing in your /usr/bin directory. Please, check it. Regarding the RAM+SWAP combination, maybe this is one thing I forgot to mention previously. The script works well provided you have at least 1GB of combined RAM+SWAP. That means if you have, say, 256MB of RAM you'll need approx. 750MB of space on the SWAP partition (or a SWAP file). And, as far as I know, this is the minimum requirement for remastering from the CD with a chrooted environment too. I've read of some people stating that they suceeded having less than this, but all the guys here in Brazil who reported this same kind of problem you're facing were lacking this 1GB combined RAM+SWAP. As soon as they corrected this (usually increasing their SWAP partition or desabling it and creating a SWAP file instead) they got sucess in remastering. Please, try to check this and give another chance to the script, I granted it works and very fine. Thanks for replying.
Cheers!

[nelsongs]

sakiZ,
After I sent the previous post I realized that in the scenario you described problably all the procedures the script does before trying to compress the system to the KNOPPIX file using the create_compressed_fs command were preserved and went well, so, you can try this small script that follows in order to try creating the compressed KNOPPIX file and the final iso image that you can burn using k3b, for instance. Please, copy it on a text editor and save wherever you have enough rights to with the name you want. I suggest mknewiso.sh. Then you make it executable with chmod +x mknewiso.sh and set its owner to root with chown root.root mknewiso.sh. Then run it as su or using sudo this way: sudo ./mknewiso /mnt/hdb2/source /mnt/hdb2/master /mnt/hdb2/sakiZ.iso sakiZ.
If it's ok it will produce the iso KNOPPIX file at /mnt/hdb2/KNOPPIX/KNOPPIX and the iso image at /mnt/hdb2/sakiZ.iso, which you can then burn to a CD. If it crashes or generate again a very small iso file then it's confirmed you're running in shortage of combined RAM+SWAP.
Here's the script:
#!/bin/bash
#mknewiso src_dir dest_dir iso_name vol_id
#Creates the compressed KNOPPIX file and then the iso image

if ls "$1" >/dev/null 2>&1 ; then
mkisofs -R -hide-rr-moved -cache-inodes -pad "$1" | nice -5 /usr/bin/create_compressed_fs - 65536 > "$2"/KNOPPIX/KNOPPIX
else
echo "Use: ./mknewiso src_dir dest_dir iso_name vol_id"
echo '"'
echo "Creates the compressed KNOPPIX file then the iso image"
fi

volid=$4
if volid=""
then volid="Knoppix"
fi

if ls "$2" >/dev/null 2>&1 ; then
if ls "$2"/KNOPPIX/boot.cat >/dev/null 2>&1 ; then
#Problably old knoppix versions
cd "$2"
mkisofs -pad -r -l -J -v -V "$4" -b KNOPPIX/boot.img -c KNOPPIX/boot.cat -hide-rr-moved -o "$3" "$2"
elif ls "$2"/boot/isolinux/boot.cat >/dev/null 2>&1 ; then
#New Knoppix versions
cd "$2"
mkisofs -r -l -J -v -no-emul-boot -pad -boot-load-size 4 -boot-info-table -hide-rr-moved -V "$4" -b boot/isolinux/isolinux.bin -c boot/isolinux/boot.cat -o "$3" "$2"
else
echo "Unknown folder structure"
fi
else
echo "Use: ./mknewiso src_dir dest_dir iso_name vol_id"
echo '"'
echo "Creates a remastered Knoppix ISO image"
fi

[sakiZ]

Configuration:

/dev/hdb2: My Linux formatted drive is 3.1. It's empty when I start this remaster. It's my alternate drive/partition where I normally keep a mirror of my install. But for this project, I am temporarily using it for the remaster.

Memory: 768 megs.

Swapfile: Currently 350 megs. It was plenty before I started this. :-)

So, I plan to create a swap partition of about 1.5 megs.


Then, I'll retry this.

Thanks for all your input and help.

sakiZ

[sakiZ]

I successfully created a swap partition of 1.7 gigs.

(I like the idea I can activate the swap partition when I need it with swapon /dev/hda10.)

I still got a 5.5 meg iso file using the 1.0 script. No iso file gets generated with the 1.2 script.

I think the problem is my 3.1 gig Linux area on the alternate drive to do this is also too small.

This means, sigh, I will need to buy an bigger HD to actually do this remaster.

Well, I needed to create a bigger swap area anyway for my attempts to run Photoshop under wine.

sakiZ

[nelsongs]

sakiZ,
It's a pitty you still couldn't finish one complete remastering. Anyway, I found out one small bug (two words that remained in portuguese) on version 1.2, although they don't mess up anything it's good to have it corrected., so you can download it again from the same place.
Cheers.

[sakiZ]

If I can find an even slightly larger alternate 2nd hard drive, I pick one up.

Then I'll start in again.

sakiZ

[nelsongs]

Knopx4me,
Since the first time you tried the script you did it on the kanotix and it failed I decided to download kanotix-2005-02 and see the differences between it and knoppix. I found out they are almost the same, regarding structure of couse. They differ mainly on the use od Grub and squashfs by kanotix. So I adapted the same script, in its last incarnation (1.2), to remaster both distros. It's available at:http://www.ngsys.eti.br/remasters/kanoppixhdremaster_en-1.2.sh. I still didn't solve that problem with firefox not loading the extensions when you run it from the CD, but I realized it behaves normally with all the extensions loaded and running when you install the remastered copy on HD.
Cheers!

[nelsongs]

Hello to ALL,
There's new version of the script, which was renamed to knxhdremaster_en-1.2.sh. It's available for download at http://ww.ngsys.eti.br/remasters/knxhdremaster_en-1.2.sh.
This script allows you to install the remastered copy to hd via an icon on the Desktop called hd install.
You would prefer clicking on this icon to install the system to hd instead of the traditional method of opening a terminal and calling the knoppix-installer, since the script associated with the icon will fisrtly delete the S00knoppix-autoconfig link from /etc/rcS.d (which was causing double checking and, consequently, taking longer to complete), the 45xsession script from /etc/X11/Xsession.d, which is useless after hd installation and the hd install icon itself. After this the script calls the knoppix-installer script.
All the previous recomendations (which I reproduce here) still stand:
1 - download the script and put it wherever you want (I put it on the /usr/local/sbin folder).
2 - Set its permissions and ownership to: chmod +x /where_you_put_it/knxhdremaster_en-1.2.sh and chown root.root /where_you_put_it/knxhdremaster_en-1.2.sh.
3 - You can run it via, say, konsole, as a regular user (not root, nor su, neither sudo). The script problably will ask you the root password. Enter it for proceeding. Alternately, you would set it on the menu, via kmenuedit, and call it with: konsole -e /where_you_put_it/knxhdremaster_en-1.2.sh.
4 - Answer the few questions the script asks you and go for it! After some minutes you'll have a perfect copy of your custom box already burned to a cd..
Remarks 1 - At the very first time you use it the script will ask you to insert the original knoppix cd.
Remarks 2 - Although this script also remasters the kanotix 2005-02 distro, this remastering comes with a bug which prevents you from logging into the system, after having it installed on HD. The remaster process itself is smooth, the iso burned to a cd runs perfectly, but this bug appears after you install the remastered copy to hd and try to login. I'm still investigating this behavior and if someone would like trying to find out what is causing this I would appreciate a lot. So, use it with knoppix 3.7 and 3.8 and you won't have any trouble with it.
Cheers.

[nelsongs]

Sorry Guys, but there's another bug fix revision in the script and it can be downloaded at http://www.ngsys.eti.br/remasters/knxhdremaster_en-1.3.sh. The bugs problably won't affect you if kde is being used and you're using the openoffice that comes with knoppix. If you're trying to use gnome or the openoffice from debian repository, then maybe you'll have some troubles.
Regarding gnome I'm working in it and as soon as I have it up and running I will update the script again and let you all know.
Thanks and best regards.
nelsongs

[markc]

Thank you nelsongs for this wonderful script... it is the first one I've tried that actually "just works" with Knoppix 3.8.1. I'm still in the process of remastering my particular liveCD so I hope to provide more feedback about the script itself when I get a chance to look at it directly. After having spent a month looking into remastering various liveCDs (kunbuntu, kanotix, amarok, gentoo etc) I really do appreciate how much effort you have put into getting your script to work properly :-)

The only issue I've noticed is that I end up with a /home/knoppix/knoppix after the remaster but it may be that I am doing something silly.

One thing that might be useful is some reliable, perhaps associated script or an option, to clean out as much stuff on the liveCD as possible without upsetting the actual bootup procedure. It's taken me a few attempts at removing various sets of packages before I realized what NOT to remove.

[nelsongs]

markc,
Thank you for your words.
Sorry, but I couldn't help understanding exactly what you meant with "...end up with /home/knoppix/knoppix after the remaster...". Where? In your installed system or in the remastered CD?
Several people has tested this script here in Brazil and nobody has reported anything like that.
Again I couldn't understand your last paragraph. Maybe if you could be more specific it'll be a pleasure for me to help.
I would appreciate if you contact me on [email protected], so we can exchange ideas and maybe discuss the issues a little bit more deeply, without boring the readers of this forum.
Anyway, I will be producing a detailed tutorial on using this script and let you know.
Thanks again and hope hearing from you soon.
Best wishes,
nelsongs.

[nelsongs]

Hi Guys,
I´m making available another version of the script, which can be downloaded at http://www.ngsys.eti.br/remasters/knxhdremaster_en-1.4.sh.
This new version corrects the bug on the kanotix iso generation. Now it´s ok, so the script remasters Knoppix 3.7, Knoppix 3.8 and Kanotix-2005-02.
I recommend that you run this script logged as su. There´s now an extra dialog that shows the initial logged user name. If it´s not the root user, you just accept it. If it shows root, change it to the name of the normal user that logs to the system.
That´s all.
Any commnets are welcomed.
Cheers.

[kornetto]

hi nelsongs!

your script is generally the most easiest to handle so far.
unfortunatly I am getting not the result I wished.

1.) first of all, the remastered LiveCD does not contain anymore the links I have on the hd install in rc2.d
while files in /etc/init.d, /var/www and /var/sbin remained so far.
2.) furthermore the /etc/passwd is the default one, i.e. does not contain the users I created.
3.) also the /home folders created for my users are not there anymore
4.) runlevel 5 (Xwindows, KDE) does not work at all anymore.
I just receive tons of errors saying "There was an error setting up inter-process communications for KDE". Could not read network connection list. /home/knoppix/.DCOPserver_Knoppix__0. Please check that the "dcopserver" program is running!"
/home/knoppix/.kde/share/config/.kdeglobals not writeable
/home/knoppix/.kde/share/config/.kwinrc not writeable
...
...

when first booting into level 2 and then afterwards changing to runlevel 5 it works only with "startx"
using init 5 does give the errors like mentioned at 4.)

btw, in the master i never started into kde at all. i did all remastering on init 2, maybe that's some reason.



Do you have any clue, what I am doing wrong?
I was using Knoppix 3.8.2 and knxhdremaster_en-1.4.sh

thanks for your help!

/christian

ps: greetings from rainy austria

[kooladi]

Hi,
You are awsum man. Got my remastered version up and working in no time. Thanks, you have no idea how great a help you were. I just need to ask you a small question. I need to un-install KDE and then install a regional or localized version of KDE. How do I go abt doing that. I already have the localized KDE.
Thanks in advance.

[sakiZ]

Nelsongs,

I reorganized my HD to clear out some space on it.

Using your latest script, v1.4 I started the remaster process. Boy, you have made this very easy.

It went perfectly. The bad news is that the iso image is 752megs. Too big for a CD.

So, I burned the image to a DVD.

The trouble with that is my DVD drive is a USB external drive that I can't boot to.

So, I don't know for sure yet what I've got. :-)

I'm stuck for figuring out how to make it any smaller for a CD. I've already removed a LOT of programs. Anything that was redundant, plus emacs, plus some games, etc. etc.

I could remove Mozilla at some point after I install Firefox when ver. 1.5 comes out. That would save some space.

sakiZ

[knopx4me]

nelsongs

Thanks again for the script. I used the 1st version to create a Gnome live CD and it did work great.

[sakiZ]

I tried remastering again today with the 1.4 script. I'm definately getting closer.

I can boot to level init 2. But not to the GUI -- level 5.

Unfortunately I get some weird error messages.

Here are some: chown: 'knoppix.knoppix' : invalid user

And
I have no name!@knoppix: /etc/init.d/xsession start

And

X init: connection to X server lost

I did get into the GUI once by endless fiddling. I don't even know know what I did. None of my Crossover Office stuff would run because it said I was not the owner.

Any ideas anyone?
sakiZ

[/quote]