PDA

View Full Version : Revised version of Remastering HowTo



Werner P. Schulz
06-22-2011, 11:07 PM
Something in this HowTo (http://www.knoppix.net/wiki/Knoppix_Remastering_Howto) didn't work with new Knoppix. For that reason I've done an update of all the script lines you need for remastering purpose. All remainig I put in chapter "Old stuff of this HowTo".

Greetings Werner * http://www.wp-schulz.de/knoppix/summary.html
Own Rescue-CD with Knoppix (Knoppix V6.4.4 remaster)

utu
07-05-2011, 09:00 PM
@ Werner P. Schulz

I wonder if you are aware of the executable script /usr/bin/mylivecd
that is a part of PCLinuxOS 2011.06-full?

It would be so nice for us less adept than yourself if your 'How-To'
were actually an executable script like that in PCLinuxOS.
Texstar, the author of PCLinuxOS, says we're welcome to adapt his script,
but notes that, in his words:

" I don't think the mylivecd script would be of much use to anyone at
knoppix since it was specifically designed to work on a PCLinuxOS
installation using the mandriva drakxtools and ldetect packages
but you are welcome to do whatever you wish. Tex"

I thought at least the organization of Tex's script might be a
help to someone trying to develop a similar executable re-
mastering script for Knoppix. Forgive my ignorance if one
already exists. I'm not aware of it.

Werner P. Schulz
07-05-2011, 09:23 PM
... the revised version of the 'Remastering HowTo' isn't my HowTo. I only tested all the command lines for V6.4.4 and excluded all the very old and useless stuff.

I did remastering Knoppix a little different, using a harddrive installation of Knoppix in a VirtualBox. You can donwload my scripts. For more information have a look at my page.

Greetings Werner * http://www.wp-schulz.de/knoppix/summary.html
Own Rescue-CD with Knoppix (Knoppix V6.4.4 remaster)

utu
07-05-2011, 10:14 PM
@ Werner

You over-estimate my competence and courage. For example,
on your 'Summary' page, I do not understand your meanings in the
notes on yellow background with red, black & blue text and => signs.

With the PCLinuxOS script, you choose a big-enough dedicated partion,
name the iso, and mylivecd does the work from there.

I can say which programs to acquire and which to delete, but managing
a number of scripts is probably beyond my capabilities.

I thought perhaps the PCLinuxOS idea might help you or someone to
organize your separate scripts into a single script like mylivecd.

Werner P. Schulz
07-05-2011, 10:53 PM
notes on yellow background with red, black & blue text and => signs.... don't take care of it! The scripts will do all for you. You only need three scripts:
kn-prepare to establish the remastering enviroment
kn-chroot to work in chroot jail
kn-final to build the ISO

All the other scripts in my deb (kn-purge, kn-favorit, kn-what, kn-bootcd) you may use or not.

I thought perhaps the PCLinuxOS idea might help you or someone to
organize your separate scripts into a single script like mylivecd. ... you cannot do remastering with a single script! With a single script you have no change to enter and leave the chroot jail.

Greetings Werner * http://www.wp-schulz.de/knoppix/summary.html
Own Rescue-CD with Knoppix (Knoppix V6.4.4 remaster)

utu
07-06-2011, 09:15 PM
@ Werner

I've not used Virtual Box, but I have many MBs on my Win7 laptop.
Can I use a Window-based Virtual box to do what I need to do?

I'd rather not partition my HDD, if I don't need-to.

Werner P. Schulz
07-06-2011, 09:49 PM
... VirtualBox also runs on Windows host - why not on Windows 7?

But "many MBs" will not be enough :-). For remastering with my scripts you need 7 GB.

Please reload "summary.html". Today I changed this page and I hope, now all is clearly arranged.

Capricorny
07-07-2011, 11:52 AM
@ Werner
I'd rather not partition my HDD, if I don't need-to.

Strictly speaking, you never need to. You can just create a big file on an NTFS partition, set it up with a Linux file system, mount and run from that. As you know, that's exactly the same way persistent store is set up and used. You can of course also add rc.local commands to do this at boot time. Would run a bit faster than in VM, I'd guess.

Werner P. Schulz
07-07-2011, 12:29 PM
... yes you never need to partition your HD. VirtualBox creates a big and dynamic growing file and uses it as a virtual harddrive. Creating a filesystem in this virtual harddrive and installing grub as bootmanager will be done by the Knoppix CD.

utu
07-07-2011, 04:19 PM
Greetings to Werner & Capri Corny

Comiing to you in a virtual way. Got a Virtual Box from Oracle and got my 6.4.4 CD virtualized. Download from Oracle took an hour Went thru one bout with Werner's scripts. Not sure whether a success or not. Took several things out and made an iso.
iso was not std iso9660 or whatever; also resultant iso larger than what I started with.

Was having problem with no usb; solved that. Now trying to figure how to make a cd from the iso. Using a DVD+R blank rather than a 700Mb CD, but not sure what to do next. Werner's scripts take all the pain out of the details, but there are a number of Virtual Box details I need to get under control.

Tried several times to restore whitespace here, unsuccessfully.
It was a script blocker.

Werner P. Schulz
07-07-2011, 04:50 PM
.. also resultant iso larger than what I started with... yes, the compression isn't optimized but quick.

You don't need to burn a CD (or DVD) each time; select your new created ISO as boot device in VirtualBox. If the ISO runs well in the VirtualBox enviroment you can burn it in a few days and it will work.

By the way, VirtualBox has an excellent manual.

utu
07-07-2011, 07:51 PM
... yes, the compression isn't optimized but quick.


My impression is nothing in the re-mastering arena is 'quick'. Several phases took me around twenty minutes each. Like watching paint dry.

The version of VirtualBox I have does not recognize two cpus. Is there an easy fix for this?

Also, is the hour download from Oracle the only way people get VirtualBox?

I downloaded a supplemental program from Oracle that had usb support. Should there be a manual in that somewhere?

klaus2008
07-07-2011, 09:29 PM
The version of VirtualBox I have does not recognize two cpus. Is there an easy fix for this? Did you set the number of processors for your virtual machine?

Werner P. Schulz
07-07-2011, 09:47 PM
... yes, of course, 'quick' is relative. You have VirtualBox installed in Windows 7 - and Windows 7 is a very lame OS, which does not leave enough CPU power for applications like VirtualBox.

The last version of VirtualBox is 4.0.10. For USB-support you need the extension pack. To read the manual use [F1].

For each virtual machine you want to run in VirtualBox you can do many decisions. For example 'System > Processor > 2 CPUs and many other things.

utu
07-07-2011, 11:11 PM
For each virtual machine you want to run in VirtualBox you can do many decisions. For example 'System > Processor > 2 CPUs and many other things.

Specifically can't find this cpu choice now.

All my current problems are VirtualBox problems it seems.
Werner has taken care of all the re-mastering details.

Initially, I told VirtualBox the guest would be linux/Debian, that's all.

only one cpu recognized.
guest x11 resolution not right, even tho' CD had it ok.
...note: virtualbox-guest-x11 isn't installed and won't.
can't read auxiliary fat32-formatted data USB from guest apps.
don't know how to export final iso to host, and
...don't know how to get current iso into iso9660 format.
not sure whether current guest image is what will appear
...as the /remaster/iso.
Can't locate earlier snapshots, only the last one.

I am surprised how much IS working.

utu
07-08-2011, 03:44 AM
Did you set the number of processors for your virtual machine?

Hi, Klaus

For the life of me, I can't find the gui you mention.
When I did answer the question, it only took 'one' for an answer,
for a reason having to do with some missing interface programming.
Is this qui available after the initial question phase has been passed?

An update: using a cheatcode on the kernel default line to remedy 1366x768 problem. Biggest immediate problem is how to bring in text from an auxiliary usb.
Can't figure out the 'shared folder' secret.

I gather what's going on here is that what's been created is a virtual hard drive install of Debian.
Is that the big idea behind re-mastering?

kl522
07-08-2011, 03:53 AM
All my current problems are VirtualBox problems it seems.


:) I wonder this feedback is typical or not. If we think this is typical, then perhaps doing the remastering in VirtualBox is not so good an idea. Instead of completing the task of remastering, one ends himself up in learning how to use virtualbox. :)

Some time ago I proposed a remastering using a typical poor-man install. Of course, as usual, I am not a very patient guy, I never carry the idea all the way through but I ***THOUGHT*** Forester later came out with a script which make it even more friendly. He published the script somewhere. Though I haven't use it but I think remastering using this method is a better approach. I don't like the 'chroot' method because I think in a typical 'chroot', the final system behaviour is only tested during the few hours of 'chroot' - which is often inadequate to represent the final thing the user want to see and there maybe some "side-effects" which were not noticed during the few hours of usage. Whereas using 'poor-man' install, what you see is what you get. There is no urgency to remaster if you think it is not yet proven stable. You can remaster after 3 months of usage if you like.

Werner P. Schulz
07-08-2011, 08:38 AM
Specifically can't find this cpu choice now.... look at this screenshot (http://www.wp-schulz.de/grafiken/VirtualBox.PNG)

guest x11 resolution not right,You have to install GuestAdditions (http://www.wp-schulz.de/grafiken/guestadd.PNG). Please read in VirtualBox manual> 4.2.1. Guest Additions for Windows

don't know how to export final iso to host,First of all you have to copy the new ISO to the directory of isoimages (http://www.wp-schulz.de/grafiken/filemanager.PNG) from VirtualBox. Don't know how this looks like beneath Windows7 and how you can do it. Perhaps you need an USB-stick to make the copy from VirtualBox to the directory in Windows. With Linux it is all very easy to do :-)

After this you can mount the new ISO in VirtualBox (http://www.wp-schulz.de/grafiken/rem-ISO.PNG) to start from it.


Greetings Werner * http://www.wp-schulz.de/knoppix/summary.html
Own Rescue-CD with Knoppix (Knoppix V6.4.4 remaster)

Werner P. Schulz
07-08-2011, 10:12 AM
don't know how to get current iso into iso9660 format... this is only a warning, the ISO will work.
Can't figure out the 'shared folder' secret.
Please read in the manual "4.3 Shared Folders"

The idea of my way of remastering is
1. create an enviroment to work by the Debian like Installation of Knoppix (it is not a Installation of Debian!)

2. avoid problems by working in a not exactly defined enviroment (Gentoo, BSD, Windows an so on)

3. use scripts, which will do all the job for you and are well tested beneath the new Knoppix version

4. a separate enviroment dosn't disturb your productive OS

Of ourse, the other ways are using Remaster HowTo (http://www.knoppix.net/wiki/Knoppix_Remastering_Howto) or simply use flash disk Installation (http://www.knoppix.net/wiki/Install_FAQ) with persistent memory.

My description is for experienced users; users who never worked with VirtualBox have problems. VirtualBox isn't a tool you can learn within a few minutes.

Werner P. Schulz
07-08-2011, 10:32 AM
I don't like the 'chroot' method because I think in a typical 'chroot', the final system behaviour is only tested during the few hours of 'chroot' - which is often inadequate to represent the final thing the user want to see and there maybe some "side-effects" which were not noticed during the few hours of usageWith my method of remastering in a virtual drive enviroment you can create new ISOs over and over and test them with VirtualBox without the necessity to reboot always your conputer.

Or you install the remastered new ISO to an USB-Stick and make extensive tests with it in the Virutalbox. And so you have the ability to eliminate all "side-effects" or integrate new features.

Capricorny
07-08-2011, 10:50 AM
:) I wonder this feedback is typical or not. If we think this is typical, then perhaps doing the remastering in VirtualBox is not so good an idea. Instead of completing the task of remastering, one ends himself up in learning how to use virtualbox. :)

Some time ago I proposed a remastering using a typical poor-man install. Of course, as usual, I am not a very patient guy, I never carry the idea all the way through but I ***THOUGHT*** Forester later came out with a script which make it even more friendly. He published the script somewhere. Though I haven't use it but I think remastering using this method is a better approach. I don't like the 'chroot' method because I think in a typical 'chroot', the final system behaviour is only tested during the few hours of 'chroot' - which is often inadequate to represent the final thing the user want to see and there maybe some "side-effects" which were not noticed during the few hours of usage. Whereas using 'poor-man' install, what you see is what you get. There is no urgency to remaster if you think it is not yet proven stable. You can remaster after 3 months of usage if you like.

I tend to support this.
I have used Forester's script as a starting point, the method works very well for me, but I want to debug my script version a bit before I post it. As for chroot, it may of course be used, but I think it is a great advantage to avoid it, as I do now. The simplified method concentrates on the cloop/squashfs and persistent images, and leaves everything else alone. Except, of course, for changes made to scripts in the poor man's install being copied over to the new version.

Remember, it is only for CD/DVD use we need to create the final ISO image nowadays. So why bother if that is not the goal? I am running my remastered version from all sorts of media, and, when I have reached kind of final version, I may create a DVD. But I'm far from there now,

But this doesn't mean I think remastering through VMs needs to be a bad idea. In particular, when using tools like Virtualbox daily, it can be efficient. For my own part, however, I wouldn't even consider trying VMware Workstation, the tool I use, for this. And there is no need, really, to involve anything else than basic Knoppix here. Just drop the Knoppix package, with persistent image, onto an NTFS partition, run Knoppix from there, and if desirable, create another image on that partition for working with the temporary files in the remastering. Standard, simple, speedy and safe (relatively).

utu
07-08-2011, 05:22 PM
First of all you have to copy the new ISO to the directory of isoimages (http://www.wp-schulz.de/grafiken/filemanager.PNG) from VirtualBox.

You will be amused to know that I just used my LiveCD, not an iso,
and told VirtualBox where my guest OS source was to be found.
A few snapshots later I had something like my LiveUSB up and
running, albeit with a few hardware emulation errors.
I didn't know that was the wrong way to go about things.

I'll stop troubling you with all my growing pains.
Thanks for getting me going on both VirtualBox & re-mastering.

Werner P. Schulz
07-08-2011, 06:14 PM
... in case of trouble you can contact me per email (look at summary.html) :-)

klaus2008
07-10-2011, 02:05 PM
When I did answer the question, it only took 'one' for an answer,
for a reason having to do with some missing interface programming.
Hi, utu!

Does the problem still exist?

In an internet forum I read that one needs to enable virtualization technology (VT) in the BIOS on some machines with VT capable processors.

utu
07-10-2011, 03:26 PM
@ Klaus

Yes, the dual cpu problem still exists with the current state of my VirtualBox.

The dual cpu problem is not THE show stopper for me, but not having USB access is
just too inconvenient. It looks like a lot of folks with my sort of set-up have
a problem. That is, Win7, 64-bit and dual cpus. Werner uses a Debian host, but
the only big supply of Gb's I have is on Win7. Here's a ref on this:

http://www.virtualbox.org/ticket/5343

My VirtualBox seemed to be handling things like synaptic very well, and began to look
like a good avenue for me toward re-mastering. But somewhere in trying to get USB working, my VB has gotten damaged, I think, and even re-installs are not solving this problem.

I do have to wonder: if the hardware emulation in VirtualBox is poor, can we be
comfortable with the products it helps us produce, say by remastering? Might
it not inflict some distortion or artifacts into the process?

@ Werner

Thanks again for your help.
I am embarrassed at having co-opted your re-mastering thread for this VB diversion.
It may be a useful demo that one needs his VB house to be in good order BEFORE tackling
re-mastering with it.

klaus2008
07-10-2011, 04:06 PM
@ utu

Did you read the section "USB settings" of Chapter 3 in the Virtual Box manual (available online at http://www.virtualbox.org/manual/ch03.html ). The manual says that you first have to create a filter for the USB device after installing the extension pack. "On a Windows host, you will need to unplug and reconnect a USB device to use it after creating a filter for it."

I think that the developers should invent a simpler procedure to use USB device.

Capricorny
07-10-2011, 04:10 PM
It looks like a lot of folks with my sort of set-up have
a problem. That is, Win7, 64-bit and dual cpus. Werner uses a Debian host, but
the only big supply of Gb's I have is on Win7.

...And I still don't understand why you won't try with a poor man's install on all that NTFS real estate. Plus a volume for remastering workspace. I'm running off NTFS writing this, and I just created an extra volume yesterday, to shrink my persistent storage.

Werner P. Schulz
07-10-2011, 05:02 PM
if the hardware emulation in VirtualBox is poor, can we be
comfortable with the products it helps us produce, say by remastering? Might it not inflict some distortion or artifacts into the process?
... I think "Windows 7 + VirtualBox" isn't a good choice. Perhaps you will consider doing a Multiboot installation on your Laptop: for example Windows 7 and Debian? (Don't forget backup!)

"Debian + VirtualBox" runs for me without any problems. But why not work and do your experiences with Debian first?

Greetings Werner

utu
07-10-2011, 05:32 PM
@ Klaus

I'll try that again, but I think something's broken here.
Do you know of a successful instance of VB with my hardware setup?
If so, that would motivate me to persevere.
The ticket 5343 docket is pretty discouraging; I interpret it to
indicate lots of folks with my hardware setup are not getting the
results they'd expect with a Win7 host arrangement.

@ Capricorny

You force me to reveal yet another layer of my ignorance.
I think I've done poor man's installs to linux-formatted partitions,
but I thought the rule was NEVER to write linux stuff to ntfs.
Virtual box didn't seem like a violation of that rule.
Kindly point me to some reading material.
Re-formatting was my initial interest; VirtualBox is interesting,
but I can come back to that.

@ Werner

One thing I do know; I don't want to further partition the hard drive
on this laptop. As it stands, I don't have to worry about grub or lilo,
I can select my OS-du-jour just by using the built-in media select
to use HDD/DVD or USB. If I can do a 'poor-man's install' to the ntfs
without clobbering the Win7, that'll probably serve. That's where all
my Gbs are right now.

Again, thanks to all.

Werner P. Schulz
07-10-2011, 06:13 PM
From the manual:

In the Settings dialog, you can first configure whether USB is available in the guest at all, and in addition also optionally enable the USB 2.0 (EHCI) controller for the guest. If so, you can determine in detail which devices are available. For this, you must create so-called "filters" by specifying certain properties of the USB device.... you can define a filter for USB, but you need not. I never defined any filter for my USB sticks - simply plug in and they will be recognized.

utu
07-10-2011, 08:03 PM
I'm learning that Win7 may have a capability itself like VirtualBox,
and that VirtualBox uses an M$ format to do its thing:

Wikipedia says:

"Third-party products also use VHD file format. VirtualBox, part of
Sun xVM line of Sun Microsystems supports VHD in versions 2 and
later." Where VHD was earlier described as Virtual Hard Drive,
a Microsoft-acquired format.

It wasn't clear if Win7 Home Premium would have a VHD capability;
maybe only an Enterprise/Server level of Win7?

And I found this:

http://www.3dhomejoe.info/jsite/uploads/vhd.pdf

And I note I can display a Microsoft Management Console (MMC) that
looks like the pdf examples.
The MMC on my Home Premium Win7 looks like it might be capable of
doing a lot of manipulation, including formatting to fat or ntfs. Not clear
what the implication is of some other formatting, say ext or reiser.

I hope this is converging toward an accommodation of Werner's re-
mastering, Capricorny's poor-man's install and my up-till-now
unused ntsf capability. This still feels like re-partioning by another name,
and diddling with the Win7 install is problematic.

Capricorny
07-10-2011, 11:55 PM
@ Capricorny

You force me to reveal yet another layer of my ignorance.
I think I've done poor man's installs to linux-formatted partitions,
but I thought the rule was NEVER to write linux stuff to ntfs.
Virtual box didn't seem like a violation of that rule.
Kindly point me to some reading material.
Re-formatting was my initial interest; VirtualBox is interesting,
but I can come back to that.


No, I don't think you are ignorant! Conventional wisdom is don't write to NTFS, and for general,unrestricted use, I think that is still the safest advice. But poor man's installs are somewhat simpler than the general case, as just a few files are written, and except for knoppix-data.img, the rest are, mostly, read-only.

I have heard few reports of problems using NTFS this way, and I did my first poor man's install to NTFS/Win7 a few weeks ago - I try to avoid it, but not at any cost. Seems to work very well, and the same method that is used for creating the persistent image can be used for creating a larger volume for working - so you can effectively have two (or more) Linux partitions residing on an NTFS file system.

Again, I would try to minimize writing to NTFS, but for setting up a workplace for remastering, it should be ok. For permanent storage, I would have tried to shrink NTFS to make room for a native Linux partition, but I have no grave objections to using NTFS for a poor man's install - provided there are good back-up routines in place. After all, we only have to take care of one single file, and if the "install" gets corrupted, we just clean up and copy the files to disk once more.

Oh, and I should add: From Win7 viewpoint, cleaning up after such an install is just a matter of normal deletion of a few files. There is a small element of unsafety in that NTFS is mounted rw at /mnt-system - which means you have "automatic" access to all Windows files - also for corrupting them. But as long as you stay within /UNIONFS, there is not much of a danger AFAIK.

utu
07-11-2011, 12:12 AM
@ Capricorny

Please confirm I have this right.
Your Virtual Hard Drives are ntfs and not linux formatted?
Are you using M$ MMC to set these up?
Also is your Windows a Win7 Home Premium, or something grander?
Thanks.

Capricorny
07-11-2011, 12:33 AM
@ Capricorny

Please confirm I have this right.
Your Virtual Hard Drives are ntfs and not linux formatted?
Are you using M$ MMC to set these up?
Also is your Windows a Win7 Home Premium or something grander?
Thanks.

As I wrote: The entities residing on NTFS are files. Nothing else, nothing special. So I do nothing at all about them with Windows tools. In fact, I think I haven't booted Windoze for 6+ months. I have it installed because I _may_ need it, not because I use it :) For running Windows, I use VMware. Run off the cloop-mounted KNOPPIX file on NTFS, but the virtual machines reside on an ext3 (will probably turn to ext4 soon) partition.
My version: It's a Home Premium, but I don't think that matters.

But inside some of those NTFS files are Linux file systems - I mostly use ext2/ext3, but that doesn't matter much. And they are loop- or cloop-mounted.
As for VM technology, I have been using VMware workstation for 11 years - and I try to know as little as possible about it, and the alternatives, as long as it just works, and it mostly does. So I don't know about M$ MMC etc, which I think is much newer stuff than VMware.

I can see that it could be somewhat practical to test new Knoppix versions in a VM, but I normally copy new remasterings to USB media in any case, and then I can test run from them.

kl522
07-11-2011, 04:16 AM
I'm learning that Win7 may have a capability itself like VirtualBox,
and that VirtualBox uses an M$ format to do its thing:

Wikipedia says:

"Third-party products also use VHD file format. VirtualBox, part of
Sun xVM line of Sun Microsystems supports VHD in versions 2 and
later." Where VHD was earlier described as Virtual Hard Drive,
a Microsoft-acquired format.

It wasn't clear if Win7 Home Premium would have a VHD capability;
maybe only an Enterprise/Server level of Win7?


VHD ( virtual harddisk ) and virtual machine (VM) are different thing. Well,
a VM might use VHD, but having VHD only is no where near to having
the capability of VM. It's like you have the wheels it does not mean you
have the car. I am not sure if the Oracle's VM ( virtualbox ) is
able to use M$ VHD, but that's the last thing I need to worry about. When
you create a VM, the VM creates VHD automatically.

kl522
07-11-2011, 04:19 AM
For running Windows, I use VMware. Run off the cloop-mounted KNOPPIX file on NTFS, but the virtual machines reside on an ext3 (will probably turn to ext4 soon) partition.

VMWare on Linux can't have the virtual storage stored on NTFS. Somehow the VMware has serious performance problem with fuse driver.

kl522
07-11-2011, 04:33 AM
As for VM technology, I have been using VMware workstation for 11 years - and I try to know as little as possible about it, and the alternatives, as long as it just works, and it mostly does.

On my Knoppix system I have 3 different implementations of VMs :-

1. Oracle VirtualBox
2. VMware
3. Qemu

VMware and Virtualbox are nice to use because they are provided with GUI.
Some of my guest operating systems are in VMWare, and some are in VirtualBox.
I don't really know the pros and cons of VMware and virtualbox but virtualbox
is 'vde' aware, so when it comes to simulation of interconnecting multiple VMs,
it is more friendly. I have a setup where I interconnect 5 or 6 Virtualbox/
Qemu VMs networking with each other on my host Knoppix system.

Virtualbox and Qemu are in-conflict with one another, ie you can't run
virtualbox and qemu at the same time.

VMware and Virtualbox both are using their proprietary virtual harddisk, it
makes it hard to do file system check and fix file system errors. When I use
Qemu, I use native raw format, so in case of errors, I could fsck/fdisk the
harddisk using native OS (Linux ) tools.

utu
07-12-2011, 07:53 PM
I can report that I have tried VMWare Player at Capricorny's suggestion,
and that it cures ALL the complaints I have previously raised against VirtualBox.
'Player is free, as in 'Box, so I can afford it.

That said, I intend to resume trying Werner's re-mastering ideas, replacing
only VirtualBox with VMWare Player.
Any comments on this modification of Werner's procedure?

Werner P. Schulz
07-13-2011, 08:40 AM
... some years ago I also used VMWare; in the meantime I prefer VirtualBox. But for remastering purpose with my scripts there may bee no difference. In both cases you work with a HD installation of Knoppix, no matter whether using a virtual drive within a virtual machine of VMWare or VirtualBox.

Greetings Werner * http://www.wp-schulz.de/knoppix/summary.html
Own Rescue-CD with Knoppix (Knoppix V6.4.4 remaster)

utu
07-14-2011, 05:46 PM
@ Werner

Greetings, Werner.
In regard to your otherwise fine instructions:

The MBR question is ambiguous in regard to the term 'first hard disk'.
I am attempting this on a Win7 host.

Does 'first hard disk' refer to the hardware that the laptop boots into Win7 from or do you mean the virtual system I've just created on the guest machine?

I want to avoid diddling with the MBR on the laptop.

Werner P. Schulz
07-14-2011, 07:13 PM
The MBR question is ambiguous in regard to the term 'first hard disk'.

@ utu
I'm not quite sure, whar you mean with 'first hard disk'? If you refer to "Install GRUB bootloader to MBR", then it means: MBR of the virtual drive within the virtual machine.

utu
07-14-2011, 08:14 PM
@ Werner

The 'first-hard-drive' nomenclature is in the prompts
that come up during 'Knoppix-install-to-hd'. I'm really
finding this a tough slog, not knowing what next to
expect. I have the following suggestion for your Summary
page, that you have an opening paragraph entitled
Pre-requisites. I am coming to the conclusion that the
following at least must be anticipated (and I didn't):

1. A working VirtualBox or VMWare Player setup, and
some familiarity of working with such setup starting from
a working iso. I'm not at all sure Windows 7 qualifies
as an adequate host, either.

2. Previously prepared linux and linux-swap partitions of
some dimension or other of adequate size for the work
intended.

3. The usual expected familiarity with command line
operations and synaptics, with some notion of just what
changes might be interested in a re-mastering effort.
That is, what programs to add, which to remove. And
probably a good working familiarity with boot processes
is also necessary.

I'm sure this is old hat to you and others, but items one
and two are real speed bumps to those of us even fairly
well into Windows and Linux as working distributions, but
not versed in all the behind the scenes necessary for really
'involved' enterprises like re-mastering.

This lack of spelling out the prerequisites is a pervasive
negative characteristic of a lot of the interesting cutting
edge discussions on this forum, I might add. Poor-man's-
install comes to mind.

Werner P. Schulz
07-15-2011, 07:05 PM
To remaster a Knoppix isn't for beginners. The user must have experience with VirtualBox and especially with Linux.

I remaster a Knoppix
a) to obtain a Rescue CD with all the tools I like
b) to create a image of Ubuntu within a Knoppix DVD

If a user isn't experienced or he only needs little changes of Knoppix, then a "Flash disk Installation" or a "Poor man harddrive Installation" is a good choice.

My "summary" isn't as detailled as my explanations on my pages in German language. But in the summary is the instruction, how to do HD Installation within the virtual drive.

Greetings Werner * http://www.wp-schulz.de/knoppix/summary.html
Own Rescue-CD with Knoppix (Knoppix V6.4.4 remaster)

itman007
08-04-2011, 10:04 PM
Where is the revised version of Remaster? (url) Looking at the above post from Schlz I am lead to belive that I can install to flash or hard disk and make my changes there.? I need to make my changes / add or remove packages then start terminal services and the clients have access to the changes Ive made. I need also to auto start things like rdesktop or vm view or have the icons launchable from the desktop when the clients start . Any help or directions here? Thanks greatly!

Capricorny
08-04-2011, 10:52 PM
There are actually several rather different methods working. My (and some others') preferred way is to

1. Copy the compressed KNOPPIX image plus the persistent image to harddisk
2. aufs-mount them like in a running KNOPPIX instance
3. "unwrap" the /UNIONFS file system by copying (rsyncing) most of it to a remastering directory.
4. Do some minor modifications, like copy /home and /var over to a new persistent image, and copy the original Knoppix versions of these directories to the file system to compress.
5. Create isofs from the remastering directory, and compress that to a new cloop image. Squashfs is a good alternative to cloop, but requires modification of minirt.gz
6. Create new persistent image from /home and /var.
7. Copy the new versions of KNOPPIX(.sq) and knoppix-data.img into an existing KNOPPIX directory, and test run that in qemu. By setting up main drive grub to boot the remastered KNOPPIX image, you can test run it by running qemu /dev/sda & and choose the new boot option from grubs menu. There you may also make modifications.

As for net booting Knoppix, it works fine, but I don't know what has to be done to make adaptation for the clients. Everything from the cloop image is available, but I don't know about persistent store etc. But you can pack any amount of startup scripts into the remastered image, I think.

Werner P. Schulz
08-04-2011, 11:37 PM
Where is the revised version of Remaster? (url)Remastering HowTo (http://www.knoppix.net/wiki/Knoppix_Remastering_Howto)

... you can combine:
1. remaster Knoppix
2. "Flash disk Installation" of remastered Knoppix (Install FAQ (http://www.knoppix.net/wiki/Install_FAQ))

Greetings Werner * http://www.wp-schulz.de/knoppix/summary.html
Own Rescue-CD with Knoppix (Knoppix V6.7.0 remaster)

itman007
08-08-2011, 10:12 PM
Ok Ive Selected Flash Disk Install and all seems to have installed to the flash disk. I set size for persistent home. So if I understand correctly I can now remove things that I dont want and add apps that I wish? Then once thats done an iso of this flash install can be created?

Werner P. Schulz
08-09-2011, 09:43 AM
Oh, no! You can remaster a Knoppix with all your changes as you prefer. Then do a "Flash disk Installation" of this remastered ISO. In this Flash disk Installation you can additional changes and they will be stored in the persistent memory. But you cannot create a new ISO of "remasterd ISO" + "persistent memory".

Why? If you for example purge GIMP and then remaster Knoppix, then GIMP isn't in the new ISO. The new ISO is not as big as before.

But if you "purge" GIMP in the Flash disk installation. then GIMP isn't really purged. It remains in the compressed filesystem KNOPPIX; only some remarks will be done in the UNIONFS to make GIMP invisible.

Now, if you try to build a new ISO using your Flash disk Installation, then there would be no benefit; all is as big as before.

That is the reason, why I prefer remastering first. I get a small ISO to burn on CD or to use for Flash disk Installation. And a small ISO boots faster than a big one.

Capricorny
08-09-2011, 10:07 AM
Oh, no! You can remaster a Knoppix with all your changes as you prefer. Then do a "Flash disk Installation" of this remastered ISO. In this Flash disk Installation you can additional changes and they will be stored in the persistent memory. But you cannot create a new ISO of "remasterd ISO" + "persistent memory".

Why? If you for example purge GIMP and then remaster Knoppix, then GIMP isn't in the new ISO. The new ISO is not as big as before.

But if you "purge" GIMP in the Flash disk installation. then GIMP isn't really purged. It remains in the compressed filesystem KNOPPIX; only some remarks will be done in the UNIONFS to make GIMP invisible.

Now, if you try to build a new ISO using your Flash disk Installation, then there would be no benefit; all is as big as before.

That is the reason, why I prefer remastering first. I get a small ISO to burn on CD or to use for Flash disk Installation. And a small ISO boots faster than a big one.

This is one of the reasons we include a copying step in our "poor man's remastering" process: rsyncing most of /UNIONFS from a flash install to a new directory, we purge all removed packages, and then create a new isofs/squashfs. I know that some of the remasterers of live Ubuntu have just created the new image directly from the aufs-mounted system. This goes faster, but I don't think it is as safe. Also, I myself prefer not working from a live file system.

kl522
08-09-2011, 11:21 AM
But if you "purge" GIMP in the Flash disk installation. then GIMP isn't really purged. It remains in the compressed filesystem KNOPPIX; only some remarks will be done in the UNIONFS to make GIMP invisible.

Now, if you try to build a new ISO using your Flash disk Installation, then there would be no benefit; all is as big as before.


Hmm it depends. It depends on ***HOW*** you remaster your ISO using the flash disk installation. If one does it the correct way, it is possible to remaster the new ISO to make it smaller ( ie leave out the deleted files). In fact, ***ALL** the remaster of flash installation mentioned in this forum, one of the purposes is to make the ISO/compressed file system smaller.

So if one asks me which should come first, flash installation or remaster ? I would say flash install first. Flash install first, customize it the way you want it to be. Test it fully, 3 months or later, then remaster.

Werner P. Schulz
08-09-2011, 12:12 PM
Hmm it depends. It depends on ***HOW*** you remaster your ISO using the flash disk installation. If one does it the correct way, it is possible to remaster the new ISO to make it smaller ( ie leave out the deleted files). Have you a script example, how to do this?

kl522
08-09-2011, 12:41 PM
Have you a script example, how to do this?

http://www.knoppix.net/forum/threads/28447-Re-master-your-flash-installation/page2

Forester who has disappeared from this forum has written a few scripts on this. I must admit that I haven't tried any of his scripts and I also don't really know which is the latest and best, but I believe he has gotten the essence of flash remaster, which is to clean up the wiped-out files.

My original post requires a huge disk space to be allocated for the remaster. But I believe he took the idea one step further, he probably remaster based on the structure of flash installation.

itman007
08-09-2011, 02:26 PM
Well now Im totally lost! The link I followed previously for remaster didnt work. The ISO wasnt created in a useable state. The ISO was 11Mb and seemed to be a corrupt file. Per statement from Schulz, "... you can combine: 1. remaster Knoppix 2. "Flash disk Installation" of remastered Knoppix"
So which is it? Is Flash disk and remaster not combined? My Flash Disk install is pretty much customized to what I need. Now that was a waste of time if I cant make an ISO of it.

utu
08-09-2011, 04:39 PM
There are actually several rather different methods working. My (and some others') preferred way is to

1. Copy the compressed KNOPPIX image plus the persistent image to harddisk
2. aufs-mount them like in a running KNOPPIX instance
3. "unwrap" the /UNIONFS file system by copying (rsyncing) most of it to a remastering directory.
4. Do some minor modifications, like copy /home and /var over to a new persistent image, and copy the original Knoppix versions of these directories to the file system to compress.
5. Create isofs from the remastering directory, and compress that to a new cloop image. Squashfs is a good alternative to cloop, but requires modification of minirt.gz
6. Create new persistent image from /home and /var.
7. Copy the new versions of KNOPPIX(.sq) and knoppix-data.img into an existing KNOPPIX directory, and test run that in qemu. By setting up main drive grub to boot the remastered KNOPPIX image, you can test run it by running qemu /dev/sda & and choose the new boot option from grubs menu. There you may also make modifications.



I think this might be a good outline to enlarge upon.
I suggest that some code or pseudo code be added to make more specific what goes on in each step.
I other words, convert this outline into a script or several scripts.
If appropriate capitalize on Werner's style and content.

As an aside, I note that the PCLinuxOS 'mylivecd' approach uses bleachbit as a clean-up step
to remove unnecessary leftovers in the remastering process. Does that have any part in this
procedure? Should it?

Werner P. Schulz
08-09-2011, 05:01 PM
In your original post (http://www.knoppix.net/forum/threads/28447-Re-master-your-flash-installation/page1), I think

# mkdir -p /lots_of_free_space/source/KNOPPIX
# mkdir -p /lots_of_free_space/master/KNOPPIX
# cd /lots_of_free_space/source/KNOPPIX
# cp -a /dev .
# cp -a /UNIONFS/bin .
...
can do the job, to really purge the "hidden" packages. But the whole procedure you described to get at least a new and small ISO, seems to be very complicated and not for an inexperienced user.

By the other way, if you first remaster, then purging all not needed packages and the lots of dependencies will be done automatically by "aptitude".

In the scripts of Forester I cannot find a hint, how to purge "hidden" packages by building a new ISO. Perhaps I didn't searched for it enough.

Werner P. Schulz
08-09-2011, 05:28 PM
Well now Im totally lost! The link I followed previously for remaster didnt work. The ISO wasnt created in a useable state. The ISO was 11Mb and seemed to be a corrupt file.... which link for remaster did you follow? Be aware, I did the revised version of Remaster Howto in the middle of June. If you used old parts of this HowTo, then remaster will not work.

By the way, to remaster a Knoppix DVD isn't a good idea; the DVD is to big for remastering purpose. Make a Flash disk Installation of the DVD and use the persistent memory only for some little modifications and especialy for your user files.

Per statement from Schulz, "... you can combine: 1. remaster Knoppix 2. "Flash disk Installation" of remastered Knoppix" So which is it? Take care of this order!

Greetings Werner * http://www.wp-schulz.de/knoppix/summary.html
Own Rescue-CD with Knoppix (Knoppix V6.7.0 remaster)

kl522
08-09-2011, 09:12 PM
In the scripts of Forester I cannot find a hint, how to purge "hidden" packages by building a new ISO. Perhaps I didn't searched for it enough.

I thought everything is in his post #11 where he has already provided a script for it ?

Purging is encapsulated and granted. As long as one takes the files from /UNIONFS and send it to mkisofs, you don't have to worry about wiped-out files, the aufs handles it.

Actually, for the purpose of creating a new ISO from installed flash installation, it is a matter of ***ONE*** script ( as long as the needed space and virtual memory/swap space requirement are met ). As you said, synaptics (plus aufs) handles it already.