How shall I migrate to the DVD version?

[Harry Kuhman]

The DVD version of Knoppix is fast approaching and I just realized that I'm likely to have a problem. How am I going to download and make that new DVD?

I do have a DVD burner, that's not the problem. But it's on my Win98 system. The one with all FAT partitions on the hard disk. The one that can't have files larger than 4 gig. If The Knoppix DVD iso is larger than 4 gig, how am I going to burn it?

I do have XP, but it's on my notebook, with a combo CDRW writer and DVD reader there. Can't make my DVD there. Of course, I can boot Knoppix on the desktop with the DVD writer, but I'll still have the FAT file sizes to deal with.

What should I do? Can I pull off downloading the ISO into XP (unfortunately, I shrunk my NTFS partition to make space for a Linux campatable FAT partition on the small notebook hard disk and don't have enough space for a >4 gig download there now) and somehow burn a DVD with Knoppix from a shared file across the network? That seems like an invitation to buffer underruns, even with burn-proof technology. Or is there an obvious solution I'm missing? I sure don't want to get XP for this old desktop just to burn a Knoppix DVD.

Of course, the first Knoppix DVD iso might be less than 4 gig, but I don't expect it, unless it were done that way exactly because of the problem I've stated. I rather expect though that the focus will be on packing every application they can onto that DVD. I just hope it includes all of the great games that have vanished from the ISO since I saw my first 3.1 version.

Maybe I can figure out a way to shrink a partition on the desktop system and then make a Linux partition that will not have the 4 gig limit. Use Knoppix to download the ISO to it and hope that Knoppix supports my Dual Layer Benq DVD burner. Sure wish that Knoppix contained one of the newer versions of Bittorrent though, the new versions download cleanly for me, the old versions overwhelm my upload capacity and as a result the download grinds to a halt (that will sure be an issue getting a popular DVD size download).

[OP4Latino]

Hmm, that sucks.

The easiest solution (but expensive) I figured out is buying an External Harddisk (>$80)

[Markus]

If you have extra dvd rw's lying around, you could backup a partition on the desktop and make it ext3. After downloading and burning just reformat fat and put the files back. Easier at least than burning over a network. Timeconsuming though.

[dakota]

Why not download it on your Laptop, and burn it over the network? Don't know wheather your burningsoftware can do it, but there are tool taht can burn with a burner in another PC over the network. When you dont make haste (e.g. burn it as single speed DVD 8) ) there won't be any bufferunderuns for sure.

Like you already guessed right, the iso won't be too small, since there are over 9 gig software from the debian-pool in the next Knoppix release.
And the cloop-Compression is no "wonder-in-a-box" that shrinks it until there is nothing left for the single-sided-DVD...

Wheather your beloved games are in once more, you can have a look under ftp://ftp.uni-kl.de/pub/linux/knoppix/packages-4.0dvd.txt if you know the old package-names.

[Harry Kuhman]

Hmm, that sucks.

The easiest solution (but expensive) I figured out is buying a External Harddisk (>$80)

Actually, I'm thinking that might be the fix too, and although I didn't mention it, I already have one.

I had a serious problem with it when I used it with both the Win98 and XP systems though. I had put two FAT partitions on it from the Win98 system. I moved it to the XP system and read and wrote some files. But when I moved back to the Win98 system I found that the partitions were severaly corrupted and all of the files were lost!!! Lost from the XP view at that point too. I've pulled the drive and have been trying various recovery techniques on-and-off fom the Win98 system, but so far have recivered very little.

I had seen a somewhat similar problem when I made a multi-boot Win95/Win98 system years ago. The problem tracked down to our good friends at Microsoft doing different LBA translations in each OS. So for certain capacity drives that used LBA, the sectors didn't map quite the same. In that case the problem only showed itself after I reached a certain block address on the drive, so I could fill it about 80% full of files before I destroyed everything with the LBA issue between the different OSs.

So after corrupting yet anoher hard drive with two different MS OSs, I decided to format the next disk as NTFS and just use the external drive with the XP notebook. That was the primary reason for getting it anyway, but it would have been nice to be able to move it between the desktop system and the notebook. But I can't afford to destroy another disk full f files doing so.

Since Knoppix can mount the NTFS partition read-only, it should be safe to move the hard drive over to this system and mount the NTFS partition from Knoppix. I would likely download the ISO with Bittorrent on the XP system (to avoid the older version of Bittorrent on Knoppix that gives me problems), and if it is over 4 gig then move the external hard drive to the desktop system system and try to burn the DVD from Knoppix. This still presents a couple of questions: Will Knoppix see my firewire external hard drive (it should but I've never had occasion to test that), and will it be able to burn to my new model DVD burner, and from an ISO out on the firewire drive? But it seems likely it may be the best approach I have available to me at this point.

I should also mention that my desktop system and a few other test systems I also keep running (including my Debian hard drive install system) all have one hard drive in a removable hard drive tray, so swapping hard drives between systems (other than the notebook) isn't a big issue. Can't put the firewire hard disk into the tray though, it's larger capacity than the hardware on the other systems will support.

[Harry Kuhman]

Why not download it on your Laptop, and burn it over the network? Don't know wheather your burningsoftware can do it,....
Yup, I don't know either. It's another option to consider, if it works. I'm using Nero 6 as the burning tool (own a copy of NTI as well, and did get an update from their website that provided drivers for my DVD drive, so that will be another thing to try if Nero has issues). One issue about the burning will be that I'll likely have to have the ISO on a firewire box on the notebook, I shrunk the NTFS primary partition down to just what was needed for XP and the clutter that came on the notebook to make space for a Knoppix friendly FAT. So the burn will have to happen both across firewire and the network. In transfering files back and forth this way I can tell you it's not particularly fast. I guess I'll know in a few weeks if it's fast enough.


Like you already guessed right, the iso won't be too small, since there are over 9 gig software from the debian-pool in the next Knoppix release.
Yea, but there are said to be about 2 gigs of files on the 700 meg Knoppix CD. So 9 gig is cutting it close to the 4 gig FAT file size limit. Still, I betting thet they will not waste that extra space and the ISO will exceed the FAT limit.


Wheather your beloved games are in once more, you can have a look under ftp://ftp.uni-kl.de/pub/linux/knoppix/packages-4.0dvd.txt if you know the old package-names.
Unfortunately, I don't remember package names. Not sure I even ever knew the package names. I do remember there was a nice Pool game that vanished after the first Knoppix CD I saw. And a lot of arcade type games seemed to vanish shortly after that. Maybe I'll have to dust off an old 3.1 CD and take a look. Is there a way I can determine the package name from looking at the CD and the game itself?

[OP4Latino]

I had seen a somewhat similar problem when I made a multi-boot Win95/Win98 system years ago. The problem tracked down to our good friends at Microsoft doing different LBA translations in each OS. So for certain capacity drives that used LBA, the sectors didn't map quite the same. In that case the problem only showed itself after I reached a certain block address on the drive, so I could fill it about 80% full of files before I destroyed everything with the LBA issue between the different OSs.

I don't know if the LBA that you are mentioning is the same media descriptor byte that MS is mentioning here Floppy Disk is Not Accessible, Not Formatted, or Not Recognized by Windows (http://support.microsoft.com/default.aspx?scid=kb;en-us;140060)


Will Knoppix see my firewire external hard drive

For some reasons Kno 3.8 and up have a hard time to detect my USB and firewire devices, incluing my External HD. (I haven't make a post yet, but I will in a future). My workaround is start Kno and unplug/replug the device.


and will it be able to burn to my new model DVD burner, and from an ISO out on the firewire drive?

I have burned ISOs from the External HD before, but I don't know about your DVD burner.

[OP4Latino]

I shrunk the NTFS primary partition down

How did you do it without loose your data? Did you use a paid software?

[UnderScore]

I shrunk the NTFS primary partition down

How did you do it without loose your data? Did you use a paid software?maybe he did but I know that Knoppix includes qtparted which interfaces with all the filsystem tools like mke2fs & ntfresize. The key to resizing a NTFS partition is to first scan disk it. Next defrag it. Then disable the Windows Virtual Memory & this will delete the swap file. Then boot knoppix & use qtparted to resize the ntfs partition. Boot back into windows & then either windows will detect the C: drive changed & will issue its own scan disk (in a Blue Screen) or you must issue a scan disk manually. Then enable Windows Virtual Memory which will rewrite the swap file.

Of course, There is ALWAYS some risk of losing data. Please ALWAYS make sure that you have a working backup of your important data prior to
experimenting with ANY new software, no matter if Windows, Linux or anything else. :)

[Harry Kuhman]

I don't know if the LBA that you are mentioning is the same media descriptor byte that MS is mentioning here Floppy Disk is Not Accessible, Not Formatted, or Not Recognized by Windows (http://support.microsoft.com/default.aspx?scid=kb;en-us;140060)
Nope. That was an interesting link, I wasn't aware of that issue, but it seems to be a different issue. What I saw between Win95 and Win98 was a LBA issue (Logical Block Addressing, a hack that has been added because modern drives had too many real cylinders to fit in with the MS way of accessing hard drives). It should not affect floppy drives. I'm guessing, but only guessing, that what corrupted the FAT partition between Win98 and XP was also an LBA issue. But whatever it was, it's there, and I don't move drives between Win98 and XP any more.




Will Knoppix see my firewire external hard drive For some reasons Kno 3.8 and up have a hard time to detect my USB and firewire devices, incluing my External HD. (I haven't make a post yet, but I will in a future). My workaround is start Kno and unplug/replug the device.
Thanks for that tip. I may also try just backing up to my 3.7 copy of Knoppix and hope that it supports the DVD burner.


I have burned ISOs from the External HD before, but I don't know about your DVD burner.
Nor do I. Coud experiment now but I might as well wait until I have the ISO. Just considering all my options at this point since it looks like I'll have to put a bit more work into this than I normally do to burn an ISO.


How did you do it without loose your data? Did you use a paid software?
Unfortunately, yes. I have a friend that owns a copy of Partition Magic. Since it was a notebook system it was easy enough to carry it to his place and shrink the partition. Made a backup first though, always a good move when doing stuff like that.

[UnderScore]

Facts from my point of view :), OK fine, my opinions.

Knoppix cloop is compressed to a 100%:37% ratio. When making remastered CDs I typically see a compression rate of 37% or 38%. This means that a resulting KNOPPIX cloop of 695MB is close to 1850MB uncompressed. Based on the fact that mkisofs & ISO 9660 limits any single file in the filesystem to 2GB. So a KNOPPIX cloop file on a DVD is going to be at or just under 2GB. Assuming the compression rate is still around 37%, then Reverse the compression & it is roughly 5.5GB uncompressed. Still not enough for the whole Debian repository ;) but that is simply HUGE compared to what a CD can hold.

I hope they have made 2GB DVD ISO image & not have made it bigger. I am gonna kick myself someday for saying this, but I think 5.5 uncompressed is enough for everyone. :D

I noticed the linked package list & I am impressed. Knoppix 4.0 will include both KDE 3.4.1 & GNOME 2.8.

[dakota]

And nobody will ever need more than 640 kByte RAM. Ask Bill Gates. :D


... Reverse the compression & it is roughly 5.5GB uncompressed.
As already mentioned, it is over 9 gig out of debian.

[UnderScore]

As already mentioned, it is over 9 gig out of debian.Ouch thats big!

[gildedlink]

Well you could try downloading it to the external drive, remastering it (I assume if there's that much space then alot of those programs can be done without) and trying to transfer to the FAT drive and burn it that way. I dunno if I'm describing it right, I've never done a remaster myself but if it has KDE AND Gnome is it possible to get rid of one? Getting rid of 700 megabytes of compressed space is probably quite a challenge but possible.

Other than that, there are some things you can buy which house an external drive in a case that makes it internal. If knoppix didn't recognize it externally, maybe it'll recognize it internally.

[Fabianx]

You'll have no problem at all!

KNOPPIX still has the 2 GB limit on the CD, because of the iso9660 filesystem restrictions.

What to do then?

Easy: Split it to _two_ KNOPPIX images. KNOPPIX and KNOPPIX2.

I guess (though I have not tried) that it'll also work with tohd and fromhd, so there should be no problem extracting it from the ISO without burning it and using the light version to boot the maxi one. Even from FAT partitions.

cu

Fabian, Happy Owner of Knoppix 4.0 Linuxtag DVD sitting on Klaus Knoppers PC ;-)

[tablet_guy]

The DVD version of Knoppix is fast approaching and I just realized that I'm likely to have a problem. How am I going to download and make that new DVD?

I do have a DVD burner, that's not the problem. But it's on my Win98 system. The one with all FAT partitions on the hard disk. The one that can't have files larger than 4 gig. If The Knoppix DVD iso is larger than 4 gig, how am I going to burn it?
.

I think I have a solution for you, if your linux cd/dvd recorder setup is functional (i.e. cdrecord/2.6 kernel complications haven't been causing CD burning headaches for you).

It uses an lvm2/dmsetup approach: from your debian installation, create two loop devices on your FAT drive and join the two loop devices using dmsetup. I am attaching a sample script for doing this, please adopt it to suit your needs, and use judiciously.

After you set the variables and run the script, you can download the massive DVD iso on to the $MTPT directory (as you define in the script), and then burn it from linux using k3b or some other program.

I only recently learned of this rather well kept-secret of lvm2. It gives powerful ways to create volumes. Very nice!

Hope it works for you, and please post your adventures here.


#!/bin/sh -x
#
# How to create an LVM2 volume
# using two loop devices
# (can be easily extended to more than 2 loop devices)
# Ideas from:
# http://www.tldp.org/LDP/LGNET/114/kapil.html
# https://wiki.ubuntu.com//LiveCDPersistence
#
# This script creates two image files
# in the specified directory (KDIR variable)
# of specified size (LOOP_SIZE variable below),
# mounts them as loop devices, merges them using
# device mapper, creates an ext2 filesystem
# on the resulting volume, and finally mounts it
# on the mount point $KDIR/mnt

# setting LOOP_SIZE to 2560 gives a combined
# volume (with two images) of exactly 5GB, but the filesystem
# headers may eat up a sizeable chunk of it
# so, adjust it as necessary

KDIR=/C/knx40 # directory where the image files are to be created
MTPT=$KDIR/mnt # mount point for the logical volume
# set the size of each loop device image in MB
LOOP_SIZE=2560
# other variables
IM1=$KDIR/part1.img
IM2=$KDIR/part2.img
LP1=/dev/loop1
LP2=/dev/loop2
DVOL=snap
TBLFL=/tmp/dmsetup_table1
CHUNK_SIZE=8 # sectors

# the programs and modules
apt-get install dmsetup
modprobe dm_mod
modprobe dm_snapshot

[ ! -d $KDIR ] && mkdir $KDIR
[ ! -d $MTPT ] && mkdir $MTPT

dd if=/dev/zero of=$IM1 bs=1M count=$LOOP_SIZE
dd if=/dev/zero of=$IM2 bs=1M count=$LOOP_SIZE
losetup $LP1 $IM1
losetup $LP2 $IM2
SZ1=$(blockdev --getsize $LP1)
SZ2=$(blockdev --getsize $LP2) # redundant in this case, but just in case
# SZ1plus2=$(expr $SZ1 + $SZ2)
echo 0 $SZ1 linear $LP1 0 > $TBLFL
echo $SZ1 $SZ2 linear $LP2 0 >> $TBLFL
# echo $SZ1plus2 $SZ3 linear $LP3 0 >> $TBLFL
dmsetup create $DVOL $TBLFL
#echo "0 $SZ1 snapshot $LP1 $LP2 p $CHUNK_SIZE" | dmsetup create $DVOL
mke2fs /dev/mapper/$DVOL

mount /dev/mapper/$DVOL $MTPT
df $KDIR/mnt

exit 0

# device deletion script
# cut and save as a separate script
# and then run it when the volume is no longer needed
# running it
#!/bin/sh -x

KDIR=/C/knx40
MTPT=$KDIR/mnt
IM1=$KDIR/part1.img
IM2=$KDIR/part2.img
LP1=/dev/loop1
LP2=/dev/loop2
DVOL=snap
TBLFL=/tmp/dmsetup_table1

umount $MTPT
dmsetup remove $DVOL
losetup -d $LP1
losetup -d $LP2

# uncomment as needed
# be careful!
# rm -f $IM1
# rm -f $IM2
# rm -rf $MTPT $KDIR

[Harry Kuhman]

Hope it works for you, and please post your adventures here.
Well, any effort is going to wait until an Official release of the DVD ISO. I'm not going for the current ISOs popping up. But you completely lost me at from your debian installation, create two loop devices on your FAT drive and join the two loop devices using dmsetup. I have no idea what that even means in English. It's not likely to happen for another reason too, my Debain install is not on the system that has the DVD burner. So if I burn the ISO it will have to be from Win98 or from a live CD (likely Knoppix). There are both a CDRW and a DVD burner on this system, so I hope I can boot Knoppix from the CD and then burn on the DVD.

Fabianx gives us a reason to expect that this will not be a problem (http://www.knoppix.net/forum/viewtopic.php?p=86969#86969) and we'll be able to do it all from Win98 and a FAT partition. But the proof is yet to be seen.

I will post back my results at burning, but many people may be too busy playing with their new Knoppix systems to read them by then.

[eco2geek]

Um, Harry, how many hard drives do you have in that Win98-based system of yours?

Parallel ATA hard drives are out; serial ATA is in. That means they're clearing out the old hard drive models for ridiculous prices. For example, you can get a 200GB Seagate UATA drive from Fry's (http://shop3.outpost.com/product/4008252?site=sr:SEARCH:MAIN_RSLT_PG) for $75 after the rebate. That's cheap.

Buy a second hard drive, and install some version of Linux on it. Problem solved.

[Harry Kuhman]

Um, Harry, how many hard drives do you have in that Win98-based system of yours?

Parallel ATA hard drives are out; serial ATA is in. That means they're clearing out the old hard drive models for ridiculous prices. For example, you can get a 200GB Seagate UATA drive from Fry's (http://shop3.outpost.com/product/4008252?site=sr:SEARCH:MAIN_RSLT_PG) for $75 after the rebate. That's cheap.

Buy a second hard drive, and install some version of Linux on it. Problem solved.
Actually, I have 3, as well as the 2 optical media writers. And a hard drive tray that indeed might end up holding a drive for Debian. Unfortunately, the 200 gig drive is not an option, this hardware has the 128 gig limit (thus the 3 smaller drives). But Fabianx's answer is still the most promising, sounds like I may not have to add anything. Sure don't want to buy a small drive, better to save my sheckels and be closer to a new system and new large hard drive when the time comes.

[HLS]

I did not see this posted, but I believe that Azureus can chop up the download into several files. I was using Bit Tornado the other day and cruising at 1350kbps on my ADSL and about finished at 3.99G when the program went belly up at 4.00G, fat32 limitation. That download had taken less than an hour. I almost croaked; that was the best time ever, probably due to a lot of T3 connections and few on the take. The next attempt on an NTFS partition took almost 24 hours, but I finally got it. 8)

[audioaficionado]

I didn't think about any file size limitations on FAT32. I just started downloading the torrent version of the iso to a FAT32 partition on my data drive. I'm using XPproSP2 if that makes any difference. My only NTFS partition is the one XP is on. I have the room but I'd rather just use the 2nd drive's storage partitions.

Should I stop Torrent and move everything over to the NTFS partition?

Nevermind. I saw the post just prior to this and decided to move it all over to the desktop. My seed rate is already exceeding my leech rate by 3x.

[ketilfroyn]

The DVD version of Knoppix is fast approaching and I just realized that I'm likely to have a problem. How am I going to download and make that new DVD?

I do have a DVD burner, that's not the problem. But it's on my Win98 system. The one with all FAT partitions on the hard disk. The one that can't have files larger than 4 gig. If The Knoppix DVD iso is larger than 4 gig, how am I going to burn it?

I do have XP, but it's on my notebook, with a combo CDRW writer and DVD reader there. Can't make my DVD there. Of course, I can boot Knoppix on the desktop with the DVD writer, but I'll still have the FAT file sizes to deal with.

What should I do? Can I pull off downloading the ISO into XP (unfortunately, I shrunk my NTFS partition to make space for a Linux campatable FAT partition on the small notebook hard disk and don't have enough space for a >4 gig download there now) and somehow burn a DVD with Knoppix from a shared file across the network? That seems like an invitation to buffer underruns, even with burn-proof technology. Or is there an obvious solution I'm missing? I sure don't want to get XP for this old desktop just to burn a Knoppix DVD.

Of course, the first Knoppix DVD iso might be less than 4 gig, but I don't expect it, unless it were done that way exactly because of the problem I've stated. I rather expect though that the focus will be on packing every application they can onto that DVD. I just hope it includes all of the great games that have vanished from the ISO since I saw my first 3.1 version.

Maybe I can figure out a way to shrink a partition on the desktop system and then make a Linux partition that will not have the 4 gig limit. Use Knoppix to download the ISO to it and hope that Knoppix supports my Dual Layer Benq DVD burner. Sure wish that Knoppix contained one of the newer versions of Bittorrent though, the new versions download cleanly for me, the old versions overwhelm my upload capacity and as a result the download grinds to a halt (that will sure be an issue getting a popular DVD size download).

You're hung up on the file size limitation in FAT. The file size limitation doesn't mean you can't store the entire image. Boot knoppix on your desktop, and do something like this:

wget -O - http://url/to/knoppix.iso | split -b 2147483648 - knoppix.iso-part
cat knoppix.iso-part* | md5sum
cat knoppix.iso-part* | cdrecord -v dev=/dev/hdc speed=4 -

You are of course using bootfrom=... or toram or something like that to make the DVD-burner available.

Cheers,
Ketil Froyn
ketil@froyn.name
http://ketil.froyn.name/