Knoppix 7.0.2-EN DVD notes & comments

[ruymbeke]

Hello,
Did someone managed yet to get the 64 bits compile & run time environment working on Knoppix 7.0.2 ?
The steps provided by Klaus for Knoppix 6.7.1 are no longer working (cf link below)
http://knoppix.net/forum/threads/29479-Knoppix-V6.7.1?p=125222&viewfull=1#post125222
and my other attempts have failed so far.
Best Regards,
Gilles

[utu]

Greetings, Gilles.
Is your 7.0.2 running a 64-bit kernel?

[ruymbeke]

Hi Utu,
Yes of course.
The following line was working well enough on Knoppix 6.7.1 with the 64 bit Linux kernel:
sudo apt-get -y -t sid --reinstall install libc6 libc6-dev libc6-dev-amd64 gcc-4.6-multilib lib64gcc1
But this does not work any more with Knoppix 7 (7.0.0 nor 7.0.1 nor 7.0.2).
There are conflicting packages that are in use and switching into single user mode (init 1) does not help.
Cheers,
Gilles

[klaus2008]

Hi Gilles,

I have successfully tried the following with Knoppix V7.0.2.

You have to replace the packages libc-bin, libc-dev-bin, libc6, libc6-dev with those of the unstable branch.

Open a terminal an issue
sudo apt-get update
sudo apt-get -d -t unstable install libc-bin libc-dev-bin libc6 libc6-dev
sudo chvt 1
init 2
apt-get -t unstable install libc-bin libc-dev-bin libc6 libc6-dev
chroot /UNIONFS/
apt-get -t unstable -f install
exit
init 5
chvt 5 Now you can install the current packages gcc-4.6-multilib and g++-4.6-multilib from the unstable branch.

sudo apt-get -y -t unstable install gcc-4.6-multilib g++-4.6-multilib Download and extract pi_quick_start.tar.gz

Change to your pi_quick_start directory and load start.sh into a text editor. Replace all occurances of "cc" with "gcc -m64". Save the file and exit the text editor.

Set the environment variable CFLAGS:
export CFLAGS=-m64Run the script:
./start.shBest Regards

[ruymbeke]

Hi Klaus,
This works great, thank you very much !
Would you consider adding these 64 bits packages in a future Knoppix DVD
such as this 64bits libraries and compile environment would be ready
to be used by the user when booting with the 64 bit kernel ?
Would you also consider adding the oprofile kernel module ?
Best Regards,
Gilles

PS: I noticed that the "libc" and "ld-linux" 64 bits libraries are now located into "/lib/i386-linux-gnu".
Shouldn't they be located into /lib64 instead (as it is on Knoppix 6.7.1) ?

[otropogo]

I must confess I couldn't quite follow Klaus's explanation on booting 7.0.2 in 64-bit mode, especially since I have no experience in running 64-bit OSs. However, I tried booting with the "knoppix64" argument and when I do, System Monitor shows my PC as having 3.9GB of RAM available, whereas when booting normally, the utility is only showing 2.9GB of RAM. So I assume this means that the 64-bit kernel is working...

[ruymbeke]

Hi otropogo,
Yes you are running a 64 bits kernel since you can see most of your RAM (3.9GB instead of 2.9GB with a 32 bits kernel).
You can also see that you are running a 64bits kernel by running the command "uname -a" from a shell console:
"Linux Microknoppix 3.3.7-64 #40 SMP PREEMPT Tue May 22 08:47:38 CEST 2012 x86_64 GNU/Linux"
As Klaus explained previously, besides the fact that a 64bits kernel allows you to use almost all of your RAM
as most 64 bits applications the 64 bits kernel is also more efficient (runs faster) but at a price of a larger footprint
which is not desirable for a CD/DVD (especially when speed does not really matter for most users and applications),
reason for Klaus to only have the 64 bits Kernel available on his great Knoppix distribution (and have all the libraries
and other applications only as 32 bits code, knowing that 32bits code can be used with a 64bits kernel).
Now if for some reason you really like or need 64 bits native code to run on that platform then you need to have
some extra libraries (as glibc) and other tools to create these 64 bits applications (as gcc) and this is exactly
these installation steps that Klaus explained previously. Hope this helps.
Best Regards,
Gilles

[otropogo]

Hi otropogo,
Yes you are running a 64 bits kernel since you can see most of your RAM (3.9GB instead of 2.9GB with a 32 bits kernel).
You can also see that you are running a 64bits kernel by running the command "uname -a" from a shell console:
"Linux Microknoppix 3.3.7-64 #40 SMP PREEMPT Tue May 22 08:47:38 CEST 2012 x86_64 GNU/Linux"
As Klaus explained previously, besides the fact that a 64bits kernel allows you to use almost all of your RAM
as most 64 bits applications the 64 bits kernel is also more efficient (runs faster) but at a price of a larger footprint
which is not desirable for a CD/DVD (especially when speed does not really matter for most users and applications),
reason for Klaus to only have the 64 bits Kernel available on his great Knoppix distribution (and have all the libraries
and other applications only as 32 bits code, knowing that 32bits code can be used with a 64bits kernel).
Now if for some reason you really like or need 64 bits native code to run on that platform then you need to have
some extra libraries (as glibc) and other tools to create these 64 bits applications (as gcc) and this is exactly
these installation steps that Klaus explained previously. Hope this helps.
Best Regards,
Gilles

Thanks Giles. If I've understood correctly now, the optional kernel allows one to use more than 3.GB of RAM, but not to run 64-bit applications. I gather there may also be some performance loss, when running 32-bit applications on the 64-bit kernel?

[utu]

@ Otropogo

I think part of what Klaus K is saying is that for many/most Linux 'ordinary' Knoppix
users, using 64-bit applications won't show MUCH, if any performance improvement over
using the same 32-bit application. Think spread-sheet, browser, word-processor, etc.

I don't think there should necessarily be ANY performance penalty in using 32-bit
applications with a 64-bit kernel enabled.

[ruymbeke]

Hi Otropogo,
I do not see any reason to not always use a 64bits kernel if your system processor does support it.
Klaus still uses a 32bits kernel as the default because some older machine are not 64bits capable
and it is desirable to have his live CD/DVD to run by default (without tweaks) on most systems.
Best Regards,
Gilles

[ruymbeke]

Hello,
In the Knoppix forum (links below) there was some discussion about how to add
the ntfs-3g support to Knoppix 7.0.2 as it is missing from the official released DVD:
http://lists.debian.org/debian-knoppix/2012/06/msg00002.html
http://lists.debian.org/debian-knoppix/2012/06/msg00005.html
http://lists.debian.org/debian-knoppix/2012/06/msg00013.html
Please find below a link to a simple patch type solution which does not require remastering:
http://knoppix.net/forum/threads/11589-ISO-boot-from-FAT-NTFS-USB-%28GRUB.exe-grldr-from-boot.ini%29?p=126975&viewfull=1#post126975
Please note that this technique can also be used to live patch Knoppix with whatever
other feature you may need. Hope this helps. Please provide some feedback...
Best Regards,
Gilles

[ruymbeke]

Hello,
Please find below a link to a live patch to add 64 bits support (run-time + gcc) to Knoppix 7.0.2.
http://knoppix.net/forum/threads/11589-ISO-boot-from-FAT-NTFS-USB-%28GRUB.exe-grldr-from-boot.ini%29?p=127058&viewfull=1#post127058
Hope this helps and please provide some feedback...
Best Regards,
Gilles

[Capricorny]

Hi Gilles,

I have successfully tried the following with Knoppix V7.0.2.

You have to replace the packages libc-bin, libc-dev-bin, libc6, libc6-dev with those of the unstable branch.

Open a terminal an issue
sudo apt-get update
sudo apt-get -d -t unstable install libc-bin libc-dev-bin libc6 libc6-dev
sudo chvt 1
init 2
apt-get -t unstable install libc-bin libc-dev-bin libc6 libc6-dev
chroot /UNIONFS/
apt-get -t unstable -f install
exit
init 5
chvt 5 Now you can install the current packages gcc-4.6-multilib and g++-4.6-multilib from the unstable branch.

sudo apt-get -y -t unstable install gcc-4.6-multilib g++-4.6-multilib Download and extract pi_quick_start.tar.gz

Change to your pi_quick_start directory and load start.sh into a text editor. Replace all occurances of "cc" with "gcc -m64". Save the file and exit the text editor.

Set the environment variable CFLAGS:
export CFLAGS=-m64Run the script:
./start.shBest Regards

Last week this recipe worked perfectly, but now it hangs, seemingly in the check_dir() function in libc:i386.preinst. Is there any way to get around this? The script is found in /var/lib/dpkg/info/libc6:i386.preinst, but editing this doesn't seem to help.

For me, this is a very serious problem.
Here is what seems to be the culprit:



# Sanity check.
# If there are versions of glibc outside of the normal installation
# location (/lib, /lib64, etc.) then things may break very badly
# as soon as ld.so is replaced by a new version. This check is not
# foolproof, but it's pretty accurate. This script ignores libraries
# with different sonames, and libraries incompatible with the
# to-be-installed ld.so.
check_dir () {
msg=$1
dir=$2

# Follow symlinks
dir=$(readlink -e $dir || true)

# Ignore inexistent directories
if ! test -d "$dir" ; then
return
fi

# Detect possible candidates
files=$(ls $dir | egrep '^(ld|lib(d|c|m|pthread|rt|dl))-2.*.so' 2>/dev/null || true)
if test -z "$files" ; then
return
fi

for file in $files ; do
lib=$dir/$file

# Skip if it is a symlink (as installed by lsb-core)
if test -L "$lib" ; then
continue
fi

# Skip if it is the currently dynamic loader
if test "$lib" = "$ldfile" ; then
continue
fi

# See if the found libraries are compatible with the system ld.so;
# if they aren't, they'll be ignored. Check e_ident, e_type (which
# will just be ET_DYN), and e_machine. If a match is found, there
# is a risk of breakage.
libbytes=`head -c 20 $lib | od -c`
if test "$ldbytes" != "$libbytes" ; then
continue
fi

# Binaries owned packages are considered to do the right thing
# First try a quick lookup which should catch all cases on a
# normal system
if echo $libcfiles | grep -q "[ ^]$lib[ $]" ; then
continue
fi

# Slower lookup to confirm
if dpkg-query -S "$lib" >/dev/null 2>&1 ; then
continue
fi

# Output an error message and exit
echo
echo "A copy of the C library was found $msg:"
echo " '$lib'"
echo "It is not safe to upgrade the C library in this situation;"
echo "please remove that copy of the C library or get it out of"
echo "'$dir' and try again."
echo
exit 1
done
}


"A copy of the C library was found in an unexpected directory /UNIONFS/lib/libc-2.13.so..." And we are asked to remove it.



# Try to detect copies of the libc library in the various places
# the dynamic linker uses.
ldfile=$(readlink -e /lib/ld-linux.so.2)
ldbytes=$(head -c 20 /lib/ld-linux.so.2 | od -c)
libcfiles=$(dpkg-query -L $(package_name) 2>/dev/null)

dirs="/lib/i386-linux-gnu /lib /lib/tls /lib32 /lib64 /usr/local/lib /usr/local/lib32 /usr/local/lib64"
for dir in $dirs ; do
check_dir "in an unexpected directory" $dir
done

[ruymbeke]

Last week this recipe worked perfectly, but now it hangs... Is there any way to get around this ? ... For me, this is a very serious problem...

Hello Capricorny,
I don't have any problem using the script below:



#!/bin/sh
sudo apt-get update
sudo apt-get -y -d -t unstable install libc-bin libc-dev-bin libc6 libc6-dev gcc-4.6-multilib g++-4.6-multilib
sudo apt-get -y -t unstable --reinstall install libc-bin libc-dev-bin libc6 libc6-dev
sudo chroot /UNIONFS/ apt-get -y -t unstable -f install
sudo apt-get -y -t unstable --reinstall install gcc-4.6-multilib g++-4.6-multilib
After booting Knoppix 7.0.2 from the DVD and using the 64 bits kernel
I can compile and execute 64 bits libraries and applications such as Pi
cf: http://h2np.net/pi/pi_record_e.html
(after editing the start.sh script to add the "-m64" parameter on the two lines using "cc" to compile "v" and "pi")

That being said, during the install I get a very worrying error message:


apt-get -y -t unstable --reinstall install libc-bin libc-dev-bin libc6 libc6-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 4 reinstalled, 0 to remove and 1453 not upgraded.
Need to get 0 B/10.2 MB of archives.
After this operation, 0 B of additional disk space will be used.
(Reading database ... 430147 files and directories currently installed.)
Preparing to replace libc-bin 2.13-33 (using .../libc-bin_2.13-33_i386.deb) ...
Unpacking replacement libc-bin ...
Processing triggers for man-db ...
Setting up libc-bin (2.13-33) ...
(Reading database ... 430147 files and directories currently installed.)
Preparing to replace libc6:i386 2.13-33 (using .../libc6_2.13-33_i386.deb) ...

A copy of the C library was found in an unexpected directory:
'/UNIONFS/lib/i386-linux-gnu/libc-2.13.so'
It is not safe to upgrade the C library in this situation;
please remove that copy of the C library or get it out of
'/UNIONFS/lib/i386-linux-gnu' and try again.

dpkg: error processing /var/cache/apt/archives/libc6_2.13-33_i386.deb (--unpack):
subprocess new pre-installation script returned error exit status 1
Errors were encountered while processing:
/var/cache/apt/archives/libc6_2.13-33_i386.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
==> dpkg: error processing /var/cache/apt/archives/libc6_2.13-33_i386.deb (--unpack): <==
Is it possible that the "libc6_2.13-33_i386.deb" package be corrupted on the Debian repository ?
Even with that error, it looks like that I can still compile and execute 64 bits code...

BTW, did you try my patch yet ?
Best Regards,
Gilles

[Capricorny]

Hello Capricorny,
I don't have any problem using the script below:



#!/bin/sh
sudo apt-get update
sudo apt-get -y -d -t unstable install libc-bin libc-dev-bin libc6 libc6-dev gcc-4.6-multilib g++-4.6-multilib
sudo apt-get -y -t unstable --reinstall install libc-bin libc-dev-bin libc6 libc6-dev
sudo chroot /UNIONFS/ apt-get -y -t unstable -f install
sudo apt-get -y -t unstable --reinstall install gcc-4.6-multilib g++-4.6-multilib
After booting Knoppix 7.0.2 from the DVD and using the 64 bits kernel
I can compile and execute 64 bits libraries and applications such as Pi
cf: http://h2np.net/pi/pi_record_e.html
(after editing the start.sh script to add the "-m64" parameter on the two lines using "cc" to compile "v" and "pi")

That being said, during the install I get a very worrying error message:


apt-get -y -t unstable --reinstall install libc-bin libc-dev-bin libc6 libc6-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 4 reinstalled, 0 to remove and 1453 not upgraded.
Need to get 0 B/10.2 MB of archives.
After this operation, 0 B of additional disk space will be used.
(Reading database ... 430147 files and directories currently installed.)
Preparing to replace libc-bin 2.13-33 (using .../libc-bin_2.13-33_i386.deb) ...
Unpacking replacement libc-bin ...
Processing triggers for man-db ...
Setting up libc-bin (2.13-33) ...
(Reading database ... 430147 files and directories currently installed.)
Preparing to replace libc6:i386 2.13-33 (using .../libc6_2.13-33_i386.deb) ...

A copy of the C library was found in an unexpected directory:
'/UNIONFS/lib/i386-linux-gnu/libc-2.13.so'
It is not safe to upgrade the C library in this situation;
please remove that copy of the C library or get it out of
'/UNIONFS/lib/i386-linux-gnu' and try again.

dpkg: error processing /var/cache/apt/archives/libc6_2.13-33_i386.deb (--unpack):
subprocess new pre-installation script returned error exit status 1
Errors were encountered while processing:
/var/cache/apt/archives/libc6_2.13-33_i386.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
==> dpkg: error processing /var/cache/apt/archives/libc6_2.13-33_i386.deb (--unpack): <==
Is it possible that the "libc6_2.13-33_i386.deb" package be corrupted on the Debian repository ?
Even with that error, it looks like that I can still compile and execute 64 bits code...

BTW, did you try my patch yet ?
Best Regards,
Gilles

Thank you very much for your response, Gilles!

First, my current working version is remastering #4, so I havent tried out your patch. As told, I also already did this successfully, but in another "branch", which has been overwritten.

The important observation for me is that you get precisely the same error message as me, effective halting the upgrading of libc6, leaving a "broken" package from apt's point of view. And I pointed out the approximate location of the error in the preinst script -it's corrupted logic, not corrupted files. The ordinary upgrade of libc6 stable also crashes the same way.

To fix the problem, we must have access to and patch the offending downloaded .deb packages. This may be accomplished in a number of ways, right now I tend to support the more drastical ones, to be able to stop Debian apt madness on the spot when needed. :)

[Capricorny]

To fix the problem, we must have access to and patch the offending downloaded .deb packages. This may be accomplished in a number of ways, right now I tend to support the more drastical ones, to be able to stop Debian apt madness on the spot when needed. :)

Here is a general solution to the problem, found on github:gist, https://gist.github.com/1729559

It amounts to running a script on the troublesome downloaded .deb package, to remove the associated pre/postrm scripts. I think it should be fairly safe to use, and in any case far safer than the default dive-into-it Debian "method", which may leave us with a broken system even if everything is correct, because the scripts are buggy - as they very often tend to be.

The script is called stripdeb.sh, and I installed it in /usr/local/bin:


#!/bin/bash
# Usage: stripdeb.sh something.deb
# From: https://gist.github.com/1729559
# Patch /etc/apt.conf.d/70debconf: DPkg::Pre-Install-Pkgs {"xargs -rL1 bash /usr/local/bin/stripdeb.sh 2>&1 | logger -t stripdeb"}
# 20120624 tay: This patching didn't seem to work, and using on a regular basis may not be that smart either.
# Safer to run it only on individual packages basis:
# #> sudo stripdeb.sh /var/cache/apt/archives/libc6_2.13-33_i386.deb
# Don't use apt-get afterwards, it may notice something is changed and download the old madness again
# #> dpkg -i /var/cache/apt/archives/libc6_2.13-33_i386.deb

echo "Args: $*"
echo "Stripping $1 of pre/post maintainer scripts"
tmpdir=$(mktemp -d)

[ ! -f $1 ] && exit 1

genldconfigscript() {
# Maybe we should just make the always run 'ldconfig'
cat << EOF
#!/bin/sh
[ "\$1" = "configure" ] && ldconfig
[ "\$1" = "remove" ] && ldconfig
true
EOF
}

# The .deb is an 'ar' archive, grab the control files.
ar -p $1 control.tar.gz | tar -C $tmpdir -zxf -

# Kill the stupid package scripts, but log what we do.
for i in $tmpdir/{post,pre}{rm,inst} ; do
if [ -f $i ] ; then

# Linux sucks, so we have to run ldconfig on any library changes.
# So if the post/pre script includes ldconfig
if grep -q ldconfig $i ; then
echo "$1: Replacing $i with a generic 'ldconfig' script"
genldconfigscript > $i
chmod 755 $i
else
echo "$1: Stripping $(basename $i)"
rm $i
fi
fi
done

# Rebuild the control tarball
tar -C $tmpdir -zcf control.tar.gz .

# And replace the old one with the stripped one back into the .deb
ar -r $1 control.tar.gz

# Clean up
rm control.tar.gz
So, to get around the Debian package bugs, I ran the script before chvt'ing, and used dpkg instead of apt-get after init 2. The modified version of Klaus2008's recipe thus becomes:



sudo apt-get update
sudo apt-get -d -t unstable install libc-bin libc-dev-bin libc6 libc6-dev
sudo stripdb.sh /var/cache/apt/archives/libc6_2.13-33_i386.deb

sudo chvt 1
init 2
dpkg -i libc-bin libc-dev-bin libc6 libc6-dev

chroot /UNIONFS/
apt-get -t unstable -f install
exit
init 5
chvt 5

sudo apt-get -y -t unstable install gcc-4.6-multilib g++-4.6-multilib

It worked for me, and with due modifications it ought to work more generally.
We must of course be pretty sure that the scripts aren't really needed for the particular package.

[ruymbeke]

Hi Capricorny,
Thank you very much for offering a fix proposal. I tried it and I found two problems:


sudo stripdb.sh /var/cache/apt/archives/libc6_2.13-33_i386.deb
The first one is a typo in your modified Klaus2008's recipe: stripdb.sh ==> stripdeb.sh
(to match the location of the /usr/local/bin/stripdeb.sh script)


dpkg -i libc-bin libc-dev-bin libc6 libc6-dev
The second: dpkg -i complains about not finding the archive, please see the log below:


dpkg -i libc-bin libc-dev-bin libc6 libc6-dev
dpkg: error processing libc-bin (--install):
cannot access archive: No such file or directory
dpkg: error processing libc-dev-bin (--install):
cannot access archive: No such file or directory
dpkg: error processing libc6 (--install):
cannot access archive: No such file or directory
dpkg: error processing libc6-dev (--install):
cannot access archive: No such file or directory
Errors were encountered while processing:
libc-bin
libc-dev-bin
libc6
libc6-dev
Best Regards,
Gilles

[Capricorny]

Right you are, Gilles!
This reporting was very quickly done, a bit too fast.
What I actually ran was



sudo apt-get update
sudo apt-get -d -t unstable install libc-bin libc-dev-bin libc6 libc6-dev
sudo stripdeb.sh /var/cache/apt/archives/libc6_2.13-33_i386.deb

sudo chvt 1
init 2
# The other libs were correctly installed already, next line implicit - didn't run it
# sudo apt-get -y -t unstable --reinstall install libc-bin libc-dev-bin libc6-dev

# Give dpkg exact file address
dpkg -i /var/cache/apt/archives/libc6_2.13-33_i386.deb

chroot /UNIONFS/
apt-get -t unstable -f install
exit

init 5
chvt 5

sudo apt-get -y -t unstable install gcc-4.6-multilib g++-4.6-multilib


But I wasn't able to copy from the chvt 1 commands, thus only edited the line from the recipe. apt-get may possibly be used, but told not to download again? It just downloaded the unmodified .deb libc6 over the modified one for me.
The most robust thing to do might be to adjust the dpkg command.

PS: The autoformatting here REALLY SUCKS!

[Capricorny]

After another round with the recipe (I maintain squashfs and cloop remasterings separately), I 'm pretty sure that this way of handling install problems will normally be OK when there are no underlying problems. But, turning off all scripts is a rather drastical measure, and a more careful way of handling it would be to patch the offending script(s). Which I think in this case would be quite simple, the main problem is getting access to it. Here, the technique in the stripdeb.sh script could be useful.

For example, instead of removing the script, a patch could be applied to it. In the actual case here, inserting a simple "continue" statement could be enough. In other cases I have come across, running checks on non-used (but installed by default) components like new grub have lead to problems, and the irrelevant checks should be omitted.

[ruymbeke]

Hello,
Please find below my latest script working with the latest Debian repository changes (using gcc 4.7.1):

sudo apt-get update
sudo apt-get -y -d -t unstable install libc-bin libc-dev-bin libc6 libc6-dev gcc-4.6-multilib g++-4.6-multilib
sudo apt-get -y -t unstable --reinstall install libc-bin
sudo stripdeb.sh /var/cache/apt/archives/libc6_2.13-33_i386.deb
sudo dpkg -i /var/cache/apt/archives/libc6_2.13-33_i386.deb
sudo apt-get -y -t unstable --reinstall install libc-dev-bin libc6-dev
sudo chroot /UNIONFS/ apt-get -y -t unstable -f install
sudo apt-get -y -t unstable --reinstall install gcc-4.6-multilib g++-4.6-multilib
It is using the /usr/local/bin/stripdeb.sh script from Capricorny, see above:
http://knoppix.net/forum/threads/29878-Knoppix-7.0.2-EN-DVD-notes-comments?p=127069&viewfull=1#post127069
Best Regards,
Gilles

[ruymbeke]

Hello,
If you are interested you can find the latest cloop patch files in here:
http://knoppix.net/forum/threads/11589-ISO-boot-from-FAT-NTFS-USB-%28GRUB.exe-grldr-from-boot.ini%29?p=127086&viewfull=1#post127086
Best Regards,
Gilles