HTTP-FUSE KNOPPIX

[suzaki]

Dear,

We developed HTTP-FUSE KNOPPIX which gets pieces of cloop file form a HTTP server and re-configure virtual cloop file with HTTP-FUSE driver. HTTP-FUSE KNOPPIX is small bootimage(6MB) and downloads necessary pieces of cloop files on demand. HTTP-FUSE KNOPPIX can save the downloaded pieces and reuse them at next boot.
The detail is shown the following URL and you can download the HTTP-FUSE KNOPPIX.
http://unit.aist.go.jp/itri/knoppix/http-fuse/index-en.html

Following figures (Figure1 and 2) show difference of normal KNOPPIX and HTTP-FUSE KNOPPIX.


Figure 1: normal CD KNOPPIX
http://unit.aist.go.jp/itri/knoppix/http-fuse/image2_s.gif


Figure 2: HTTP-FUSE KNOPPIX
http://unit.aist.go.jp/itri/knoppix/http-fuse/image1_s.gif


HTTP-FUSE KNOPPIX is developed by a bitter lesson of SFS-KNOPPIX. SFS-KNOPPIX lacks extensibility and suffers network latency. HTTP-FUSE KNOPPIX solves these problems.
http://unit.aist.go.jp/itri/knoppix/sfs/index-en.html
http://www.knoppix.net/forum/viewtopic.php?t=14904

HTTP-FUSE KNOPIX uses HTTP to get pieces of cloop image and can reuse them. HTTP-FUSE KNOPPIX can use mirror servers and proxies to distribute the pieces. Furthermore HTTP-FUSE KNOPPIX has a function to select the nearest site by "netselect".

http://unit.aist.go.jp/itri/knoppix/http-fuse/image9.jpg



HTTP-FUSE KNOPPIX is based on FUSE(Filesytem in USERspace).
http://fuse.sourceforge.net
If you are interested in the implementation, please visit the following HP.
http://unit.aist.go.jp/itri/knoppix/http-fuse/index-en.html#detail
http://unit.aist.go.jp/itri/knoppix/http-fuse/image5_s.gif

Have fun, HTTP-FUSE KNOPPIX!

[lilsirecho]

Suzaki;

Please advise the correct format for static IP:

I tried ....Knoppix26 lang=us pdt nodhcp "IPaddress:", xxx.xxx.xxx.xxx "Netmask:", xxx.xxx.xxx.xxx."DefaultGateway:", xxx.xxx.xxx.xxx "NameServer:", xxx.xxx.xxx.xxx

It did not provide a static IP and stopped after 3 tries at dhcp.

The description was given in your data and I followed it exactly...an example would be best description.

The description also leaves out the Kernel version...knoppix26.

[tyagi]

Suzaki;

Please advise the correct format for static IP:

I tried ....Knoppix26 lang=us pdt nodhcp "IPaddress:", xxx.xxx.xxx.xxx "Netmask:", xxx.xxx.xxx.xxx."DefaultGateway:", xxx.xxx.xxx.xxx "NameServer:", xxx.xxx.xxx.xxx

It did not provide a static IP and stopped after 3 tries at dhcp.

The description was given in your data and I followed it exactly...an example would be best description.

The description also leaves out the Kernel version...knoppix26.

try
knoppix26 lang=us pdt staticipaddress
at boot prompt.
after boot, there appears
IP address:
prompt, and input your IP address here. and following
Netmask:
Default Gateway:
Name Server:
prompt, you can input yours.

regards.

[lilsirecho]

suzaki;

Entered the staticaddress boot option and was able to enter the data but had to change to normal keyboard....no usb keyboard control.

Selected data-hotel Knoppix37EDU....waiting until fs wrapper is done........done

Then the system hangs with blinking cursor.

Do I have to make an entry at this point?

[suzaki]

lilsirecho,



Selected data-hotel Knoppix37EDU....waiting until fs wrapper is done........done
Then the system hangs with blinking cursor.
Do I have to make an entry at this point?

Please choose "NETSELECT" at second menu. "NETSELECT" finds automatically the nearest site for your PC.
And please tell me the delay of NETSELECT. IF the delay is more than 1000, it takes more than 15 sec to boot KNOPPIX.

Can you save the downloaded split-and-compressed blcok files on your PC?
After second boot, HTTP-FUSE KNOPPIX uses the downloaded block files.

[UnderScore]

I tried HTTP-FUSE KNOPPIX and found that the delay is high. Netselect shows the delay ranging from as low as 390 to as high as 1081. I was not able to get HTTP-FUSE KNOPPIX to fully load. It stopped after "setting paths" and I waited 5 minutes and it did not progress. I though it might have lost the connection. I used a separate PC with Knoppix and ethereal to inspect the packets and found that HTTP-FUSE KNOPPIX was still downloading the compressed blocks. I decided to shut the PC off since I no longer wanted to wait.

I will try again tonight and will wait the full amount of time for the system to boot.

I would like to setup a local http mirror of the HTTP-FUSE KNOPPIX compressed blocks. Is there a HOWTO or a guide to do so? Can I rsync from a HTTP-FUSE KNOPPIX mirror to my local network?

Thanks for your time,
James

[lilsirecho]

Suzaki;

I used NETSELECT for this trial...

The result was the display of all possible "ring" sources and the delay incurred for each as follows:


ring so net 9999
tains tohaku 9999
toyama/arch 9999
toyama-u/arch 23036
data-hotel 11622
nihon-u 17249
sakura 19084
aist.go 22324
aist nyud 8090 3738

When I selected data-hotel in the first trial reported in previous post, the delay was 312.

That trial was at 10pm pdt
The listings in this post occurred at 9:20am this day, Thursday morning.

The same...... waiting until fs wrapper is ready came on screen and never finished...the screen went blank...no further activity..reset computer.

[lilsirecho]

Repeated the trial again after report of ISP delays this morning.

The netselect values are as shown below in the order given for the ring sources in your listing:


461
388
372
505
351
435
397
762
275

The system hung in blinking cursor again after fs_wrapper reported "done".

I am not certain that the ISP has the system back up to speed since he will be doing maintenance at 3am tomorrow.

Still have no Knoppix running with FUSE-KNOPPIX...guess I blew a fuse!!!!!!

[suzaki]

UnderSocre and lilsirecho,

I prepare block files for your local HTTP server to check HTTP-FUSE KNOPPIX.

http://unit.aist.go.jp/itri/knoppix/knxblock/knxblock.tar
MD5: 5bed82f71f1d594d6ac6c81cb320f0f1 knxblock.tar
The size is 1.5GB. It includes images for KNOPPIX37JP, KNOPPIX37Edu, KNOPPIX37Math and KNOPPIX37FSFE.
Same block files are shared.

Please extract the tar file and setup to the "knxbock" directory on your HTTP server.
And boot HTTP-FUSE KNOPPIX with the "fuse_uri=" option. For exmaple,

boot: knoppix26 utc nodhcp fuse_uri=http://SERVER-NAME/SERVER-DIRECTORY/knxblock/KNOPPIX37FSFE/

Please add "utc" and "nodhcp" option as magic word. They keep connection to HTTP server.

If possible, plese open your server for other users, because HTTP FUSE KNOPPIX suffers network latency strongly.

P.S.
Does anyone have an idea to distribute block files world wide?
Can we use Google cache to distribute the block files?

[lilsirecho]

Discovered that my download was only 5MB whereas your listing says it is 6MB.

Tried to re-download to obtain the 6MB download ...after deleting the .iso file previously downloaded.

I am unable to receive the full-size download of FUSE-KNOPPIX by clicking on the link. It always downloads 1MB only (probably the 1MB I was missing!!!

Please advise as to corrective action to get a correct download.

[suzaki]

Sorry for my late reply.

We made sets of block files for HTTP-FUSE KNOPPIX.
Full sets include all block files. They are split-and-compressed block files of cloop. The volumes are around 700MB.
Minimum sets include block files for boot. The volumes are are 150MB - 200MB. If you use a minimum set file, you have to download rest block files from HTTP server after boot.

Please download a compressed file and extract all block files at a top direcoty of a device. For example,
# cd /tmp
# wget http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37FSFE.zip
Move to the top directory of a device and extract the download file. For example,
# cd /mnt/sda1/
# unzip /tmp/KNOPPIX37FSFE.zip
You will find "knxblock" directory.
Afther that, re-boot with HTTP-FUSE KNOPPIX with magic option "maxconnection=0" option. For example,
boot: knoppix26 utc nodhcp maxconnection=0

At selection menu, please select downloaded set file name. HTTP-FUSE KNOPPIX finds the block files and use them.


KNOPPIX FSF-Edurope full-set 707MB, minimum-set for boot 157MB
http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37FSFE.zip
http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37FSFE-dl.zip

KNOPPIX Japanese full-set 633MB, minimum-set for boot 158MB
http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37JP.zip
http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37JP-dl.zip

KNOPPIX Education full-set 755MB, minimum-set for boot 205MB
http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37Edu.zip
http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37Edu-dl.zip

KNOPPIX Education full-set 707MB, minimum-set for boot 223MB
http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37Math.zip
http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37Math-dl.zip


You can setup a HTTP service with the full-set of block files.
Extract a full-set file on your HTTP server and boot HTTP-FUSE KNOPPIX on other machine with the "fuse_uri=" option. For exmaple,

boot: knoppix26 utc nodhcp fuse_uri=http://SERVER-NAME/SERVER-DIRECTORY/knxblock/KNOPPIX37FSFE/

Have fun, I hope. :-)

[heinzen]

Hello suzaki,


Sorry for my late reply.

We made sets of block files for HTTP-FUSE KNOPPIX.
Full sets include all block files. They are split-and-compressed block files of cloop. The volumes are around 700MB.
Minimum sets include block files for boot. The volumes are are 150MB - 200MB. If you use a minimum set file, you have to download rest block files from HTTP server after boot.

Please download a compressed file and extract all block files at a top direcoty of a device. For example,
# cd /tmp
# wget http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37FSFE.zip
Move to the top directory of a device and extract the download file. For example,
# cd /mnt/sda1/
# unzip /tmp/KNOPPIX37FSFE.zip
You will find "knxblock" directory.
Afther that, re-boot with HTTP-FUSE KNOPPIX with magic option "maxconnection=0" option. For example,
boot: knoppix26 utc nodhcp maxconnection=0

At selection menu, please select downloaded set file name. HTTP-FUSE KNOPPIX finds the block files and use them.


KNOPPIX FSF-Edurope full-set 707MB, minimum-set for boot 157MB
http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37FSFE.zip
http://unit.aist.go.jp/itri/knoppix/knxblock/KNOPPIX37FSFE-dl.zip

...

You can setup a HTTP service with the full-set of block files.
Extract a full-set file on your HTTP server and boot HTTP-FUSE KNOPPIX on other machine with the "fuse_uri=" option. For exmaple,

boot: knoppix26 utc nodhcp fuse_uri=http://SERVER-NAME/SERVER-DIRECTORY/knxblock/KNOPPIX37FSFE/

Have fun, I hope. :-)

yes i have tested it and it works great :D.

But after analysing the http-fuse-knoppix-20050408-EN.iso there are still 2 questions:
1. i have found out that the magic is done by fs_wrapper, where can i download the sources
2. how are the " split-and-compressed block files" built.

Thanks for work.

[suzaki]

yes i have tested it and it works great :D.

But after analysing the http-fuse-knoppix-20050408-EN.iso there are still 2 questions:
1. i have found out that the magic is done by fs_wrapper, where can i download the sources
2. how are the " split-and-compressed block files" built.

Thanks for work.

Thanks for your trial.

The source code of fs_wrapper is opened at the following URL.
http://unit.aist.go.jp/itri/knoppix/http-fuse/fs_wrapper.0.36.tar.gz
The source code to make " split-and-compressed block files" is also opened.
http://unit.aist.go.jp/itri/knoppix/http-fuse/mkmd5files.tar.gz

They aren't well organized, because they are under construction. Please use them on your risk and have fun!

[suzaki]

We released 4 Debian Packages for "HTTP-FUSE". They are effective on KNOPPIX 3.9 using UNIONFS.
http://unit.aist.go.jp/itri/knoppix/http-fuse/libfuse2_2.2.1-4_i386.deb
http://unit.aist.go.jp/itri/knoppix/http-fuse/fuse-utils_2.2.1-4_i386.deb
http://unit.aist.go.jp/itri/knoppix/http-fuse/fuse-module-2.6.11_2.2.1-4_i386.deb
http://unit.aist.go.jp/itri/knoppix/http-fuse/fs-wrapper_0.1-2_i386.deb

Orignal Home Page: http://unit.aist.go.jp/itri/knoppix/http-fuse/index-en.html

Usage:
1. Change to ROOT user by "su" command. Download the debian packages.
2. Install Debian Packages. (The order of package means the dependency relation.)
# dpkg -i libfuse2_2.2.1-4_i386.deb fuse-utils_2.2.1-4_i386.deb fuse-module-2.6.11_2.2.1-4_i386.deb fs-wrapper_0.1-2_i386.deb
3. fuse-utils requires "group name" for http-fuse mount. Please designate "root".
http://unit.aist.go.jp/itri/knoppix/http-fuse/fuse-utils-q.gif
4. Make a directory for http-fuse.
# mkdir /tmp/fuse
5. Run "fuse-wrapper" with a directory and URL of HTTP-FUSE. For example,
# fs_wrapper /tmp/fuse -f http://aist.ring.gr.jp/archives/linux/knoppix/knxblock/KNOPPIX37FSFE/
This process tell us log of fs-wrapper.
6. Please open another terminal. You find a cloop file at "/tmp/fuse/knoppix". Please setup loopback mount.
# losetup /dev/cloop1 /tmp/fuse/KNOPPIX
# mkdir /mnt/http-fuse
# mount /dev/cloop1 /mnt/http-fuse
You use files at "/mnt/http-fuse/". When you touch a files under /mnt/http-fuse, the blocks are downloaded at "/tmp/blocks/[00-ff]/".

mkmd5files makes "split-and-compressed block files" for HTTP-FUSE from a block device.
Usage: mkmd5files <block device> <target directory for split-and-block files> <split-block size>
You try on KNOPPIX CD as following.
# mkmd5files /dev/cloop /tmp/http-fuse/ 262144
256KB spit-and-block files are created under /tmp/http-fuse/. The file space requires approximately 700MB.
The split-and-block files are opened by HTTP server and can be mounted by "fs_wrapper".