Page 1 of 10 123 ... LastLast
Results 1 to 10 of 93

Thread: Using Knoppix to make a Linux cluster on your home network

  1. #1
    Senior Member registered user
    Join Date
    Nov 2002
    Posts
    1,353

    Using Knoppix to make a Linux cluster on your home network

    I recently read about clusterknoppix on another thread in the forums. For those interested in an easy way to use set up a Linux cluster at home this is a great tool. It is really not hard. It also makes use of the terminal server so you can set up one machine as the server and one as the client and share resources.

    I plan on doing a hdinstall with clusterknoppix this weekend and see if I can set this up in a more permanent way. I'll post more details back later, but it should work fine and it will make administrating a lot easier.

    This is just very, very cool. Tell this to Steve Balmer and Bill Gates who keep saying that Linux doesn't innovate.

  2. #2

    Re: Using Knoppix to make a Linux cluster on your home netwo

    Quote Originally Posted by aay
    I recently read about clusterknoppix on another thread in the forums. For those interested in an easy way to use set up a Linux cluster at home this is a great tool. It is really not hard. It also makes use of the terminal server so you can set up one machine as the server and one as the client and share resources.

    I plan on doing a hdinstall with clusterknoppix this weekend and see if I can set this up in a more permanent way. I'll post more details back later, but it should work fine and it will make administrating a lot easier.

    This is just very, very cool. Tell this to Steve Balmer and Bill Gates who keep saying that Linux doesn't innovate.
    What can and are you use/using clusterknoppix to do?

    other than the fact that it is cool why would we do this?

  3. #3
    Senior Member registered user
    Join Date
    Nov 2002
    Posts
    1,353
    I'm not using it to do any serious cpu crunching, but the nice thing about clusterknoppix is that you can use it in conjunction with knoppix terminal server. I have a 1gig athlon and a 450 amd K6 and the 450 machine works significantly faster when connected to the 1gig athlon machine as a clusterknoppix client than it does by itself.

    I was also pleasantly surprised to see that hardware that I expected to have problems on the client didn't: namely, the sound card. Playing mp3's over the internet worked fine on the client. I even popped in a compressed movie burned onto a cd and it worked. So did my tv tuner card. This is a real sweet way to maximize older hardware through the use of the terminal server. You get the added benefit of pooled cpu, and administration is easier because if you make updates to the server then all the clients are immediately updated as well. I think I'm going to really like this.

  4. #4
    Senior Member registered user
    Join Date
    Nov 2002
    Location
    Long Island, NY, USA
    Posts
    1,256
    OK, this sounds promissing. So what you're telling us is that clusterKNOPPIX will automatically distribute the work load over all the hardware on the cluster. You don't have to use any fancy compilers or write any fancy code for, say, massively parallel architectures or pipelined thruput, right? It just works?? This is VERY astounding!!!

    Now, I'm installing a new network for my Computer Science students next year. I was thinking of using the thin-client tip elsewhere in the forum, but clustering has potential. OK, so what if I cluster 25 Pentium IV PC's, each running at 2Ghz with 512MB RAM and 100GB hdds. Will this run, say, one process as if it were a single 50Ghz machine with 12.5GB RAM and 2.5TB storage space????

    TIA,

  5. #5
    Senior Member registered user
    Join Date
    Nov 2002
    Posts
    1,353
    Well it looks like a clusterknoppix hdinstall is a work in progress. Currently ext2 is the only fs that will work for hdinstalls. I haven't yet been able to get clients up and running. I think that the /etc/init.d/knoppix-terminalopenmosixserver script needs some hacking for hdinstalls. I've done a bit of this myself but no luck yet. I've been corresponding with ClusterKnoppix's creator, Wim Vandersmissen, about this stuff. He has updated the documentation on his site for hdinstalls and added some forums so I welcome people interested in this to visit his site (listed above).

    I'm pretty sure this will work. There just need to be some changes made from the cd version to the hd version of the script.

  6. #6
    Senior Member registered user
    Join Date
    Nov 2002
    Posts
    1,353
    Quote Originally Posted by A. Jorge Garcia
    Now, I'm installing a new network for my Computer Science students next year. I was thinking of using the thin-client tip elsewhere in the forum, but clustering has potential. OK, so what if I cluster 25 Pentium IV PC's, each running at 2Ghz with 512MB RAM and 100GB hdds. Will this run, say, one process as if it were a single 50Ghz machine with 12.5GB RAM and 2.5TB storage space????

    TIA,
    Hehe! That would be pretty nice wouldn't it. Storage space will not be distributed. RAM will not be distributed. Processes will be distributed.

    Here's a quote from the openmosix homepage.

    openMosix is a Linux kernel extension for single-system image clustering. This kernel extension turns a network of ordinary IA-32 computers into a supercomputer for Linux applications.

    Once you have installed openMosix, the nodes in the cluster start talking to one another and the cluster adapts itself to the workload. Processes originating from any one node, if that node is too busy compared to others, can migrate to any other node. openMosix continuously attempts to optimize the resource allocation.

    We achieve this with a kernel patch for Linux, creating a reliable, fast and cost-efficient SSI clustering platform that is linearly scalable and adaptive. With openMosix’ Auto Discovery, a new node can be added while the cluster is running and the cluster will automatically begin to use the new resources.

    There is no need to program applications specifically for openMosix. Since all openMosix extensions are inside the kernel, every Linux application automatically and transparently benefits from the distributed computing concept of openMosix. The cluster behaves much as does a Symmetric Multi-Processor, but this solution scales to well over a thousand nodes which can themselves be SMPs.
    Sounds pretty nice eh?

    Sounds like an openmosix cluster would greatly benifit your classroom project.

    Here are a couple of other links that I found that might be helpful to you. They combine openmosix with the Linux Terminal Server Project.

    http://www.lpmo.edu/~daniau/ltsp-mosix/
    http://openmosix.sourceforge.net/ltsp-omr4-1.html

    If you eventually decide to do something like this, let us know how it goes.

  7. #7
    Senior Member registered user
    Join Date
    Mar 2003
    Posts
    298
    btw, don't forget Quantian (url1 | url2) (formerly Quantix). It has the same purpose as ClusterKnoppix, don't know if they are compatible, though. It's Knoppix distro with a bunch freeware 'big math' programs.

    btw, does anybody know what happens when a node fails (aka someone presses a reset button)? I hope all transported threads are just halted ('stopped') on the originating machine, so they can be resumed when the 'answer' is lost.

    [Edit]Okay, found it. You might want to disable the reset keys to be sure:
    From the OpenMosix FAQ

    What happens to a job if its node fails?:

    If a node crashes all jobs started from that node and all jobs which are currently running on that node due to migration are lost, just as on any ordinary computer.

    If you halt a node gracefully (halt, shutdown, or poweroff commands), remote jobs will be sent back to the node where they were initiated (they may then migrate to other running nodes), but obviously all locally initiated jobs should be ended before issuing the halt. (Disconnecting the power on a running machine is never a good idea.)[/Edit]

  8. #8
    Senior Member registered user
    Join Date
    Nov 2002
    Posts
    1,353
    Quote Originally Posted by Henk Poley
    What happens to a job if its node fails?:

    ...all jobs started from that node and all jobs which are currently running on that node due to migration are lost...
    Deffinitely something to remember!!!

  9. #9
    Senior Member registered user
    Join Date
    Mar 2003
    Posts
    298
    Quote Originally Posted by aay
    Quote Originally Posted by Henk Poley
    What happens to a job if its node fails?:

    ...all jobs started from that node and all jobs which are currently running on that node due to migration are lost...
    Deffinitely something to remember!!!
    I guess that keeping the thread 'stopped' on the originating machine gives all kinds of (time-)dependency problems. But still...

    btw, if you open your case you can just unplug the reset switch. And with an ATX case you might want to disable the '5s power-button pressed is reboot' in the BIOS. Anyways this could be a really bummer, you need to be sure all your nodes are rock stable. One whackling NIC contact brings down all machines, or at least the ones which sent out processes to this machine.

    I don't know in howfar the Linux kernel supports the ATX specs. But you might be able to 'bend' the reset button to 'reboot'.

    remember, users can be lusers...

  10. #10
    Senior Member registered user
    Join Date
    Jan 2003
    Location
    Podunk, Idaho, USA
    Posts
    470
    What I did yesterday was a clusterKnoppix hd install and found out that it would only act as the openMosix terminal server when running from cd. So I went to the openMosix Howto and did a bit of reading and created the proper /etc/openmosix.map file (or /etc/hpc.map if you have problems look for the /etc/hpc.map and either edit it or get rid of it) and got openMosix to startup on that machine. Then I got the openMosix 2.4.20 kernel image from the clusterKnoppix site and put it on a Morphix hd install, then also did an apt-get for openmosix (the tools) and openmosixview. I put the same /etc/openmosix.map file on that machine, started openmosix (/etc/init.d/openmosix start) and I had a two machine cluster up and running, that easy!

    Check out this page for apps that migrate smoothly and this one for apps that DoNot migrate under openMosix.

    HTH

    rock

Page 1 of 10 123 ... LastLast

Similar Threads

  1. Spaces in passwords (make permanent /home image)
    By eris_pluvia in forum Hdd Install / Debian / Apt
    Replies: 0
    Last Post: 10-19-2004, 03:15 PM
  2. Knoppix in a small home network
    By YannLeMerle in forum General Support
    Replies: 0
    Last Post: 03-07-2004, 04:33 PM
  3. Replies: 1
    Last Post: 02-17-2004, 02:30 PM
  4. Knoppix linux remastered for the home ... ESSENTIAL
    By law_student in forum Customising & Remastering
    Replies: 12
    Last Post: 06-16-2003, 11:52 PM
  5. How to make my extra /home partition reiserfs
    By anibal in forum Hdd Install / Debian / Apt
    Replies: 5
    Last Post: 06-06-2003, 05:35 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


A-Tech 8GB DDR3 1600 PC3-12800 Laptop SODIMM 204-Pin Memory RAM PC3L DDR3L 1x 8G picture

A-Tech 8GB DDR3 1600 PC3-12800 Laptop SODIMM 204-Pin Memory RAM PC3L DDR3L 1x 8G

$13.99



HyperX FURY RAM DDR4 16GB 8GB 32GB 4GB 3200 2666 2400 2133 Desktop Memory DIMM picture

HyperX FURY RAM DDR4 16GB 8GB 32GB 4GB 3200 2666 2400 2133 Desktop Memory DIMM

$9.64



A-Tech 16GB 2 x 8GB PC3-12800 Laptop SODIMM DDR3 1600 Memory RAM PC3L 16G DDR3L picture

A-Tech 16GB 2 x 8GB PC3-12800 Laptop SODIMM DDR3 1600 Memory RAM PC3L 16G DDR3L

$27.98



CRUCIAL DDR3L 8GB 16GB 32GB 1600 MHz PC3-12800 Laptop Memory RAM SODIMM 204-Pin picture

CRUCIAL DDR3L 8GB 16GB 32GB 1600 MHz PC3-12800 Laptop Memory RAM SODIMM 204-Pin

$14.35



A-Tech 8GB PC3-12800 Desktop DDR3 1600 MHz Non ECC 240-Pin DIMM Memory RAM 1x 8G picture

A-Tech 8GB PC3-12800 Desktop DDR3 1600 MHz Non ECC 240-Pin DIMM Memory RAM 1x 8G

$13.99



HMT84GL7AMR4C-RD 32GB DDR3 Server Memory RAM 14900L ECC REG 4Rx4 SK Hynix Cisco picture

HMT84GL7AMR4C-RD 32GB DDR3 Server Memory RAM 14900L ECC REG 4Rx4 SK Hynix Cisco

$13.99



Crucial Pro 64GB (2 x 32GB) PC4-25600 DDR4-3200 Memory - CP2K32G4DFRA32A picture

Crucial Pro 64GB (2 x 32GB) PC4-25600 DDR4-3200 Memory - CP2K32G4DFRA32A

$90.00



A-Tech 16GB 2x 8GB PC3-12800 Desktop DDR3 1600 MHz 240pin DIMM Memory RAM 16G 8G picture

A-Tech 16GB 2x 8GB PC3-12800 Desktop DDR3 1600 MHz 240pin DIMM Memory RAM 16G 8G

$27.98



Hynix 64GB 4Rx4 PC4-2133P-L LRDIMM DDR4-17000 ECC Load Reduced Server Memory RAM picture

Hynix 64GB 4Rx4 PC4-2133P-L LRDIMM DDR4-17000 ECC Load Reduced Server Memory RAM

$64.99



A-Tech 64GB 4x 16GB 2Rx4 PC4-19200R DDR4 2400MHz ECC REG RDIMM Server Memory RAM picture

A-Tech 64GB 4x 16GB 2Rx4 PC4-19200R DDR4 2400MHz ECC REG RDIMM Server Memory RAM

$87.96