If you don't want to install to the hard drive, you can install to any persistent directory that has enough room. Personally, I favor USB pen drives mounted as a persistent home for this sort of thing so that I can port things around with me.

The previous poster's example of how to set the configure prefix is excellent. Keep in mind, though, that not every application will compile cleanly on Knoppix, although my luck so far has been excellent in that regard.