Thanks for replying!
I already thought this could be the problem, so I downloaded the kernel-source-2.4.20-xfs and put it in /usr/src/
Then I edited .config and pointed the KERNEL_SRC to this map.
But when I do make install it still shows

Build all (y/n) : n
Kernel Version Running 2.4.20-xfs
Found Kernel Source Directory ()

and later on I still can see some errors during make all that some file or folder is nowhere to be found.......

How can I make these sources to be recognized? Do I have to compile the kernel?

Thanks for helping me making this card work properly!!!