It sounds like you don't have the Java bin directory added to your PATH environment variable. The PATH variable tells the system where to look for programs to execute. Open a terminal and type:

echo $PATH

That will show you the current value of the path variable. There's a file in your home directory called .bash_profile. Open this with your favourite text editor and change the line that begins with PATH= to include the java directory. Something like this:

PATH=$PATH:$HOME/bin:/usr/java/jre1.3.1_01/bin

Your system may be different so make sure you enter the correct location for the java bin directory.

Logout then log back in and test that the system can now find the java program with:

java -version

You should see some information about the particular version of java you're using.