there's a version of Java Webstart in the Applications/Utilities/Java folder.

The red arrow means you have a more recent version installed, and thus the installer won't overwrite a newer version.

So, try deleting the version of WebStart that you have (and the receipt, if merely removing the Web Start doesn't work).

Check, too, if you have any login items ticked in the System Prefereces->Accounts->Login Items. A safe boot should disable any of those, so you could try installing java while in safe boot mode.