I personally don't used MS-ISA Server but the reason your gnutella software does work with it is pretty simple based on what I understand about proxy-servers in general.
Essentially you are attempting to proxy an application that makes requests for the the clients initiating the connection.
Client ==>Proxy ==> Internet
Your email and Web browsers can be configured to point to the proxy server and they will work correctly.
However Gnutella Application makes a request on the wire and is not heard or picked up by the proxy. (Remember the proxy server is listening on a specific port for request.)
If you tried to point your gnutella client to the proxy server in hopes that the proxy server would forward the request I'm relatively confident that this didn't work at least not correctly.
In ISA server you have the ability to create a firewall client. (I recommend that you read the documentation on how to do this)
You need to install this client on each machine running the gnutella application. I am assuming that you guys are not trying to run Limewire from the ISA server itself.
You will have to create the firewall client at the ISA server console so I am assuming that you have access to the ISA server.
Once you have sucessfully created /configured and installed the ISA firewall client on all of the machine you should be able to use limewire without issue.
Essentially the firewall client will funnel all of the traffic leaving the machine through the proxy server.
LocalMachineTx >Firewall_client ==>ISA-Server-Component==>Internet.
Again I don't know the specific details but this should solve your problem.
Hope this helps.
http://www.leeware.com