The ad banner in the main window is really not that bad. It is there so Xolox is free. If you would like to pay for a Gnutella servant I don't know of any to recommend.
As for the files that are launching browser windows, that is a problem with that actual files you are downloading. There have been many files found on the network (independant of Xolox) that have been planted for commercial reasons. This is an issue that is not specifically related to Xolox.
Xolox has only one popup (immediatly at startup), so the rest of the popups might be the resutls of third party programs left on your computer by other servents. I recommend using a program like Adware (
http://www.lavasoft.de) to help get rid of any such programs.
Your problems with downloading may have to do with the specific files you are trying to get. Rare files are hard to get regardless of the servent you are using. Also, try and be pataint. Because Gnutella is p2p, you have to depend on other people's computers. Your problem may not be on your end.