I have run many different p2p packages in both Windows and UNIX, and I still use only Kazaa-Lite (on windows) or gtk_gnutella (on UNIX), because these are the only programs that seem to handle searching properly.

I want to be able to find 200 to 500 sources for a program, knowing that only 5 or 10 of these sources might actually be able to provide a good download thruput. My systems are capable of download speeds of 200K/sec or more, and only these two programs have been able to provide the sources that can support those speeds.

Finding 20 or 30 files on a search, of which only 2 or 3 have multiple sources is rediculous, since many of the common sources on p2p networks are misconfigured, or have routing problems that result in thruputs of a couple of hundred characters per second.

If I wanted to download an Anime video file, which might be 200MB in size, at those speeds it would take almost 12 days to obtain one file. And this is what I typically see out there.

We need a search feature that can repeatedly search again and again, until the user either decides to stop, or it reaches some maximum (but a very large maximum). The large maximum is needed because some file groupings really do have hundreds of files associated with them. For example, the Anime video series Inu Yasha has 125 episodes, and there are multiple versions, both dubbed and undubbed out there. The only search term you can count on is the title, because of the way users list their shared files.

Limiting a search return to 20 or even 50 files would prevent the user from ever finding the complete set of files.

Note that on both Kazaa and gtk-Gnutella, it's often needed to let the search run for more than an hour to turn up some of the episodes. While this may reflect a bad protocol design, it's the real world effect of how these programs and users work in concert. And while an immediate search will sometimes turn up 5 to 15 sources for the most popular files, letting the search run for a long time often results in 200 or more sources for the files.

Yet even with 200 sources, I often am unable to find a machine that has an open download slot, has the file shared, and has a reasonable thruput. I usually sit and watch the download flip through source after source, only to see them time out, or be rejected. Once in a rare while, I see someone allow me to queue on their machine, but with these size files, waiting for an unused slot can take hours.

So with 100 sources or more, I see an aggregate download average speed of 10 to 20K per second. And I find that acceptable, although it's nowhere near the speed capabilities of the machine. Once in a very rare while, I find a fast source that I can connect to, and see a download reach 200K per second. That's the way it should work, but there is so much wrong with the common protocols, that it seems that it's really the rare exception to the rule.

Further, sources come and go with time, and the approach used to build the aggregate needs to select the fastest available sources which actually will grant a connection at that moment when a slot opens in the aggregate. Kazaa-Lite seems to handle this fairly well, considering the number of sources that say they have the file, but actually refuse the connection for one reason or another.

While I do not often agree with the approach used to build a fast aggragate download queue, and to maintain the highest thruput possible, it's very clear that most of today's programs are unable to reach this goal. Perhaps it's because of legacy code, which people do not want to break. Or for the sake of compatibility, they are unable to make the best use of the available bandwidth.

But it's clear we have a long way to go in reaching a low frustration level in p2p networking.

LimeWire, in specific the beta currently available, has no ability to retry a search automatically, and only seems to allow one additional re-search before clearing the search and starting over.
Yet after leaving the program running for a day, and then searching, I was unable to find more than 27 files in my catagory.
In the same period of time, gtk-gnutella found 2789 files, with literally thousands of sources between them. And Kazaa-lite found almost 200 sources for each of the main files of interest, and in some cases nearer to 300 sources!

Limewire Beta feels like I am only reaching a few machines in mostly a local area. I'm sure that's not the case, and that some sources could easily be on the other side of the world. But that is what it 'feels' like, because of the way it responds.

Unless such a feature would completely break the network protocol, or interfere with the network in some way, it really must have a auto-retry search capability, to compete with the other programs out there.

I suppose if I was looking for more common files, it might not be such a big problem. But these other programs have clearly solved the problem, and LimeWire must also, or be unable to be useful to the general user who is looking for more than just the popular mp3 files of the week.

Please make sure the beta team sees this request.
Thanks,
Rockkeys

A low throughput can mean that a remote servent is misconfigured. But it can also mean, that this as fast as the connection allows it because there are parallel uploads going on.

128 Inuyasha episodes are available via different p2p networks, AFAIK, - but you won't be able to find & download many of them with Gnutella.

That's why the limit is at ~150 results.

LimeWire's lack of automatic research is intended. Finding additional sources via keyword search is very inefficient that's what the download mesh is for. Once your download has started, you will be informed about other sources for this particular file automatically.

KazaaLite and gtk-gnutella repeat searches automatically. They can do that because they have a smaller market share on their respective networks. If LimeWire were to do the same, the performance of the Gnutella network would deteriorate quickly.
All those additional query packets would have to be routed throughout the network not only causing huge redundancies but also overloading the ultrapeers that have to route queries.

We had that a while ago and it sucked. Too many queries were overloading the network so instead of finding more sources you were actually finding less because queries had to be dropped before reaching very far.

A new machine ? Nice stuff - I bet it's a screamer! ;-)

>128 Inuyasha episodes are available via different p2p networks, AFAIK, - but you won't be able to find & download many of them with Gnutella.

Actually, both Kazaa and gtk_gnutella find hundreds of the files, and usually hundreds of sources for each of them. I recently grabbed the most recent IY episode using gnutella, had no problems finding it, and got over 50K/sec download average speed. Kazaa lite can hit 170K/sec if I can find the right sources, but that's rare. Most people seem to be in the 1 to 10K/sec range.

>That's why the limit is at ~150 results.
>LimeWire's lack of automatic research is intended. Finding additional sources via keyword search is very inefficient that's what the download mesh is for. Once your download has started, you will be informed about other sources for this particular file automatically.

Kazaa works in a similar fashion. Regardless of the number of sources that turn up in a search, it only passes the 20 best bets to the search process, which then opens up 200 slots for each file, to be filled in as they are discovered, or as needed when sources shut down, or become disconnected.

What the large result list can do is offer you a better choice of the initial 20 sources for a file, and it also gives the user a feel of what is out there, so that they know what to expect. If they let the search run for a 1/2hour, and only find 5 sources, they are not likely to be able to download it. If they find 100 or more, chances are good they can get the file.

I find that typically only 1 out of 10 to 20 sources is really going to give you a connection right at that moment. The others disappear, time out, put you in a queue, or whatever. Even limewire does this. Files marked with 4 stars often stall right away, even with 6 to 10 sources. Some refuse the connection, some are too busy (or say so), some place you in a queue, but never get freed up because these files are large and can take 1 to 2 hours to de-queue, even with a fast connection.

By the way, if limewire is a gnutella client, why does it only connect to limewire peers? Is this intentional to force the restrictions of limewire onto every source that can be accessed?
While gnutella says there are hundreds of copies of the files out there, I can't find them with limewire, because it restricts the TTL needed to get to those sources.

>All those additional query packets would have to be routed throughout the network not only causing huge redundancies but also overloading the ultrapeers that have to route queries.

>We had that a while ago and it sucked. Too many queries were overloading the network so instead of finding more sources you were actually finding less because queries had to be dropped before reaching very far.

THere must be a happy medium that will work better than what we have. While you say that Kazaa and gnutella can't really provide good performance, I'm getting 50K/sec downloads with gtk-gnutella, and peaks of 170K/sec on Kazaa when I can find a source that has a fat pipe.

Yet the fastest I've seen on limewire yet is about 11K/sec, and it's using the same hardware to reach the net. It's also on the fastest machine i have, a dual processor system running Solaris 8, with lots of memory and Ultra-160 SCSI disks. So it's not I/O bound. All the machines are attached to the same local net through a switch. And I only run 1 p2p app at a time.

Perhaps it's my ISP, though. They have a screwed up routing scheme that makes my data travel 6 hops to Maryland, before it reaches a boarder router to the backbone. I don't know where limewire limits, but if it's at 6 or 7 or so, I'm not even reaching the backbone before I'm out of TTLs.

If that's the case, you need to make the TTL adjustable within some small limit, so that I have a chance to reach other local nets. Adjustments from 5 to 15 would surely do the trick.

But I do wish you could have been a little more open with the number of results, and how they are handled. You don't have to pass all the results to the download, just pass alone the ones that seem like the best bets for a connection, limiting whereever you limit now. But give them more choices. Since you do open the slots up once they are queued, they may find enough active connections to get better thruput.

I guess all the virii and trojans out there have made people get firewalls (which they should have, don't get me wrong). But if the slow feeds are because they don't know how to enable forwarding, it's really sad. I've heard that over and over, and seen it in the FAQs, yet it's so simple to do with almost any firewall, hardware or software. A few click to pass the port through, and it's done.

But I guess it's either that, or there are a lot of 1200 baud modems still out there! Ick! (I'd like a T1, but wouldn't we all?)

I guess I still disagree with most of the designs out there. The problem for me is the design really has to work in the real world, where ISP's do crazy routing, people misconfigure their firewalls and proxies, and servers are setup to be overloaded to the point of being unusable. I don't know the details of what Kazaa-lite has done beyond what I've said, and I'm still reading through the source code of gtk-gnutella, and have a long way to go.

But I am able to find the files I want using their programs, and reach download speeds that don't make me wait for 12 to 24 hours for a file. To me, that's working, no matter what they are doing to their network. It may be inefficient, but it works. And they will only get better as they learn how to tune their net for better performance.

I'm hoping that LimeWire will too, and change some of their limits to work better for people like me who search for large filesets, and download files that average 200MB each. I suspec that for 4mb *.mp3 files, Limewire is plenty fast, and it's media player even works on Solaris, which is amazing.

Nice Job! for whoever wrote the player. Now if they would include a video player that handles AVI files, I'd be overjoyed.

And, oh yeah (can't resist it), the player needs a volume control. I know programming the Solaris audio mixer is difficult, and badly documented, but have them contact Mike Reilly at Sun, and he will show them how <mikeri@sun.west.com>.

Or at worst case, put in a mute button. We need a way to lower the volume when the phone rings ;-)

I do like LimeWire, it looks nice, and works pretty damn well for the curren beta. I'm impressed by the statistics pages, but do wish that 'write to disk' of the data was enabled, even it it has to be comma delimited formats, or a graphic snapshot in the local native format. But those are whole different topics, and belong in their own thread.

So if you want to follow up, how do we get LimeWire to work better in the trenches, when compared to the other apps I mentioned. While the design probably is more 'correct' than those other apps, they deliver, while I'm still waiting in LimeWire.

Regards,
--Rockkeys

ps: Please don't take this as anything but constructive criticism. I am not trying to shoot the product down, but instead find a way to improve it. I'm reporting what I see, and while my opininons on how to fix it may be in a cocked hat, the performance observations are real - details on demand.

I'm quite interested in helping if I can - I spent over 12 hours porting and building gtk-gnutella on Solaris 8 using the Sun compilers instead of gcc/g++, because I have to keep a commercial development environment here. Since that app required about 8 third-party support libs to also be ported, built and tested, from source, it took quite a while, even though I am a porting engineer. So I have a committment to this. I'm sorry to say I'm a C/C++ programmer, not Java, so I can't help in that area. But I can test, and provide feedback, which is what I'm trying to do.

Comments are welcome, and I do have some time available each week to put in on test and evaluation.

-R

LimeWire allows connections to any gnutella client as long as it's up-to-date enough.


Although you some sources may be more than 4 hops away, a TTL of 3 is enough to search a network that is more than twice as large as Gnutella.

That's because neither gtk-gnutella nor KazaaLite are giving network health much consideration. That is no argument for LimeWire however, to do the same.