i run bearshare (whatever the beta du jour is). i have noticed often that limewire seems to be downloading parts of files more than once. does limewire:

1) download in strict start-to-finish order, except at EOF to get chunks that were queued to other nonresponsive/slow sources, or can chunk download order be pretty much random (like shareaza)?

2) download overlapping pieces, perform checks, and redownload nonmatching parts? if so, does it have logic to give up eventually?

i have observed limewire downloads that proceed for well over 2x the file size, measured by bearshare's upload window "response #x" and the observed chunk size.

i also frequently observe multiple "download <chunksize> bytes, download 20 bytes, repeat" behavior near EOF.

can't answer your questions, but I've noticed that downloading from a BSh beta 5 to LW 2.9.8.2 seems to stall for a long time (I usually have to manually kill that download)

Hi

You raise an interesting question as to what you see in your download window. It is, at best, an approximation of reality. Because the download function is often shared between grouped hosts, every system has to have the ability to "pick and mix" data packets to optimise downloading shared between hosts with differing amounts of bandwidth available (consolidation and internal verification of the whole being completed during the hashing process). If one host goes off-line or runs short of bandwidth, the downloading is cascaded down the list of grouped hosts. Suppose that you are the first such host and the downloader runs down to the end of the group without completing the download. The downloader will then restart the downloading and your window may show sequential connections that, in total, may add to more than 100%. This should not be unique to Limewire.

bearshare logs activity to "console.txt" in a funny format. so:

fgrep FileName console.txt >x
<manually inspect x to be sure it's all the same downloader>
<run editor macro on x to convert \r\n to a newline>
<run editor macro on x to find all "<newline>Range:" lines, output to file y>
<run editor macro on y to put leading 0's on numbers>
sort <y >z

result:
- y contains Range: lines in order they happened.
- z contains Range: lines sorted by file position

it's obvious, looking at z (2nd file below), that limewire repeatedly downloads several parts of the file. whether that's limewire's fault or bearshare's i don't know.

here's y:
Range: bytes=0000000-0099999
Range: bytes=0099990-0199999
Range: bytes=0199990-0299999
Range: bytes=0299990-0399999
Range: bytes=0399990-0499999
Range: bytes=0499990-0599999
Range: bytes=0599990-0699999
Range: bytes=0699990-0799999
Range: bytes=0799990-0899999
Range: bytes=0899990-0999999
Range: bytes=0999990-1099999
Range: bytes=1099990-1199999
Range: bytes=1199990-1299999
Range: bytes=1299990-1399999
Range: bytes=1399990-1499999
Range: bytes=1499990-1599999
Range: bytes=1599990-1699999
Range: bytes=1699990-1799999
Range: bytes=1799990-1899999
Range: bytes=1899990-1999999
Range: bytes=1999990-2099999
Range: bytes=2099990-2199999
Range: bytes=2199990-2299999
Range: bytes=2299990-2399999
Range: bytes=2399990-2499999
Range: bytes=2499990-2599999
Range: bytes=2599990-2699999
Range: bytes=2699990-2799999
Range: bytes=2799990-2899999
Range: bytes=2899990-2999999
Range: bytes=2999990-3099999
Range: bytes=3099990-3199999
Range: bytes=3199990-3299999
Range: bytes=3299990-3399999
Range: bytes=3399990-3499999
Range: bytes=3499990-3599999
Range: bytes=3599990-3699999
Range: bytes=3699990-3799999
Range: bytes=3799990-3899999
Range: bytes=3899990-3999999
Range: bytes=3999990-4099999
Range: bytes=4099990-4199999
Range: bytes=4199990-4299999
Range: bytes=4299990-4399999
Range: bytes=4399990-4499999
Range: bytes=4499990-4599999
Range: bytes=4599990-4699999
Range: bytes=4699990-4799999
Range: bytes=4799990-4899999
Range: bytes=4899990-4927615
Range: bytes=0000000-0099999
Range: bytes=0000000-0099999
Range: bytes=0099980-0199989
Range: bytes=0199980-0199999
Range: bytes=0199980-0299989
Range: bytes=0299980-0299999
Range: bytes=0299980-0399989
Range: bytes=0399980-0399999
Range: bytes=0399980-0499989
Range: bytes=0499980-0499999
Range: bytes=0499980-0599989
Range: bytes=0599980-0599999
Range: bytes=0599980-0699989
Range: bytes=0699980-0699999
Range: bytes=0699980-0799989
Range: bytes=0799980-0799999
Range: bytes=0799980-0899989
Range: bytes=0899980-0899999
Range: bytes=0899980-0999989
Range: bytes=0999980-0999999
Range: bytes=0999980-1099989
Range: bytes=1099980-1099999
Range: bytes=1099980-1199989

here's z:
Range: bytes=0000000-0099999
Range: bytes=0000000-0099999
Range: bytes=0000000-0099999
Range: bytes=0099980-0199989
Range: bytes=0099990-0199999
Range: bytes=0199980-0199999
Range: bytes=0199980-0299989
Range: bytes=0199990-0299999
Range: bytes=0299980-0299999
Range: bytes=0299980-0399989
Range: bytes=0299990-0399999
Range: bytes=0399980-0399999
Range: bytes=0399980-0499989
Range: bytes=0399990-0499999
Range: bytes=0499980-0499999
Range: bytes=0499980-0599989
Range: bytes=0499990-0599999
Range: bytes=0599980-0599999
Range: bytes=0599980-0699989
Range: bytes=0599990-0699999
Range: bytes=0699980-0699999
Range: bytes=0699980-0799989
Range: bytes=0699990-0799999
Range: bytes=0799980-0799999
Range: bytes=0799980-0899989
Range: bytes=0799990-0899999
Range: bytes=0899980-0899999
Range: bytes=0899980-0999989
Range: bytes=0899990-0999999
Range: bytes=0999980-0999999
Range: bytes=0999980-1099989
Range: bytes=0999990-1099999
Range: bytes=1099980-1099999
Range: bytes=1099980-1199989
Range: bytes=1099990-1199999
Range: bytes=1199990-1299999
Range: bytes=1299990-1399999
Range: bytes=1399990-1499999
Range: bytes=1499990-1599999
Range: bytes=1599990-1699999
Range: bytes=1699990-1799999
Range: bytes=1799990-1899999
Range: bytes=1899990-1999999
Range: bytes=1999990-2099999
Range: bytes=2099990-2199999
Range: bytes=2199990-2299999
Range: bytes=2299990-2399999
Range: bytes=2399990-2499999
Range: bytes=2499990-2599999
Range: bytes=2599990-2699999
Range: bytes=2699990-2799999
Range: bytes=2799990-2899999
Range: bytes=2899990-2999999
Range: bytes=2999990-3099999
Range: bytes=3099990-3199999
Range: bytes=3199990-3299999
Range: bytes=3299990-3399999
Range: bytes=3399990-3499999
Range: bytes=3499990-3599999
Range: bytes=3599990-3699999
Range: bytes=3699990-3799999
Range: bytes=3799990-3899999
Range: bytes=3899990-3999999
Range: bytes=3999990-4099999
Range: bytes=4099990-4199999
Range: bytes=4199990-4299999
Range: bytes=4299990-4399999
Range: bytes=4399990-4499999
Range: bytes=4499990-4599999
Range: bytes=4599990-4699999
Range: bytes=4699990-4799999
Range: bytes=4799990-4899999
Range: bytes=4899990-4927615

The thoroughness of your investigation is to be applauded but it does little more than demonstrate what I said in my earlier post. You are always likely to see sequential downloads of some sections of the file when grouped searches are stopped and restarted. You will also see some packets resent when the internal verification at the receiving end shows that a corruption has occurred during sending. You may also see sending restarted and so duplicating different packets of data when the connection is interrupted for some other reason.

The only point of interest is your suggestion that this is a phenomenon particular to Limewire. I find this unlikely but, to be honest, I can't get excited enough about it to investigate. While it is true that there may be a certain degree of redundancy in the traffic between two linked hosts, it is unlikely to represent a serious traffic management problem for the network as a whole. There are far more serious issues in the traffic management arena to resolve. All I will venture is that this is not a malfunction on the part of either Limewire or Bearshare.

no kidding.
why can i download several gig with bearshare (which doesn't do overlap/check/redownload corrupted stuff) without any bad files, but ALMOST EVERY LIMEWIRE DOWNLOAD I OBSERVE DOES THIS REDOWNLOAD THING CONSTANTLY?
the only reason i posted this here is that it is a phenomenon unique to limewire downloads. i don't see this kind of thing (repeated download of the same part of a file) from any other servent. get excited already.
there should be NO redundant http downloads. all this is happening over TCP, which guarantees end-to-end packet health, which means it's NOT a network problem. it's an application problem. and again, bearshare and every other servent downloading from me DOES NOT exhibit this behavior, only limewire.

does limewire have the option to generate a similar activity log? specifically, does it log "i'm getting this chunk again because i smell corruption"?

Sincerest apologies

My elders and betters have corrected me. There is a problem with LW 2.8.6. The ever-wise Trap-jaw found it was incorrectly parsing the http headers, and causing all kinds of grief. My current mentor (the one who keeps telling me to shut up before I make too big a fool of myself) says that you are observing further evidence of this.

The Committee of the Good and True therefore thank you for identifying this and offer your work as a further good reason for LW users to upgrade and not use 2.8.6.

I'll shut up now before I get my foot even further into my mouth.

i'm seeing the same thing with 2.9.8 as well. i don't happen to have any 2.9.8's in my log right now, but will generate a similar list-o-chunks when i do.

Thanks for pointing that out Scott. I can't see a quick way to log similar data to console, but if some OS/2.9.8 combination is to be avoided, I'll wait for the next release before trying to share. I'd avoided connections to 2.8.6 and earlier, but looks like that wasn't enough. I've read that the developers were concentrating on the search code, so maybe the download work is the next project. Perhaps the work on partial filesharing will flush the past problems.

btw--do you get flooded with spurious results from GNUC and MRPH vendors? I gather vendor names can be spoofed, so don't know how reliable they are.

LimeWire does not redownload any chunks. Even if it detected a corruption it currently does not discard downloaded chunks and tries to get them someplace else. It just notifies the user that there has been a corruption.

What you are probably seeing is that LimeWire requests chunks multiple times. - That will happen if a request was not successful because your BearShare client was busy.

__________________
Morgens ess ich Cornflakes und abends ess ich Brot
Und wenn ich lang genug gelebt hab, dann sterb ich und bin tot

--Fischmob