Gnutella Forums  

Go Back   Gnutella Forums > Gnutella News and Gnutelliums Forums > General Gnutella Development Discussion
Register FAQ The Twelve Commandments Members List Calendar Arcade Find the Best VPN Today's Posts

General Gnutella Development Discussion For general discussion about Gnutella development.


Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old February 10th, 2002
Moak's Avatar
Guest
 
Join Date: September 7th, 2001
Location: Europe
Posts: 816
Moak is flying high
Question Swarming, Multisegmented Downloading, Partial uploads...

Hi there,
it seems those phrases often cause confusion, I'll try to give a short definition (as far as I understood):

Multisegmented Downloading vs. Swarming
'Multisegmented Downloading' is what modern download manager and Gnutella clients do, download a file from multiple sources at the same time (increasing download speed). 'Swarming' means the same + spreading small file partials over the network automatically (increasing availability and download speed, distributing often requested files, balance load).

Most time people mean 'Multisegmented Downloading' if they talk about or 'Swarmed Downloads' or 'Swarming'! If you mean the real swarming better talk about 'Real Swarming'. To show you more about the difference: When we talk about downloading multisegmented the partials (file chunks) are typically very big, up to the full file size. Partials/chunks from Real Swarming are much smaller, a few 100KBs or whatever will fit best to the needs of making modem users part of the network. Segmented downloading and Swarming does NOT exclude each other! Swarming is just a bonus and actually not available on Gnutella these days. Real Swarming is far far future on Gnutella.

Swarmed Downloads or Parallel Downloads
People mean usually 'Multisegmented Downloads', those are only further synonyms... see above.

Partial upload
When a client does upload a file before it is finished with downloading the whole file itsef. This way especially big files get distributed faster and more often.

Automatic resume
When a client automatically downloads the full file and if a peer drop (or all) it full automatically searches more peers and continues the download without any further user interaction is needed. Automatic resume does need automatic requeries or other kind of techniques to find more peers offering the original file.

Feel free tor add more phrases & definitions below.... :-)

PS: Morgwen forced me to write this post, hehe. Here some older posts with more details:
"Improve Download Host/Speed" - http://www.gnutellaforums.com/showth...light=swarming
"A better way to promote sharing" - http://www.gnutellaforums.com/showth...&threadid=5939
"Thoughts about Segmented Downloading" - http://www.gnutellaforums.com/showth...&threadid=6396

A note from me (summary from the links above): The bandwith of modem users is completely wasted right now and modem users are often "forced" to be freeloader. Actually this egoizm is contraproductive and means less files for all. While high bandwith users can share the traditional way, swarming might be a key to make low bandwith users share with the network, without any significant negative effect or traffic-overhead. For both, for high and low bandwith users, sharing should mean there is a guaranteed high percentage of bandwith available for their own use! Conclusion, there are many ideas to envolve Gnutella and all work together. 1. A superpeer concept for dynamic traffic routing = reducing backbone traffic + improves network toplogy + increases horizon (more available files). 2. Search-caches for reducing double/multiple routed traffic = reducing high amout of search backbone traffic. 3. Swarming technology (as last step) = make use of the high amout of wasted bandwith + will spread often requested files + balance load + less "busy" servants (more available files). Please discuss about swarming and other possible improvements in the threads above and use this post for Gnutella definitions.

Last edited by Moak; February 14th, 2002 at 02:26 AM.
Reply With Quote
  #2 (permalink)  
Old February 10th, 2002
Morgwen's Avatar
lazy dragon - retired mod
 
Join Date: October 14th, 2001
Location: Germany
Posts: 2,927
Morgwen is flying high
Default Re: Swarming, Multisegmented Downloading, Partial uploads...

Quote:
Originally posted by Moak
PS: Morgwen forced me to write this post, hehe.
Thank you "VERY MUCH"!

Morgwen
Reply With Quote
  #3 (permalink)  
Old February 10th, 2002
Disciple
 
Join Date: February 8th, 2002
Posts: 10
barbapappa999 is flying high
Default

Thanks, now my confusion vanished (at least for a while)
Reply With Quote
  #4 (permalink)  
Old February 10th, 2002
MtDewJunkE's Avatar
King User
 
Join Date: January 13th, 2002
Posts: 78
MtDewJunkE is flying high
Default

I get it now, Thanks Moak!
Reply With Quote
  #5 (permalink)  
Old February 11th, 2002
Unregistered
Guest
 
Posts: n/a
Default

the reason people want this is even as a modem user you are limited by a modem's upload speed of 31K on a 56K modem. so a modem user can get faster downloads from two sources at once. most cable modems have upload limits too.
cable modem users can suck the bandwidth from several thousand modem users at a time and get everything they ever wanted in 2 seconds.
thus cable modem users will prevent modem users from the benefit of this feature because all uploads are taken.
ok, so a modem user gets the file faster. now does he turn his client off cause he's happy? no, he's not happy cause he wants it all and wants it now and will never be happy. he will even go buy another 160GB hard drive if his gets full.
so he goes on to the next file and the next file and modem users get screwed.
does he share? not much! it might slow down his downloading a bit!
and developers are acting like this is something good.
Reply With Quote
  #6 (permalink)  
Old February 11th, 2002
Morgwen's Avatar
lazy dragon - retired mod
 
Join Date: October 14th, 2001
Location: Germany
Posts: 2,927
Morgwen is flying high
Default

Mr. Unregistered!

You prooved already , that you have no background information:

http://www.gnutellaforums.com/showth...&threadid=7981

So now you will tell us that Fasttrack is slow? That modem users suck and go offline - sure I have DSL too I suck my stuff and go off than I have it...

But the point is, when you are sharing while you are downloading all can get their stuff! And swarming supports this!

Do you really think Gnutella was/is faster than Fasttrack?

Swarming is the "FUTURE" of Gnutella!

Morgwen
Reply With Quote
  #7 (permalink)  
Old March 23rd, 2002
Connoisseur
 
Join Date: May 22nd, 2001
Posts: 297
zeroshadow is flying high
Default Real Swarming

Does anything have real swarming?
What about freenet?
Reply With Quote
  #8 (permalink)  
Old March 23rd, 2002
Unregistered
Guest
 
Posts: n/a
Thumbs down BS already calls multi-segment dling 'swarming'

Bearshare already calls their multi-segment downloading 'swarming'.

It is pointless to try to give the 'proper' definition of swarming here flying in the face of what they have posted on their web site.

They have 1G x as many users as you have here.

You will have to come up with a better word for swarming.

I actaully agree with calling multi-segment downloading swarming.

I also think it is all that is needed .. swarming sounds like a lot of work for benefit that can't really be measured yet. How do you know it will improve things, not just use up bandwidth on people's clients without benefitting them?

I wonder what percentage of people never download a 'commonly downloaded' file?

I would be interested to see a protocol where this happens, but I think until it is a proven benefit for like 95% of users (at LEAST!!!) then it should not be in gnutella, which works.

I would like to see file hashes and multi-segment downloads (should I just give in and call them swarmed-downloads? It's shorter to type and sounds less technical for the users) implemented widely (>75% of users) before I resorted to implement 'swarming'. You might find that it is unnecessary (I think so :-)

Commonly downloaded files are just that, common. They are everywhere. With file-hashes, you will be able to find them in an instant. With multi-segment downloads you will download from many many sources at once. How does it improve things to make some people's clients download even more copies of these files (partially)? Like I said, the benefit is so far unmeasured.

I am not just arguing against it, I am curious to see it, but I think realistically once hashes and ms downloads are wide, then it will be put on a low priority and eventually won't happen (in most clients anyway).

So I think it is OK to call ms downloads 'swarming' and just give up on the other idea. :-)
Reply With Quote
  #9 (permalink)  
Old March 23rd, 2002
Unregistered
Guest
 
Posts: n/a
Default Real Swarming already proven

'Real Swarming' does indeed exist, in Edonkey2000. Partial files are used immediately for uploading (once one chunk is complete), and upload priority is given to the rarest chunk that you have.

The result is that even HUGE archives can be uploaded to thousands of people in just a few days. Take for example the 7 CD archive of a certain giant software company's latest visual development tool. We're talking about over 2.5 GB of data (7 CDs were not all full), released as 7 image files.

Starting just 2 days after this download was announced on an Edonkey2000 forum, it took only 1-2 days to download everything. That is the advantage of real swarming - the distribution pyramid that forms to spread files happens as quickly as a single chunk can be spread. What users see is that hot new releases are easy to download right away, not weeks or months later.

So, in answer to your question:
"I wonder what percentage of people never download a 'commonly downloaded' file?"

The answer is that everyone who ever wanted a 'hot' release of some new large file, like a movie or game, were in fact asking for a 'commonly downloaded' file. That's most people, I bet.

Oggs
Reply With Quote
  #10 (permalink)  
Old March 24th, 2002
Morgwen's Avatar
lazy dragon - retired mod
 
Join Date: October 14th, 2001
Location: Germany
Posts: 2,927
Morgwen is flying high
Default Re: Real Swarming already proven

Quote:
Originally posted by Unregistered
'Real Swarming' does indeed exist, in Edonkey2000. Partial files are used immediately for uploading (once one chunk is complete), and upload priority is given to the rarest chunk that you have.
No!

This feature is missing:

spreading small file partials over the network automatically (increasing availability and download speed, distributing often requested files, balance load).

This means you share also partials from files you donīt have!

AFAIK only mojo nation and freenet have REAL swarming, but I am not sure I didnīt test them yet!

Morgwen

P.S.:

Xolox (Gnutella) has also the ability to share partials but only these you REALLY share!
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Disabling swarming dddkkk Development Open Discussion 3 February 13th, 2007 01:34 AM
uploads crash when searching/downloading Krimany Download/Upload Problems 0 July 17th, 2004 12:01 AM
Continuing downloading a partial file BIG DEE Open Discussion topics 1 August 23rd, 2002 02:50 PM
On Multisegmented Downloading maksik General Gnutella Development Discussion 9 March 5th, 2002 01:55 PM
Hashes and Swarming Abaris New Feature Requests 0 December 5th, 2001 12:51 PM


All times are GMT -7. The time now is 12:29 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
SEO by vBSEO 3.6.0 ©2011, Crawlability, Inc.

Copyright Đ 2020 Gnutella Forums.
All Rights Reserved.