Well, it's not going to make as much difference on a P2P network as it would if you were to connect to an FTP or SSL site. You might have DSL, and the person you are connecting to might have DSL but you can still get only 2kb/s from them. This is because their ISP might throttle their upstream bandwidth to approximately 20kb/s. Say they have 10 slots open for you and others to download from. That would wipe their upstream bandwidth out. Then you have to consider that they might be downloading or searching, or browsing the web - all of this uses a portion of their available upstream bandwidth.

However, you might connect to someone on a T1 or T3 line, and you'll get respectable download speeds - or you might only get 2.5kb/s, but you might be downloading 10 different files simultaneously each that speed - so your cumulative speed would be much higher.

Edit: corrected typos