Thank you for your helpful answer:
Maybe I am wrong here and overlooked something:
When I looked at the logs, every 5 sec a write is going on. most of them are 0 bytes written messages. So I searched for 5000 (ms) in the code (DownloadDataWriter) and there you go:
Line 111:
This should be at least a random value in a range, to spread the time when the threads are served again. They are all set to 5000 and are notified the same time again and most of them wait again 5000ms,...
Every thread waits the same time, that means they wake up at the same time, and later additional comes on top the forced write in Line 137/138. That means: A much higher buffersize with this 5000ms is totally ignored if you have not much downloadtraffic thats why it is never filled to an expected level, its flushed far too soon.
You can use a random value here to "spread" the threads more, set up the forced write time but the inefficient buffer usesage still exists, bec. wait time and downloads speed depend on each other to effeciencely use the buffer. You can use different values for different speeds for improved effiencency but this is still a even more bad hack.
Thats why this time check based implementation IMO is not a good solution.
My idea:
The buffer itself should write data when it reaches the 90% level independent from a time. If buffer reaches 90% enter a data writing queue and pause downloading for this file until buffer is flushed. This is easy to implement and dont have to struggle with different thread situations.