Gnutella Forums - Excessive threads

Gnutella Forums (https://www.gnutellaforums.com/)

- Support: General (https://www.gnutellaforums.com/support-general/)

- - Excessive threads (https://www.gnutellaforums.com/support-general/3470-excessive-threads.html)

Excessive threads

I shared 192 folders, with 277 files. (I shared the compaq recovery drive, lots of folders with one or two files.)

Threads:
XoloX uses 1 thread (temporarily) for indexing, 1 thread (also temporarily) for writing changes to the files, 5 threads for interface, web integration, main program execution, etc, plus one thread per shared directory and sub directory. So the more organized you are with your archiving, the more threads it's going to use. :P What's worse, is that every time there is a change to a directory (If you add or delete a file, or complete a download) XoloX kills ALL the threads then opens them up again, rebuilding the index for the directory that changed. So the less organized you are with your archiving the longer it will take upon modifying a directory. :P

XoloX terminates the threads for directories that are unshared while running.

Memory usage:
pending further re-investigation for v1.11 ...

Utilities used:
Sysmon -- Comes with windows, check Programs -> Accessories -> System Tools.
Another Task Manager -- http://atm.rm.it (you can use taskman in Win2K/XP)
Ollydbg -- http://home.t-online.de/home/Ollydbg

Hi Seeker,

Thanks for doing research on this topic.
In the next version we will eliminate those threads per directory..

Thanks again,

Greets,
Pasman

In my original "excessive threads" report (for 2.09) XoloX consumed memory while indexing at a rate of aprox 1 MB per thread. (or maybe .5 as I think Sysmon is broken, reporting disk cache usage when there is none.) With 1 thread being created per (sub)directory shared, this can add up rather quickly.

There may have been a leak in 1.09 in the CRC code which did not release RAM after allocating it for another file where one didn't exist in the directory.

Code:

DO 

  DO

    Allocate memory.

    Check to see if there is another file to index.

    If yes then

      CRC check and index file

      Free memory

    Else Exit DO

    End IF

  LOOP

  If another directory then

    Move to next directory

  Else exit DO

  End IF

LOOP
 

Instead of:
 

DO 

  DO

    Check to see if there is another file to index.

    If yes then

      Allocate memory.

      CRC check and index file

      Free memory

    Else exit DO

    End IF

  LOOP

  If another directory then

    Move to next directory

  Else exit DO

  End IF

LOOP

I say this becaues of how evenly the memory usage matched my shared folders as opposed to my shared files.

In 1.11 indexing the same 192 folders now uses ~3 MB of RAM, probably because CRC checking as been taken out. When folders are unshared, the memory used is freed properly. I think users will have a much better experience with this version.

More tests to follow.