Bitcomet dies for lack of storage

[rmkbrj]

Hi,

I have been using bitcomet for a number of years. I'm currently at ranking 31 so many times I connect to a large number of LT seeds. Since moving to Windows 7 64 bit on a Intel I7 965 quad processor with 12GB of RAM it seems that bitcomet (version 1.15 or 1.16) will fail, and when restarted the tasks show a big red X that indicates there was not enough storage to run the task. This ONLY occurs when a large number of LT seeds are present. I finally limited the LT seeds to 520kB/s and that fixed the problem since without the limit it was not unusual to get speeds of up to 1200kB/s which apparently caused bitcomet to run out of storage. I am running a cable modem connection with a 7G connection. I'm guessing that bitcomet needs some storage usage limitations or needs to properly use storage on a 64 bit machine.

[kluelos]

What leads you to think that the problem is insufficient storage?

[rmkbrj]

Well, when bitcomet dies (abends) requesting info be sent and it is then restarted, if I move my cursor over the red X by the task it indicates that there was "insufficient storage to run the task". Maybe not the exact words. In addition I can watch the working set for bitcomet consistently increase. I've seen it Higher than 1,900,000 K looking at task manager. I haven't seen the value at the time of failure but assume it's higher. I am currently running two tasks downloading at about 300-400 kB/s and the working set is 1,569,000 K and steady. Without the LT thread limit it would still be going up.

[greywizard]

1.5GB is an enormous amount of memory for an application as BitComet. I can't verify if it's a bug in BitComet or in the new Win 7. But I've seen some other reports of great quantities of memory being used by BC under Win 7.

On my XP SP3 machine BC is running 67 seeding tasks (as private torrents, therefore without DHT or LT-Seeding) and 2 downloading tasks of public torrents and uses only around 85MB of memory.

So, there is definitely something wrong with your picture of it.

I guess only the dev team might have/find a certain answer to this but I'm very inclined not to rule out a bug in the new Win 7 memory management itself (at least until a SP1 is released).

[kluelos]

Practically, try these:

1. Downgrade to version 1.15 and see if the problem continues. Try to get it to fail again in the same way, look at the memory usage, etc.
   
2. Do a clean reinstall (i.e, clean out all traces of BitComet, reboot, then reinstall) of 1.16, then test again.
   
3. Log out of BitComet Passport, and test again.
   
4. Log in, but disable LT-seeding completely, then test again.
   

One or more of these may help to find a memory leak, which it sounds like you've got.

[rmkbrj]

Thanks for the direction. I actually had installed 1.16 and had the problem then downgraded to 1.15 and the problem persisted. Since it's Christmas Eve I probably won't get anything done until Saturday but I will then clean out my current system of bitcomet and re-install and see what happens. Thanks again for the help and I'll get back to you when I've re-tested.

[rmkbrj]

Here's the results:

Attatchments 1,2 and 3 represent the status of bitcomet running 1.15 with LT on auto running stable last night.

I uninstalled 1.15 cleaned up the registry, dumped e-mule re-booted and installed 1.16. I had backed up my settings and I imported them. Without any tasks running I was at about 23K on bitcomet. I started one task that I knew had substantial LT Threads. LT threading was set to auto.

Time Storage

3:57 23K

4:00 95K

4:02 188K

4:04 256K

4:05 293K

4:12 484K

4:16 649K

4:23 865K

4:28 1000k

4:34 1200K

4:39 1500K

4:45 1620K and somewhat stable

Downloading traffic went as high as 1200 kB/s and settled down at about 500 to 600 kB/s which I assume was caused by my provider balancing my usage at my max of 7M.

Attachments 4,5,6 and 7 represent bitcomet after the above observations.

I stopped bitcomet and started it again and logged off of passport. The climb in storage was observed to be about the same as the above figures.

I stopped bitcomet and restarted it again. Turned off LT and stopped and started bitcomet again.

Storage continues to grow with LT seeding off.

4:53 66K

4:56 300K

4:58 387K

5:00 450K

5:04 592K

5:08 670K

5:12 700K

5:16 765K

5:20 823K

5:26 884K

5:31 958K

5:38 1,018K

5:43 1,063K and somewhat stable

Observations:

Storage is rising to a peek in both cases but the level is much smaller without LT seeding. Seems to make sense since there are a smaller number of peers without LT seeding.

It was observed that storage would fluctuate up about 8000K then drop back and build up to that increment then bump up drop back and build up again. I would assume this is the effect of committing the memory then using it after it is committed.

Based on my experience, and I have quite a bit, it looks to me like the application either has memory leak of its own or W7 is giving too much memory to the application when it is requested. I would guess the latter but my experience is with large system memory usage not Windows.

As above,

Attachments 8, 9, and 10 are a representation of the system at the end of the last test phase.

Hope this helps.

[rmkbrj]

PS. I'm running W7 Professional however it also occurred before I upgraded from Home Premium. I'm also assuming that no abend occurred because I was not running enough work to drive the storage above 2G which I would guess is the die point for bitcomet.

[kluelos]

For a system of your size that is not also running demanding other tasks, your reported memory use isn't very high.

BitComet is going to try to take as much memory as it can, to use as buffering until it writes to disk. That's fine as long as it releases the memory upon request. But simple speed by itself shouldn't create any problems.

IF you've got a buffer equivalent to the largest unbuffered disk write the disk will accept, then there's no point in adding to it absent some other, unrelated heavy disk demand. in that case you're simply delaying the write, meanwhile starting another buffer to fill. You normally wouldn't expect these operations to fill up GB of memory absent something else.

The question in my mind right now is whether you have indeed solved the problem or not. It may be that you just haven't let things run long enough, but you're reporting islands of stability in what sounds like a normal sort of usage. And as far as I gather, you haven't seen the same storage problem again or seen memory usage climb up into the GB range.

Have you been able to recreate the insufficient storage problem after the reinstall?

[rmkbrj]

I can try.

[rmkbrj]

I've been running a substantial amount of work through bitcomet and have been unable to drive the storage above 1,630,000 K with LT seeding on. So something apparently has changed with the re-install. As you mentioned, it is apparently possible for bitcomet to use large amounts of storage and if so it appears to be running properly now.

Thanks for your suggestions and help with this. If I blow it out of the water again I'll re-queue the problem.

Bill

[greywizard]

It's nice to see a user who can conduct a thorough testing (given some parameters) for a change.

I just want to add a question.

Could you check if the memory usage of BC climbs up to the same values, when running only one or two tasks of regular size (700MB or 1.4GB) instead of the one you were using for testing?

I mean to find out if there is any direct relation between the size of the task and the amount of memory used.

[rmkbrj]

Most interesting. I am running a single task (2 files) and the storage had crept up to about 220,000 K when I noticed that the disk cache was set at min 100M max 1536M and not on auto. I change it back to auto and the storage usage immediately dropped to about 180,000K. Based on previous observations apparently the cache setting was causing some of the extensive storage usage since the higher values observed were about the same as the max cache value. The higher cache value will also be more of a factor with tasks with many output files, which I run quite often. This does not explain why bitcomet would arbitrarily die when not enough storage was available just because the max cache value had pushed him to far or for that matter, any reason. It seems like scaling back storage usage would be a better action rather than just burping. Wow, good service today. The task I was running finished in 22 minutes (700mb).

Bill

[sophia0316]

Our team member viewed your problem and said high-speed long-time downloading and cache setting are the reasons why your BitComet dies. So limit your cache setting and LT downloading speed can work for your problem.