Zugg Software :: View topic - [3.25-3.29b]Slow, severe memory leak [Revisited]

Posted: Fri Sep 03, 2010 9:51 am

I'm not sure what it was (maybe the gmcp change, as that's the only thing that I use based off the recent version history), but something that changed between versions 3.24 and 3.25 have caused me to have a severe memory leak. It's quite slow, as I didn't notice it until it had reached 850MB of usage, and that was after CMUD had been running for a few hours.

I haven't been able to pinpoint it - I've disabled all my scripts and still I've seen the increase. I'm back on 3.24 right now, and I am not suffering the memory leak that I was with 3.25.

Hopefully you'll have better luck than I do at finding this leak. I'll try to provide as much information as I can as requested.

Charneus

Edit: I notice it's still there in 3.24, but it's not as prominent as it is in 3.25

Posted: Fri Sep 03, 2010 5:14 pm

There is no way for me to find this without a lot more details. It's going to be something specific to your scripts. If it still happens with your scripts disabled, then try it in a blank session and see if you can find a procedure to reproduce it that I can use. It might be something in GMCP, but I haven't see the leak here. And I'm always running CMUD in debug mode with a memory leak tester running to tell me when I close CMUD if there were any leaks.

Wizard Joined: 26 Mar 2008 Posts: 1547

I imagine this may be what is causing the issues I've been having with CMUD slowing down. I'll also see if I can find some way to reproduce it since my first method of trying to get it to reproduce didn't work.

Wanderer Joined: 21 Aug 2008 Posts: 85

[edit][/edit]

Wizard Joined: 26 Mar 2008 Posts: 1547

Well, I don't use GMCP (my MUD doesn't support it) and I've definitely noticed signs of something at least similar to a memory leak.

Posted: Sat Sep 04, 2010 6:58 pm

Chamenas, do you happen to use the status window? That's a common link between Myrkul and myself...

Zugg: Myrkul and I also came across this address violation when we had the status window open, then right-clicked on the caption and chose 'Close.'

Wizard Joined: 26 Mar 2008 Posts: 1547

Nope, don't use a status window. And, as of yet, have been unable to find an identifying factor for my problems which would allow it to be replicated. I'll keep looking though.

Posted: Sun Sep 05, 2010 10:43 am

Don't know if this means anything, but I created a process dump of CMUD, and noticed this:

All that is gmcp variable stuff... am I wrong to think those are memory addresses being used each time CMUD 'updates' the gmcp variables instead of reusing already allocated slots? I've never done much in the way of hex editing, but I'm just checking every last option I have to find this memory leak so I can play normally again.

Charneus

Wizard Joined: 14 Oct 2000 Posts: 1368

The memory leak might be within the JSON library used?

Wizard Joined: 26 Mar 2008 Posts: 1547

There's two signs of my issues: the lagging command line and the fact that whenever I try to start a session it takes anywhere from 10-60 seconds for the session to open. What will happen is the session choice window will disappear and it will open up the chosen session, but the window will be filled with gray and nothing will happen for awhile. Eventually it will connect and the background will go black and everything goes as per normal. I've tried resetting the layout and what not, but it hasn't solved the issue, and I'm unfortunately no closer to identifying the source. Is there any information I can grab when these things are happening that will help?

Wizard Joined: 26 Mar 2008 Posts: 1547

I'm wondering if my extra windows are causing it. Whenever I reset the layout by starting with shift, it works fine. However, I setup my layout again and voila the problem at the start happens. Also, since my windows are always open, if there was a leak there, they could easily be causing the command lag.

Posted: Mon Sep 06, 2010 9:30 pm

I'm now wondering if it's anything to do with my #WAITs that I use. Right now, I have a clean session with just a few basic triggers, and then my weather script which has a #WAIT to create a new thread so it's not locking my command line when it tries to get information from the web.

I also use it in a few other places for the same reason... with the one #WAIT, I've gained 15MB over the course of 6 hours. With 8 #WAITs, I was seeing 500MB gains within the course of three.

If I recall, I had this same issue a long while back, and I think it was fixed. Myrkul said he had an extra thread running, and when he disabled the script, it no longer ate up memory.

Charneus

Wizard Joined: 26 Mar 2008 Posts: 1547

So, I opened up the task manager to watch memory usage as I opened CMUD. I clicked on one of my sessions with its layout of several floating windows and it gave me the whole "gray background" problem and not connecting for nearly a minute and I watched as CMUD, doing nothing on the surface but show me a blank window, skyrocketed to 158,000k mem usage. Now, since this was still below where Firefox was at, it didn't worry me so much, I had no real comparison to go off of. So I opened another session a bit later (after having closed out CMUD). This session opened normally (even though it has floating window??) and ran at about 30-50k for as long as I was playing it. Both sessions have roughly the same amount of triggers and aliases, since both share my main package for the game, and their own packages don't have much else in the way of triggers and aliases.

Both have the same window layout of a Main session window, flanked to the right by four created windows. And yet one loaded up and ran normally, while the other took nearly a moment before it would connect or show anything other than an unusual gray background. This same problem happens to another session of mine, in fact, it was getting so bad that it was causing sound and graphics to freak out while I waited for it ot fix itself for the other session, so I had to force CMUD to close, which fixed everything.

Posted: Tue Sep 07, 2010 5:18 pm

If you are using #WAIT, run the #THREAD command to see how many background threads are active. You might be getting a bunch of stuck threads.

On the memory dump, it does look like the JSON parser is somehow not releasing some of it's memory and that might be causing the memory leak with GMCP. I'll try to do some more testing with GMCP and see if I can reproduce this.

Wizard Joined: 26 Mar 2008 Posts: 1547

Is there anything I can do to shed light on my problem, Zugg? So that you can look into it? I know my MUD doesn't use GMCP and so I don't, so I don't think that's related to the odd spikes in mem usage I'm seeing. The one common identifier I'm seeing so far is the layout issues, I think it may have to do with custom windows being sized differently than default, odd as that sounds.

Posted: Tue Sep 07, 2010 5:35 pm

Chamenas, your problem is different than the one originally posted to this thread, so you should really create your own thread about it. This thread is about a severe memory leak in GMCP.

Wizard Joined: 26 Mar 2008 Posts: 1547

Well the title is just about a mem leak, which is what I suppose my problem to be. But I'll make a new thread.

Posted: Wed Sep 08, 2010 8:35 am

Well, don't have multiple threads running at the same time (I have one #THREAD and the rest are #WAITs), but one oddity (I'm guessing) I found is this:

Apprentice Joined: 10 Apr 2009 Posts: 145

Hmm.. May be a little off topic, but:

Posted: Wed Sep 08, 2010 4:34 pm

The [u] in front of the thread name indicates it is the "user" command line thread. So it's normal to have one of those for each of your windows.

Charneus: This thread ID gets incremented each time you use a #WAIT command on the command line. The #WAIT causes the existing thread to be placed into the background and causes a new thread number to be created for the command line. Even running an alias from the command line that uses #WAIT will cause this. So this isn't necessarily a problem, but it does indicate that you are using #WAIT commands somewhere.

Moo: Your thread looks a bit different and might be a problem. The ID 101 thread is associated with your window and is stopped. Since there isn't any [u] at the beginning, that indicates a thread created by a trigger that used a #WAIT command of some kind. I'd definitely go ahead and kill the extra thread (#STOP 101) and see if you can trace down what script running in that window might be causing it.

However, I don't see anything in either of these #thread outputs that would explain the memory leak, so that puts it back to something related to the GMCP and JSON parser code.

Posted: Wed Sep 08, 2010 7:06 pm

Zugg: Is it only when it's initiated from the command line (i.e., I have to type out the alias or type #WAIT)? Or is it any time a script uses it, period? All of the #WAITs and #THREADs are done script-wise. I never enter anything into the command line.

Charneus

Posted: Wed Sep 08, 2010 8:48 pm

It's only from stuff initiated at the command line (like an alias call). A #WAIT within a Trigger uses it's own thread and does not cause the ID of the command line thread to be incremented.

Posted: Wed Sep 08, 2010 11:25 pm

Okay, because like I stated, none of my #WAITs nor the #THREAD are initiated by an alias on the command line. It's all through triggers (though two triggers call an alias with #WAIT/#THREAD in it). So would this be a bug, since I'm not executing it at the command line?

Posted: Thu Sep 09, 2010 4:34 pm

Yes, it might be a bug if you can pin down how it is happening. But I doubt it has anything to do with the memory leak we are talking about here.

Posted: Thu Sep 09, 2010 5:16 pm

I'm not seeing any GMCP memory leak over on Imperian, so this afternoon I'll do some testing on Aardwolf to see if there is any difference. Note that I'm just testing this by running around the MUD a lot and watching the memory usage. No scripts or anything. Just GMCP enabled with the mapper.

You did mention that it was a slow leak, but I wonder how slow. If you can do any more testing on your end, that would be a big help.