Register to post in forums, or Log in to your existing account
 

Play RetroMUD
Post new topic  Reply to topic     Home » Forums » zMUD General Discussion
hav
Wanderer


Joined: 05 Oct 2004
Posts: 61
Location: Riga, Latvia

PostPosted: Sat Apr 23, 2005 5:44 pm   

%regex, %match and performance
 
I have
#REGEX {^@liste2 destroyed (.*)\.$} {specific unimportant stuff;common stuff}
#REGEX {^@liste2 completely devastated (.*) with awesome force\.$} {specific unimportant stuff;common stuff}
and 15 or so other similiar triggers in a class. Trigger on trigger disabled to skip parsing if it matches one of these.
So far so good.
I've decided to consolidate them in one, better readability, but more importantly to flag lines that doesnt match any of that.
%regex has no option for case sensitive that I could find. Ah well, guess will have to live with that.
trigger pattern ^(*)$
#VAR t2 0
#IF (@liste!="") {
#ALIAS abortcmd {}
#IF (%regex( %1, "^<@liste2> destroyed (.*)\.$", t)) {
stuff specific to this match
#VAR t2 1
#ALIAS abortcmd {#ABORT}
}
abortcmd
#IF (%regex( %1, "^<@liste2> completely devastated (.*) with awesome force\.$", t)) {
stuff specific to this match
#VAR t2 1
#ALIAS abortcmd {#ABORT}
}
abortcmd
and those 15 others in similiar fashion here.
}
#IF (@t2=1) {common stuff}

And here we have t2 flag showing match as well as matching line in %1.
trigger on trigger gets emulated #abort-ing out of #IF on first match. Might or might not be better for performance, hard to say.
I run the trigger with a 10-15 lines and it CHOKES the system. cpu shoots up to 99% used by zmud. Granted my cpu is ye olde kranky pentium 3 450 but come on, text processing, it never goes above 30% under lots and lots of triggers, 40-50% in most extreme conditions.

I poked around a bit, first thing tried I swapped ^(*)$ for ^*$ and using %line instead of %1, not much effect, then I removed anything and everything leaving skeleton.
#IF (%regex( %1, "^<@liste2> destroyed (.*)\.$", t)) {}
#IF (%regex( %1, "^<@liste2> completely devastated (.*) with awesome force\.$", t)) {}
and those 15 others.
very minor improvement, 88-95% cpu used with 10-15 lines fed in at once.
Tried on 7.04 and 7.05, guys what am I doing wrong here, must be something retarded, couldnt be such difference between #REGEX and %regex?
Oh yeah, title mentions %match. using %match or %regex has no effect on performance just like it doesnt seem to matter for triggers.
Reply with quote
megamog75
Enchanter


Joined: 20 Nov 2002
Posts: 627
Location: USA

PostPosted: Mon Apr 25, 2005 2:16 pm   
 
You are doing nothing too wrong, you should switch back and split them to two separate triggers.
You see Zmud has a bug when you give it to much to do at one time and it will start to mess up and skip and a bunch of other things.
Zugg is aware of this problem and since a small amount of us ever run into tis problem it will be sometime before it gets fixed. So be patient I have been waiting 2 years so I could run a script I wrote to work. When the bug is fixed I will have a wonderful script to post for everyone.
_________________
megamog75 Smile
I will do this.Nothing in my life matters except this.No moment in my life exists except this moment.I am born in this moment, and if I fail, I will die in this moment. Raistlin Majere
Reply with quote
hav
Wanderer


Joined: 05 Oct 2004
Posts: 61
Location: Riga, Latvia

PostPosted: Mon Apr 25, 2005 3:05 pm   
 
Zugg is aware of this? cool.
it doesnt mess up and skip things, it chokes the living daylight out of CPU.
Later tried on a mighty 2.8Ghz CPU it choked it to 99% utilized as well.
Heres to hopes next zmud version #REGEX {^pattern$} {} about same CPU time as #TR {^(*)$} {#IF(%regex(%1,"^pattern$")) {}}
And case sensitive %regex please Very Happy
Reply with quote
megamog75
Enchanter


Joined: 20 Nov 2002
Posts: 627
Location: USA

PostPosted: Mon Apr 25, 2005 3:21 pm   
 
the problem is that the trig and regex works but when you have alot of stuff on one line going into the trig that it needs to do and then triggers on itself
or
if you have a line that has more than about 6 triggers on
mine is a #sub that gets hit about 6 to 10 times a line and you should see that slow down a cpu and skip.

sometimes there are work arounds though.
what you need to do is catch all of what you wand and store it into a database variable
then loop thourgh it one line at a time, giving zmud a breath inbetween lines.
use #alarm for that

I could give an example, but I don't have time right now I need to get back to work, heheheheh.
_________________
megamog75 Smile
I will do this.Nothing in my life matters except this.No moment in my life exists except this moment.I am born in this moment, and if I fail, I will die in this moment. Raistlin Majere
Reply with quote
hav
Wanderer


Joined: 05 Oct 2004
Posts: 61
Location: Riga, Latvia

PostPosted: Mon Apr 25, 2005 10:19 pm   
 
So the %regex idea bombed. On to plan B. I'll do it with separate "no trigger on trigger" triggers and stick a * trigger at the end which fires if nothing else gets matched, a like so
#TRIGGER {^{@liste2} destroyed (*).$} {#VAR liste4 %additem( destroy, @liste4);#ADDITEM liste5 {%1};#SUB @liste4" ("@liste5")";#VAR mobhits 1} "mobhits" {notrig}
and then
#TRIGGER {^(*)$} {#VAR mobhits 0;#VAR liste4 "";#VAR liste5 "";#PSUB "%1 - moo" %x1 // testing, delete me} "mobhits" {notrig}
Not only do I have {notrig} option, but also help file for sub states,
"Substituted strings are not processed by further triggers."
Apparently zmud has a second opinion on it though,
#VAR liste2 moo
#ECHO moo destroyed Elf.
output
destroy (Elf) - moo
and mobhits var gets set to 0, without a fail. Now what am I doing wrong?
Reply with quote
megamog75
Enchanter


Joined: 20 Nov 2002
Posts: 627
Location: USA

PostPosted: Tue Apr 26, 2005 2:08 am   
 
what exactly are you tring to do start at the beginning.
_________________
megamog75 Smile
I will do this.Nothing in my life matters except this.No moment in my life exists except this moment.I am born in this moment, and if I fail, I will die in this moment. Raistlin Majere
Reply with quote
hav
Wanderer


Joined: 05 Oct 2004
Posts: 61
Location: Riga, Latvia

PostPosted: Tue Apr 26, 2005 2:36 am   
 
What I want is so simple its embarassing I havent done it myself yet, a hit parser that briefs hits and condenses consecutive hits in one line.
One way or another I want to flag when a line doesnt match any of hits messages, there are about 17 or so.
As shown below hit message gets briefed #SUB @liste4" ("@liste5")"
then if the previous line was also a hit line, it gets cut #IF (@mobhits=1) {#GAG -1}
incoming
moo destroyed Elf.
moo completely devastated Other Elf with awesome force.
different moo completely devastated Elf with awesome force.
output
destroy|devastate|devastate (Elf|Other Elf)
If possible, I wouldnt want to get databases involved for such simple thing.
Since going through every line with ^(*)$ bombed, over 100 active triggers is fine, barely any lag even on old CPU, but put a few string compares in one trigger and splat, then I can think of only two things.
One is that trigger on trigger thing. If all 17 hit message triggers are flagged no trigger on trigger, then a * trigger should fire when a line doesnt match any of those triggers. doesnt work that way, no idea why.
The other thing is to make a trigger that says do not match any of these strings.. well what can I say, today I looked at regex syntax for NOT matching something and wept. Something about positive lookahead and conditional pattern,
(?(?=^(the_only_string_you_dont_want_to_match)$)^$|.*)
more than my little brain can handle.
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23379
Location: Colorado, USA

PostPosted: Tue Apr 26, 2005 5:02 am   
 
Quote:
You see Zmud has a bug when you give it to much to do at one time and it will start to mess up and skip and a bunch of other things.
Zugg is aware of this problem...

Umm, megamog75, I am *not* aware of any problems like this. Unless you can point me at a specific bug report in the bug database that I'm forgetting about.

As far as I know, there are not any bugs with the triggers or regular expressions. However, to answer the original post in this topic, the %regex and %match functions in zMUD call the Perl Regular Expression library (PCRE) directly with no further zMUD optimization. Whereas the built-in zMUD-specfic trigger syntax has more optimizations for wildcards.

For example, if you have the normal zMUD trigger pattern "^{@liste2} destroyed (*)$" then zMUD performs an optimization where it first quickly scans a line for the "destroyed" text, and if that text is not in the line, then it doesn't bother to test the full wild card expression. Since wildcards (like * and the {@list}) are the slowest part of triggers, this really speeds things up. But when using the regular expression version, like you have, there is no way for the PCRE library to know how to make this optimization. It has it's own optimizations, but only the people who wrote PCRE (which is a standard unix regular expression library, btw) would really know how it works.

Also, you should keep in mind that during any looping statement, or even the #IF statement, zMUD can allow additional text from the MUD to be received unless you place your script within a #PRIORITY command. Try doing this and then it should wait until your script is done before continuing to the next line from the MUD.

But overall, you should keep in mind that yes, regular expressions can be slow, especially if you have lots of wildcards. PCRE is the fastest regex library available, so zMUD is doing the best that it can. And yes, to optimize them you end up having to look at the fancier features for lookahead and lookbehind, and yes regular expressions can get *very* complicated, which is why I recommend that most users stick with normal zMUD triggers when possible.

Finally, remember that * in zMUD doesn't match lines with special characters, such as %, #, ; etc. Use the %* wildcard is you really want to match everything, but then be sure you are not opening yourself up for trigger and script abuse by other players who might send you text that you end up executing as commands.
Reply with quote
hav
Wanderer


Joined: 05 Oct 2004
Posts: 61
Location: Riga, Latvia

PostPosted: Tue Apr 26, 2005 5:23 am   
 
At any time I have about 100-200 active triggers in zmud on a mediocre computer, and it never stutters 30% CPU usage peaks, plenty of wildcards and all. Yet 17 %regex compares bogs CPU down to 99% used. From what I understand, #TR and #REGEX gets passed through optimization before handed over to regex library, while %match and %regex are handed direct. Man, must be some wicked optimization then. Would it be hard to implement that %match and %regex also gets the nifty optimize treatment? Very Happy
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23379
Location: Colorado, USA

PostPosted: Tue Apr 26, 2005 6:15 am   
 
As I said, the %match and %regex directly call the Perl PCRE library. I have no control over what that library does. There is no way for me to further optimize them since they are functions that work with normal strings, and not necessarily just triggers, and there is no way to know in advance what string you will call these functions with.

Triggers can be optimized because zMUD knows that you are going to test the same line from the MUD over and over again. That's just the way it is I'm afraid.

Regular expressions weren't added for speed. They were added for people who absolutely needed some of the advanced functionality that they can provide. For 90%+ of your triggers, you can just use normal zMUD triggers which are faster.

As someone mentioned, if you post what you are trying to accomplish, perhaps one of the Gurus can offer more suggestions on how to do things more quickly. Using a ^*$ kind of trigger and then calling %match and %regex should really be used as a last resort only since it bypasses all of the optimizations that zMUD can help with.
Reply with quote
hav
Wanderer


Joined: 05 Oct 2004
Posts: 61
Location: Riga, Latvia

PostPosted: Tue Apr 26, 2005 7:00 am   
 
Quote:
there is no way to know in advance what string you will call these functions with.

Yes and no. If the string I compare contains variables, sure. None of %regex functions I call contains user-defined variables. This reminds me of Oracle somewhat. Long story short, it handles static and dynamic select strings completely differently.
Consider these two.
#REGEX {^pattern$} {} and #TR {^*$} {#IF (%regex(%line,"^pattern$")) {}}
I havent seen zmuds code of course, so its a bit of a stretch here, but whats to stop to apply optimization for this particular example or similiar strings. And if I do use a %regex with dynamic string, that cannot be optimized, let me know with a big red warning. Psst, your %regex isnt being optimised.
I use zmud mostly to play, if at all possible, spare me lookaheads and atomic groupings while using wildcards. Regex for dummies please Very Happy
Quote:
Regular expressions weren't added for speed. ..... For 90%+ of your triggers, you can just use normal zMUD triggers which are faster.

moot point somewhat, since it all now converts to regex anyway, no point using %match instead of %regex. I sure as heck am not going to early version with no regex, skipping on all the new cool features, not sure if zmud even had %match back then.

Back on topic, I've desciibed what I need 4 posts up.
One way to do it that I could think of, without flagging lines not matched would be to put all 17 triggers in a (?:a|b|c) type of pattern in looplines type of #COND
^@liste2 (?:(destroyed) (.*)|(completely devastated) (.*) with awesome force|etc)\.$
Then from first paramater I could tell which trigger was matched. #IF ("%1"~="completely devastated") {#VAR liste4 %additem( devastate, @liste4)}
Questionable performance because it has to match the string twice, once in triggers pattern, and second time inside trigger, but it just might work.
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23379
Location: Colorado, USA

PostPosted: Tue Apr 26, 2005 6:09 pm   
 
Let me put it another way. When you use #TR or #REGEX, zMUD is using *COMPILED* code to loop through all of your triggers and test them against the current line from the MUD. When you use a trigger like ^*$ and then call the %regex or %match functions, you are running *INTERPRETED* zMUD scripting code on every line returned from the MUD. Of course this is going to be a LOT slower. It's like comparing compiled C code to interpreted VBScript code. It has nothing to do with the optimization of regular expressions. Sorry if I didn't focus on the main problem with your method...it was late last night when I replied.

zMUD scripting is slow. It's a string-based parsed language. Running a huge script on each line returned from the MUD is a sure way to completely hose zMUD and bring performance to a stop.

"Regex for dummies"?? That is what the normal zMUD trigger syntax was invented for from the beginning. It has most of the power of regular expressions, but with simpler syntax.

So first, try to give up using regular expressions for now. Unless you *really* know what you are doing, regular expressions will just give you trouble, expecially with doing something complicated with the (?:) syntax. Yes, it will give you a headache and won't work the way you think it does. Not my fault...I didn't invent how regular expressions work. I avoid them myself because they never do what I want and they give me a headache, just like you said.

If I were doing something like you described myself, the way I would probably do it is to create a trigger for each condition. Let it match the condition and set a variable when the trigger matches. Then to handle the "didn't match anything" condition, have a single trigger that fires AFTER all of the others with the ^*$ pattern and simply check the variable to see if it is set or not.

E.g.:

#TR {^%*$} {#VAR Matched 0}
#TR {^@liste2 destroyed (*)$} {#VAR Matched 1;other stuff here}
#TR {^@liste2 completely devastated (*) with awesome force$} {#VAR Matched 1;other stuff here}
...
#TR {^%*$} {#IF (@Matched = 0) {nothing matched, so do other stuff here}}

This way, you only have a very short script running on each line. All of the other triggers are optimized.
Reply with quote
hav
Wanderer


Joined: 05 Oct 2004
Posts: 61
Location: Riga, Latvia

PostPosted: Tue Apr 26, 2005 8:37 pm   
 
That was actually my second idea how to get it done.
With all due respect Zugg, you wrote the most rocking mud client, but the variable needs to be zero only if the line doesnt match hit messages. The triggers have to go in the other order. ^%*$ at the bottom as well as {notrig} for hit messages.
#TRIGGER {^{@liste2} destroyed (*).$} {#VAR liste4 %additem( destroy, @liste4);#ADDITEM liste5 {%1};#SUB @liste4" ("@liste5")";#VAR mobhits 1} "mobhits" {notrig}
#TRIGGER {^(*)$} {#VAR mobhits 0;#VAR liste4 "";#VAR liste5 "";#PSUB "%1 - moo" %x1 // testing, delete me} "mobhits"
Weirdest thing though, though, the no trigger on trigger part isnt working as I thought it should.
#VAR liste2 moo
#ECHO moo destroyed Elf.
output
destroy (Elf) - moo
and @mobhits is set to 0
Reply with quote
hav
Wanderer


Joined: 05 Oct 2004
Posts: 61
Location: Riga, Latvia

PostPosted: Tue Apr 26, 2005 9:16 pm   
 
Err, I think I picked the wrong wording. What I wanted to say is, that while your approach isnt completely wrong, it involves two ^%*$ triggers that fires on every line, while mine has one that only fires on lines that are not hit messages. What can i say, I have old CPU, I am gentle with it Very Happy
Reply with quote
megamog75
Enchanter


Joined: 20 Nov 2002
Posts: 627
Location: USA

PostPosted: Tue Apr 26, 2005 10:14 pm   
 
you guys are defiantly out of my league I stopped understanding back at regex.

I was simply referring to the issue of Zmud slowing down when you give it to much to do on one line. I had figured that was his problem and if he could find a way around it, then he would solve his problem. In the end he did just that and seems to be heading him in the right direction.

My problem comes when I use triggers to match multiple items in one line then try to rewrite them back to the mud using #sub.

It always messes up and slows to a grind, then in the end misses a lot of what I needed it to do.

But on reading your thoughts on #priority I will re-write it and see if that clears it up.
_________________
megamog75 Smile
I will do this.Nothing in my life matters except this.No moment in my life exists except this moment.I am born in this moment, and if I fail, I will die in this moment. Raistlin Majere
Reply with quote
megamog75
Enchanter


Joined: 20 Nov 2002
Posts: 627
Location: USA

PostPosted: Wed Apr 27, 2005 10:36 pm   
 
I tried it and it crashed again, well back to waiting for that bug to be fixed.
_________________
megamog75 Smile
I will do this.Nothing in my life matters except this.No moment in my life exists except this moment.I am born in this moment, and if I fail, I will die in this moment. Raistlin Majere
Reply with quote
DeathDealer
Adept


Joined: 20 Jul 2004
Posts: 268

PostPosted: Thu Apr 28, 2005 6:46 pm   
 
Zugg wrote:
Quote:
You see Zmud has a bug when you give it to much to do at one time and it will start to mess up and skip and a bunch of other things.
Zugg is aware of this problem...

Umm, megamog75, I am *not* aware of any problems like this. Unless you can point me at a specific bug report in the bug database that I'm forgetting about.

In the readme1st file from Atreides_096 fighteval script:
Quote:
NOTE: Before you send me tells, yes I know this script could have combined about 88 triggers into two; however, those triggers, possibly because of the amount of information they had to digest, or possibly because of the size of the (monstrous) variable they were required to sort through continually, caused ZMud v7.05 to crash almost immediately, every time. This is why they are split to individual triggers.

Of course i keep forgetting it ask him if he tried {dam1|dam2|...|dam48} in the trigger or from an #if %ismember stand point in the body of the trigger.
_________________
Reply with quote
hav
Wanderer


Joined: 05 Oct 2004
Posts: 61
Location: Riga, Latvia

PostPosted: Thu Apr 28, 2005 9:00 pm   
 
Status update. Tried a few things, including #TEMP which for whatever reasons reacts on {notrig} different on its creation line.
So far closest to working is emulating {notrig}, unfortunately double ^%*$ trigs couldnt be avoided after all.
#TRIGGER "mobhitsID2" {^%*$} {#PRI {#T+ mobhitsID}} "mobhits"
hit messages triggers here, now including {#PRI {#T- mobhitsID}}
#TRIGGER "mobhitsID" {^%*$} {#VAR mobhits 0;#VAR liste4 "";#VAR liste5 ""} "mobhits"
Testing with #ECHO-ing hit message simulations gives flawless performance while real muds output breaks off at a whim.
annihilate (Vesta)
devastate (Vesta)
pummel|0 (Vesta)
etc
Because offline testing works I suspect other triggers somehow interfering. I think I'll handle from here. Thanks everyone for replying :)
Reply with quote
Display posts from previous:   
Post new topic   Reply to topic     Home » Forums » zMUD General Discussion All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

© 2009 Zugg Software. Hosted by Wolfpaw.net