Zugg Software :: View topic

Beginner Joined: 30 Jul 2011 Posts: 16

I'm trying to figure out how to match MXP tags with regular expressions - any help would be much appreciated.

I gathered from the MXP specification that tags begin with <ESC>[# so I tried \e[\# but got no hits.

Sorcerer Joined: 15 Apr 2011 Posts: 809

Try using the MXP trigger type, then you only need the name of the tag as the pattern.

Beginner Joined: 30 Jul 2011 Posts: 16

I tired that, but can't get that to work either. Here's some output from a log, showing tags:
<u>@</u>

I tired an MXP trigger with pattern:
u

And got no match.

From the help files, MXP triggers SEEM to be simple, but I'm not sure what I'm doing wrong. I'm pretty comfortable with regular expressions, so I was hoping to avoid the whole MXP trigger by using regexes.

Sorcerer Joined: 15 Apr 2011 Posts: 809

Well, my MUD doesn't use MXP, so I can't really test this, but try using %%e for escape, and you might want to quote the [ too...

Beginner Joined: 30 Jul 2011 Posts: 16

from the Specification:

Sorcerer Joined: 15 Apr 2011 Posts: 809

Keep reading that specification. The # isn't a literal pound sign, it's a decimal number that represents the tag you're trying to trigger on. For the room exits tag, you'll want to replace the # with 12.
EDIT: You might also want to try using the script debugger window with the raw session input/output option on, to see exactly what the MUD is sending.

Beginner Joined: 30 Jul 2011 Posts: 16

Ah, okay. I find the specification pretty hard to understand, partially due to my lack of background knowledge about XML, HTML, etc., and partly due to the style it's written in - not really designed as a help document. So thanks for pointing that out, and I'll work on slogging through the rest of the spec. I didn't realize there was an option to see the raw output - that will help a lot.

Sorcerer Joined: 15 Apr 2011 Posts: 809

Yeah, really you shouldn't need to read the spec at all if you have the raw output from the MUD. As I said, I can't really test this because my MUD doesn't use MXP, but if you post a few lines from the script debugger window, I should be able to help you write trigger patterns for them.

Beginner Joined: 30 Jul 2011 Posts: 16

Thanks for the offer :)

Ok, I scrutinized my raw output and learned a few things, and went over the spec again. I still can't manage to match the vast majority of tags in my output with MXP or regex triggers.

If I set my terminal type to ANSI and enable MXP with my MUD server, a lot of ANSI codes appear in the raw output, such as <esc>[39;49 and <esc>[0m for default text, and various codes for colored text. I now understand MXP codes for changing modes, and those appear in the raw output, too: <esc>[4z and so on. MXP tags appear too. Some examples:
<ESC>[4z<send href="goto aLink">linkText<ESC>[4z</send> for a clickable link (I can match this one with an MXP trigger)
<ESC>[4z<BR> for line breaks
<!EN hp 500 publish> for Server-defined variable entities
<IAC><GA> at the end of the entity tags sent after the output-response to each command. I have no idea what these signify.

If I change my terminal type to MXP, the output shows no ANSI color codes. Instead, color tags are used, e.g <Green> <B_Blue>. I can't match these with MXP triggers.

If i enable MXP menus (which are off by default) I get additional entites in the output:
[4z<!EN ob "very angry bear" private>

It would be nice to match more than Exits and links with MXP triggers, for simple things, but to really make powerful triggers it seems nothing beats regexes. I was rather hoping that an ANSI trigger with a pattern matching MXP mode codes would work - it doesn't. I would really like to be able to match MXP color tags, and well, every other kind of tag probably at some point in the future.

Beginner Joined: 30 Jul 2011 Posts: 16

I have figured out that using the script editor to view ANSI codes turns a lot of MXP tags into: <esc>[5S, which can then be matched with an ANSI trigger. This is useful, and answers my most pressing need. But this doesn't work for solitary color tags, and I'd still really like to know how to match color tags with a regex (allowing the pattern to capture more than the tag and its arguments alone). I'm particularly confused by the fact that I can match a couple of tags with MXP triggers, but not the rest. All tags in my MUD are sent in secure mode, so it can't be a mode issue.

Sorcerer Joined: 15 Apr 2011 Posts: 809

It looks like the color tags are just being sent in the actual output? You should just be able to use \<Green\> to match them if that's the case. I'm not sure how you would match the custom <!EN.... tags though...

Wizard Joined: 22 Mar 2007 Posts: 2320

You can turn off MXP parsing in your mud window so that you can see what is actually being sent from the mud. You can then cut and paste stuff directly into your trigger patterns.

Posted: Tue Aug 02, 2011 9:16 pm

Normal CMUD triggers only fire on the text output shown on the screen and *not* on the raw data input. There is no way to trigger on stuff like ESC[1z for example.

For MXP, triggers fire on the actual tag value. Not sure why you had trouble with the U tag, but I created the following trigger and it fired properly:

Beginner Joined: 30 Jul 2011 Posts: 16

It was useful to learn that ESC[5S will match a lot of MXP, though it won't discriminate. Apparently ESC[5S matches MXP mode changes, which precede every MXP tag on my MUD. I'll experiment more with MXP triggers when I can, though I remain befuddled by some tags being matchable and others not. I rather expect the problem is on my end, since MXP triggers are straightforward, and I now have a much better idea of the types of data sent and how CMUD interacts with them.

It probably comes down to my terminal type registered with the server. I have a sinking suspicion that I can run in ANSI mode and trigger on ANSI codes, which allow me to capture entire lines or multiple lines, or run in MXP/CMUD mode, and trigger on tags, but not capture whole lines for analysis or manipulation. Capturing multiple lines is critical for me, and my experiments with multistate-triggers and Reparse has taught me that only the primary trigger can match multiple lines, and a subsequent reparse trigger state will begin looking for matches only at the beginning of the last line of the matched set. The rule that allows subsequent reparses to function if an earlier reparse trigger failed is also a deal-killer for me. Multi-state triggers are user-friendly, but apparently limited in functionality, which seems a good bargain if other options are available.

Maybe this will work: MXP trigger to find the tag, then an old-fashioned reparse trigger using the code template in the multi-state help file. This reparse trigger could presumably capture multiple lines with a regex. The result, hopefully, offers the same power as a solitary, ANSI, multiline regex. Come to think of it, this could only match one color, while a solitary, ansi, multiline regex can match any number of colors and specify with precision where they should be in the pattern.

So I'll do more experiments, but consider this a vote for making MXP triggers more flexible. A redesign of multistate triggers to remove the multiple-lines problem would be nice, though I don't know if that fits in with the principle underlying their inclusion in CMUD.

Alternatively, decouple MXP patterns from the trigger type, so tags can be found in regexes, just as ansi codes can. That would offer the most flexibility, tie in with the current CMUD design of a choice between easy and powerful pattern-matching, and require no redesign of the easy-to-use MXP trigger.

(As long as I'm asking for features, note that zMapper-like graphic options rate higher in my list: I'd live with ANSI only for a good while if I could prettify my maps without converting them back and forth!)

Posted: Tue Aug 02, 2011 10:21 pm

Wanderer Joined: 24 May 2005 Posts: 71

Yeah, I'm gratified by the compatibility between CMUd and zMapper, and realize that the conversion option is there (Mnem seems to also, he mentioned it), though I haven't used it yet. I've been too busy working on other stuff when I have time to focus on MUDding. And though I own a zMapper license from about 6 years ago, the HD on which it was installed is long gone, and I can't remember the email I used to register it. Though this is annoying, I will surely buy another zMapper license eventually, when i have time to work on my map appearance. Throwing another $30 Zugg's way isn't a terrible thing in my view -- surely MU* participation has been decreasing, along with zuggsoft revenue, ever since UO went live (I left the genre myself and have only recently returned). I hope CMUD will continue to be developed, and a second zMapper purchase can't hurt.

Sorcerer Joined: 15 Apr 2011 Posts: 809

Beginner Joined: 30 Jul 2011 Posts: 16

Daern, I thought I made it clear that I understood that this particular limited functionality of multi-state triggers was by-design, which is why I said:

Wanderer Joined: 24 May 2005 Posts: 71

Mnem and I have been working on similar problems, so I'll provide an example.

HP: 6,726>#look
MrAngry arrives, looking for a fight
The Old Throne Room
The moon is radiant tonight, providing enough light to see without torchlight.
This appears to have been the throne room, before Castle Redcloud was destroyed in the Great Thaumatic Conflagration. A ruined dais can be seen along the north wall, along with remnants of a great marble throne. Piles of rubbish are strewn about, and appear to have lain undisturbed for millenia. Your amulet is glowing softly in the misty air.
There is a shattered gravestone here.
A silvery portal hangs in the air.
Exits: south, west, east.
FriendlyMcBing tells you: Hey, got any spare gold?
HP: 6,726>

The room description needed for the automap lies in there, but its content and position are unpredictable, and the server won't let you change its color. Info about the moon only appears at night, outside. You know the description lies between the blue and green lines, and that it's the longest line, but it could be the 1st, 2nd, or 3rd line after the title. For extra fun, let's say you don't want to reserve blue for room titles alone.

How to do this with a multi-state trigger? You can't just capture the first three lines after every blue line and compare their length. Maybe with onRoomCreate you could weed out most of the insidious, occasional blue intruders, but every now and then one that's not a title slips in to your description-finding trigger. You could constantly watch the Room Properties window to find errors. Or you could look for another solution

You could make a different multi-state trigger for each possible number of lines between the blue and green. Assuming, that is, that there is a dependable maximum number of lines. Not reliable enough? One regex can do it. Capture all the lines between blue and green, using a non-greedy modifiers for all lines after the third. You don't need to know the maximum number of lines. If there are 3 lines or less, the description is the last. If there are more, the description is the longest of the first three. Oh, and you can weed out false-positive blue lines with the same regex. If you can come up with some clever or informed analysis techniques, you can also deduce the non-dynamic room contents with the same regex.

As far as I know, you can't do that with a multi-state trigger. If I'm mistaken, I hope someone will let me know.

Sorcerer Joined: 15 Apr 2011 Posts: 809

I think you can... you can make a trigger to match the blue room name line (which I believe you already have), then a second state with pattern ^(*)$ (matching everything), set to manual type. In that second state, check if %1 (the whole line) is the green exits line - %match would be good for that. If it is, reset the trigger with #state 0 (and sum up your analysis stuff here, #tag the room desc, etc). If it's not, do your analysis stuff, try to figure out what that line is using whatever method you already are in your regex system. But really, you seem to have your code working well with your really complex regex, why not just stick with it since you've already done the work? You're not being forced to use multistate triggers...

Also, have we gotten a little off topic here? Where does the MXP come in?

Wanderer Joined: 24 May 2005 Posts: 71

I'm not sure I see how you're capturing all the lines between blue and green with your two-state trigger, though I do also see a near-solution which may be rather similar to what you meant. If the first state matches the blue line, and the second state is the "Loop Lines" type which matches any non-colored line, and N is set greater than the number of description lines can possibly be .... You'd get every set of lines between blue and any other color, which would would include all sets of description lines, which could then be distinguished by code. You do need to know the maximum number of description lines here, or be comfortable setting N to an absurdly high number and accept that you may be processing a large share of all output lines to get the room descriptions.

I have, in fact, managed to do what I need with a regex, and I don't mean to highjack the thread. Mnem seemed to be troubled by the inability to capture whole lines with an MXP match, unlike an ANSI match, and perhaps he also found the choice between "MXP" and "Within 1" an obstacle in multi-states, whereas there's no conflict between "ANSI trigger" and "Within 1". He originally posted about his inability to match MXP tags and his desire to match them with regex. I gathered that he was concerned about his choice between single-tag matches, with reparse multistates that didn't work like he wanted them to, and multiline ANSI matches that didn't allow him to match all the colors in his output.

I'll think he will have to chime in to keep the thread on track - maybe he will report back on his effort to resolve his MXP pattern problem, or provide an example that sheds more light on his particular challenge. I shared my regex solution with him, and he seems to be quite good with regexes, while I had to struggle to write mine, so he might understand mine right away, improve on it, and adapt it to what he needs. In any case I'll hold my tongue until he makes it clear what he still needs.

Wanderer Joined: 24 May 2005 Posts: 71

Oh, I see my confusion about your solution. I misunderstood Manual trigger types. Now I understand, and see that your solution works without concerning itself with the maximum number of description lines. It still runs the risk of unnecessarily processing a great deal of output, depending on how frequently blue appears in the output.

Beginner Joined: 30 Jul 2011 Posts: 16

Indeed, I started this thread to get help with MXP tags. After reviewing everything said here, going over the documentation, and looking again at the raw server output, I figured out the problem I was having with MXP triggers: the MXP tags that I was having trouble matching have no closing tags. So it's "Prompt|nocr" to the rescue with a trigger of type MXP. It seens clear from Zugg's comment that my original goal of matching MXP tags in regular expressions (aside from the catchall ESC[5S as an ansi code) is impossible. I wanted to use regexes instead of MXP triggers because of the power of the regex engine and syntax, in particular for capturing consecutive runs of lines that contain multiple MXP color tags. Because regexes are out for this, that leaves multistate triggers, which is why I brought them up.

Hazram's example isn't quite as challenging as what I need to deal with, but at this point another example isn't really needed. The solutions offered here are instructive about multistate triggers, and I think I can see a way to do what I need with multistate triggers and MXP tags, though it would involve a fair bit of reparsing to work with multiple colors in a line. The reparse feature that allows the next state even if the current state fails is a bit of an obstacle, I do wonder why that's not optional. Using a multistate would also, I think, mean more processing than would be required with a regex. On the whole I think I'm better off going with ansi color and regexes.

Regarding regexes, Hazram you weren't quite right in your solution description above. One great thing about the Perl regex engine is infact its customizability with non-greedy and possessive modifiers, but using non-greedy modifiers as you suggested would mean you'd capture only the first three lines above. Any room contents lines would be ignored. To get all lines for use in a script you'd need possessive modifiers, and to get maximum flexibility you'd need neither.

Thanks to all who posted and helped me figure out my options.