Zugg Software :: View topic

Wizard Joined: 11 Jul 2002 Posts: 1265 Location: USA

Shalimar try matching the first item in your list in the test box when using the non-REGEX pattern; it fails.

It thinks that the first item is "(item1" instead of just "item1"

Posted: Tue May 06, 2008 1:37 pm

aha, i see what you mean, its ignoring the function call

Posted: Tue May 06, 2008 6:23 pm

When CMUD converts a Database variable in a trigger pattern, it *does* strip out the values. So my conversion example is still correct. So it *should* be the same as matching a string list. But obviously it isn't.

I wonder if your variable is marked as a Database Record type, or if it is AutoTyped. That might make a difference. But I'll try to get a chance to look at this to see why it's slower for database variables when it shouldn't be.

Posted: Tue May 06, 2008 11:18 pm

it is set as database record, but i assume it did that on its own, as i never touched the dropdown selection box

Posted: Wed May 07, 2008 12:59 am

OK, I think I found the problem. It wasn't slow unless it needed to recreate the trigger pattern. So somewhere in your scripts you are causing your database variable to think it has been changed. When the list was changed, then I saw the 10 second delay. Turns out it was creating the trigger from the hash table in a very slow manner. I changed this in 2.24 and now it is very fast for me. There is an additional optimization that I can do in the future for both database variables and string lists, but I'm leaving that for a future version because it might have more side-effects.

Posted: Wed May 07, 2008 1:11 am

Glad to hear it.

You rock man.

Posted: Wed May 07, 2008 1:45 am

Actually, your test was very useful for me to play with. I've never worked with triggers using string lists of that size. And while playing around with it, I realized that I *really* needed some way to search for keys/values in the Variables editor. So I've added a Filter box to v2.24 at the top of the Variables editor (to the right of the Name field). You can type a value into this box when viewing a string list or database variable to filter the display to only include keys or values that contain the filter text.

You can anchor the filter text using a ^ at the beginning of $ at the end (like a trigger pattern), otherwise it will assume wildcards. For example, just typing "z" into the filter will display any enter in the list that contains the letter "z" in the key or value fields.

It was quick and easy and makes it easier to find a specific value in the list that you want to edit.

Merry Christmas (in May) Very Happy

Posted: Wed May 07, 2008 1:58 am

Now that is sweet.
Any chance we can call that filter from outside the settings editor?

Posted: Wed May 07, 2008 2:22 am

Umm, no, not at all, sorry. The grid view that is shown in the settings editor is controlled by a memory Dataset, and it's just setting the Filter property of that dataset. The dataset doesn't exist outside of the settings editor.

Besides, I'm not really sure what you are asking for and why such a filter would be useful outside of the settings editor. But if you have some application for this, create a new post with a feature request and explain what you are trying to do.

But the purpose of the filter is to make the editing of the variable easier, and that doesn't make much sense outside of the settings editor.

Posted: Wed May 07, 2008 2:31 am

Yeah, what do you mean, making it useful outside the settings editor? You mean when you refer to @MySkills you only get a subset of the data? That could have some pretty bad sideeffects.

You could write your own function to return only the items you're interested in from a given variable and a given search string; it wouldn't be too hard.

Posted: Wed May 07, 2008 3:50 am

While we're on the topic I do the same type of highlighting and I was wondering if there was any more efficient way of doing this than what I have already? Each of the variables is a string list with thousands of names:

Look good? Or can I tweak some more speed out of this somehow?

Posted: Wed May 07, 2008 4:03 am

Yep, for regular expressions, that is fine. Although it's more correct to use (?@varname) if you don't need to access the resulting subpattern using %1..%99. I doubt that makes much difference in speed though.

Oh, and of course this will only work with string lists and *not* with database variables. PCRE won't know how to deal with the key=value syntax in database variables, so you *must* use a normal CMUD trigger pattern to let CMUD strip the values and just use the keys. Otherwise you'd need to use %dbkeys(@varname) or something like that.

Btw, you should keep in mind that when you do this with RegEx patterns, you are passing the value of the variables *directly* to the regular expression engine. If you use normal CMUD trigger pattern, CMUD will actually sort the list. This improves the performance of the regular expression engine, and also avoids some difficulties when you have multiple entries in your string list that start with the same substring. PCRE likes to have the list of values sorted.

So this is one of those cases where you are better off using the CMUD Trigger pattern syntax rather than making a RegEx trigger yourself. Same is true when using database variables. You can get around this by using the #SORT command on your variables.

Posted: Wed May 07, 2008 4:29 am

Good points. I'll add in the (?@varname).

So if I understood correctly, to optimize performance I either need to sort my stringlists if I intend to continue to use PCRE or I can use CMUD Trigger patterns and not worry about them being sorted.

If I were to compare CMUD trigger patterns and PCRE (on sorted string lists) would there be a performance difference? Is one inherently faster than the other for this type of work?

The only other way of doing this that occurred to me was to have a single trigger with "([A-Z][a-z]+)". Then inside the trigger I'd have switch statement:
#switch (%ismember(%1, @stringlist1) {#cw bla...}
(%ismember(%1, @stringlist2) {#cw bla...}
(%ismember(%1, @stringlist3) {#cw bla...}

This way has these advantages:
- Only one trigger
- Narrow the matching of the one trigger with [A-Z][a-z]+ since I know the form of a name I'll match on less garbage.

Disadvantages
- When the one trigger does match it has a lot more work to do than the previous triggers.

Posted: Wed May 07, 2008 6:35 am

Well, CMUD sorts your string list and then calls PCRE. So CMUD is always still using PCRE, but using CMUD trigger syntax saves you the trouble of remembering to sort the list. In general there isn't any reason to use RegEx triggers instead of normal CMUD triggers unless there is something really fancy that you are doing where you need to find-tune the regular expression yourself, or use some of the lookahead regular expression operations. Unless you are a real RegEx expert, it's pretty much always better to stick with the CMUD trigger syntax.

Using a string list in a trigger is MUCH more efficient than using the [A-Z][a-z]+ and then using the #switch statement. PCRE is a highly optimized library and I think it can do the testing faster than the zScript #SWITCH statement can.

Of course, you can always just test it yourself...that's the best way to learn what is best for your specific circumstance.