Zugg Software :: View topic - New CMUD Feature: Sequential Scripting Threads!

Posted: Thu Jul 05, 2007 10:50 pm

OK, I think this is pretty dang exciting...This is something that people have been asking for in zMUD since I first wrote it. But zMUD was never designed to handle this. When I designed CMUD from the ground up, I tried to make sure it was more modular and threadsafe. This new design has paid off, and now I can finally announce support for the kind of "sequential" scripting that people have wanted for a long time.

What is "sequential scripting"? An example is probably worth a thousand words:

Wizard Joined: 17 Jun 2006 Posts: 1201

Holy cow...I will have to rewrite everything now. This is great! Do you realize how easy this will make everything? Well of course you do. Anyway, I am looking forward to this release for sure.

Posted: Thu Jul 05, 2007 11:03 pm

You sure are packing the features into this version!

In a couple of subversions, when the bugs have been stamped out, people should be flocking to CMUD in droves!

Wizard Joined: 15 Jan 2002 Posts: 1644 Location: USA

*drool*

One small problem that comes to my mind (correct me if wrong) would be like in Zmud

Trigger A starts
Trigger B starts (but counts on stuff from trigger a which isn't done)
Trigger B stops
Trigger A stops

Which shouldn't be a problem if your using multistate triggers :p and this changed combined with multistates should be pretty powerfull

Wizard Joined: 17 Jun 2006 Posts: 1201

By the way, I am confused. You said you decided to implement it now for beta testing but you aren't going to release it until the 2.0 version? Maybe I just misunderstood. I certainly would like to try this out right now.

Posted: Thu Jul 05, 2007 11:17 pm

By "now" he means "in this version rather than in a future version". No doubt 2.0 will be a beta version.

Wizard Joined: 17 Jun 2006 Posts: 1201

Posted: Thu Jul 05, 2007 11:42 pm

Nexela: as long as you don't use #WAIT for #WAITFOR (or any other command that is added to control the thread), then it will still work the same way as the current version. In other words, triggers still wait for themselves to finish before going to the next trigger (unless the trigger thread is suspended for some reason).

So yes, if you put #WAIT in Trigger A, then Trigger B is going to run before Trigger A is finished. This is usually how you want it to work. You never want a long wait in one trigger to stop all processing in all other triggers too...that just causes horrible lag.

Over time, I expect I'll add some features to allow synchronization. Most threading systems offer these kinds of features. Using some sort of synchronization commands, you could write Trigger B so that it waits for Trigger A to finish, even if Trigger A has a WAIT command in it. I just need to figure out what sort of syntax I want to use for that.

Since my last post, I went ahead and added some other commands:

#THREAD name: When the #THREAD command is given an argument, this becomes the "name" of the thread (instead of just having a unique number for the thread)
#SUSPEND name: In addition to suspending alarm triggers, this command also now looks for a thread of the specified name and will suspend the thread
#RESUME name: In addition to resuming alarm triggers, this command also now looks for a thread of the specified name and will resume that thread (if it was suspended)
#STOP name: When an argument is given to the #STOP command, the thread that matches the name is terminated. If no argument is given, then it stops the current speedwalk as normal

If anyone has ideas on the command names to use for synchronizing threads, let me know.

Posted: Fri Jul 06, 2007 1:41 am

Hmm, having a bit of trouble with #WAITFOR setting local variables. The whole concept of local variables is that they are defined at COMPILE time. The compiler reserves space on the local call stack for them, and generates code to set/retrieve these stack values.

Well, #WAITFOR happens in the middle of a script, and the compiler does not parse the pattern field (it doesn't have the cache to store the compiled data like a trigger does...it's just a script line). So, if you reference the local variables in the pattern later in the script, the script doesn't compile ("unknown local variable") because the compiler doesn't see the local variables in the #WAITFOR pattern.

A trigger is a bit different. A trigger creates a whole new code "block" and any local variables set in the trigger pattern are active within this code block. But #WAITFOR doesn't create a new code block...it just suspends the thread until the pattern is detected.

So, I'm going to have to think about the best way to handle this. I want to make it easy to extract data from a #WAITFOR trigger and need to think of something that is easy and makes sense syntax-wise, but that is also something relatively easy to implement. I think I'll sleep on it.

Except for that minor detail, everything else with sequential scripting is now implemented and seems to be working.

Apprentice Joined: 12 Jul 2004 Posts: 190 Location: Canada

At compile time, couldn't you look through the #WAITFOR command for any variable references and place implied declarations for those variables into the compiled version of the script, just before the WAITFOR line itself?

As an aside... will there be an option for WATIFOR patterns to be pure regex, as opposed to zMud-style patterns?

Posted: Fri Jul 06, 2007 2:11 am

Wouldnt #WAITFOR just be a fancy name for #TEMP?

Posted: Fri Jul 06, 2007 2:29 am

shalimar: In a way...there was always a way to do this stuff before. Either using temp triggers (instead of #waitfor) and using alarms (instead of #wait). The whole point of this is that you don't need to do that anymore. No more extra temp triggers and alarms sitting around, and your code becomes easier to read and follow.

Old way with temp triggers:

GURU Joined: 23 Dec 2000 Posts: 4834 Location: USA

Sounds like this, coupled with the customized interface thing you mentioned, will make a lot of Simutronics users cry great crocodile tears of joy. All the power of ZMud-like scripting languages plus the builtin goodness of their custom FEs.

All that's left to capture the market is a plugin or customizable feature to process various plain-text script languages (more like VB, without the conditional compiler functions like #IF, than zscript). One that actually translates such things and builds the CMud equivalent automatically would be amazingly better (the steeper learning curve loses all importance, becoming a leisure activity rather than a required one).

Posted: Fri Jul 06, 2007 2:45 am

Do you mind clarifying that second point, Matt? I got a bit confused when you mentioned VB - from there it sounded like you were just talking about WSH. Thanks.

Apprentice Joined: 12 Jul 2004 Posts: 190 Location: Canada

Zugg: I like the idea of an inline Regex pattern... that would make a lot of things very nice to use, especially something like inline comparison "#IF (%i ~= /myregex/)".

Now, I'm a relative newbie in matters of actually designing compilers, but would it be plausible to modify the script compiler such that if the script contains an undeclared local variable AFTER an inline #WAITFOR, it doesn't trigger an undeclared variable at compile time, but instead "declares" the variable just before the #WAITFOR command... perhaps with a null value? Unless I'm mistaken, that would cover all possible instances of local variables being assigned (or not assigned) by #WAITFOR triggers, with the added benefit of a simple #IF (%null($myundeclaredlocalvar)) check to determine if the local variable was actually assigned to by the regex (great in instances where you /ARE/ using a @pattern variable, and you're not exactly sure WHAT it's going to be storing).

Again, I don't know much about compiler design, but theoretically I think it might help... just an idea :)

Apprentice Joined: 12 Jul 2004 Posts: 190 Location: Canada

Quick questions to toss into this, Zugg...

Assume the following trigger:

Posted: Fri Jul 06, 2007 7:42 am

Will it be possible to #WAITFOR expression ?

script
#WAITFOR @var>1
more script

Posted: Fri Jul 06, 2007 11:39 am

Posted: Fri Jul 06, 2007 4:51 pm

Apprentice Joined: 12 Jul 2004 Posts: 190 Location: Canada

Thanks for the response, Fang and Zugg. That's awesome.

I feel silly now that I think about it. For some reason I thought #WAIT and #WAITFOR were responsible for creating the thread... didn't even cross my mind that the trigger code itself would be an independent thread.

This does, however, bring to mind an interesting "problem". Given a trigger set that executes over a list and has some cumulative effect (let's assume something simple like counting the instances of items in a room's object list)... you would want each trigger to run in series, rather than parallel, as you'd most-likely be processing and entering data into one variable, which might get corrupted or reflect bad data if it had 30+ threads trying to write to it simultaneously. Will there be some sort of syntax we can put into a trigger to tell it to effectively #WAIT until thread (id|name) is done executing, or to wait until all previous threads from the same trigger are done? I think this may play somehow into your thread synchronization commands, but I'm not sure.

Just tossing it out there, but if your thread has to wait for another to finish executing, how about "#WAITTHREAD {id|name}" ? Fits with the other two wait commands... perhaps with an optional argument to determine whether it fires simply on the target thread's STOP, or if it also fires on the target thread's call of #WAITTHREAD... which would make back&forth independent threads work quite well...

i.e.
A starts executing
B start executing
B, seeing A executing, suspends itself with #WAITTHREAD {a} {suspend}
A does some processing
A suspends itself with #WAITTHREAD {b} {suspend}
B "wakes up", does some processing
B suspends itself again with #WAITHTHREAD {a} {suspend}
....
repeat as desired.

Posted: Fri Jul 06, 2007 6:50 pm

Yep, "#WAITTHREAD threadid" is exactly what I'm going to do. Also, keep in mind that triggers will still execute sequentially, just like in past versions, until you use a command that suspends the thread (like #WAIT, #WAITFOR, etc). When a thread is suspended, the trigger processing then continues to the next trigger in the list. So, for normal existing triggers, you won't see any change. You only need to worry about synchronization if you start using the new WAIT commands.

And yes, #WAITTHREAD has an argument to specify whether to wait for the thread to be finished, or just wait until it gets suspended again (via any #WAIT command, etc). So you can definitely do "back & forth" threads like this if you want.

The other synchronization method that I'm adding is something called "signals". You can do "#SIGNAL signalname" and in some other thread(s) you can do "#WAITSIGNAL signalname". This allows complete flexibility on synchronizing threads. You can have several threads waiting for a single signal, and then fire that signal from somewhere else. If multiple threads were waiting on the signal, then they will all start executing as simultaneous background threads. This will be one way that you can actually cause multiple threads to run simultaneously if you want that. Windows will be controlling the thread execution, so you won't be able to specify the order in which the threads run...Windows will timeslice between them all.

So, I think this gives you all of the power you could possibly want. It gives you ways to control threads and execute them in specific order (with the WAITTHREAD), but also methods to synchronize and run some threads in parallel. Pretty powerful stuff!

Apprentice Joined: 12 Jul 2004 Posts: 190 Location: Canada

Well, all I have left to say is:

*worship Zugg*

Adept Joined: 04 Feb 2005 Posts: 246

As for synchronization methods... you'll need a mutex (or semaphores) to protect critical regions. In fact, you can't reliably do signals without a mutex.

#if (something) {#WAITSIGNAL xxx}

is not safe in that "something" can become false (the other thread exits, for example) between the test and the #WAITSIGNAL leading to an infinite wait.

In fact, to deal with wait/wakeup code, you probably need to support the following construct:

Posted: Sat Jul 07, 2007 12:03 am

Wow.. all this thread talk is making drool and forcing me to think of new ways to do all this cool stuff. Still waiting on functions though Very Happy

Quick question can I have a SIGNAL be an EVENT or will I have have an EVENT that sends the SIGNAL?

Posted: Sat Jul 07, 2007 12:13 am

Zhiroc: That's why I'm trying to simplify things. I don't want CMUD users to need to worry about true synchronization. As you said, that gets really complicated. Yes, in theory people could end up with problems if they have multiple threads running, all changing the same data. But I'm not going to make this really messy by adding stuff like mutexes and semaphores. I'm just trying to provide the simple cases to allow the most common stuff to work properly. I'm not going to support full robust thread programming...it just doesn't make much sense to do that in any scripting language. CMUD already has the critical section protection so that if two threads try to access the same CMUD variable (or other setting), it takes proper turns.

Yes, I know that thread programming is complicated. Trust me, I've been pulling my hair out the last few days dealing with some obscure thread lockup problems. And when you get "stuck" threads in the background, it can be tough to get rid of them without crashes. But that's all my problem to solve. The end-user shouldn't need to worry about complicated stuff in most cases.

Tech: The Signals probably will not be events. But yes, you will be able to issue a #SIGNAL command from an event if you'd like (same as from anywhere else). Should have functions added next week.