|
Zugg MASTER
Joined: 25 Sep 2000 Posts: 23379 Location: Colorado, USA
|
Posted: Thu Jul 05, 2007 10:50 pm
New CMUD Feature: Sequential Scripting Threads! |
OK, I think this is pretty dang exciting...This is something that people have been asking for in zMUD since I first wrote it. But zMUD was never designed to handle this. When I designed CMUD from the ground up, I tried to make sure it was more modular and threadsafe. This new design has paid off, and now I can finally announce support for the kind of "sequential" scripting that people have wanted for a long time.
What is "sequential scripting"? An example is probably worth a thousand words:
Code: |
#ECHO "Starting script"
#WAITFOR "pattern from MUD"
#ECHO "Pattern from MUD received"
#WAIT 5000
#ECHO "5 seconds has elapsed" |
Yep, that's right, we are talking about multi-threaded scripting in CMUD. Everytime you execute a script in the new 2.0 version of CMUD, a background thread is created. This background thread can be stopped for any reason, such as waiting for a pattern from the MUD, or waiting for a timer to expire. While this background thread is paused, the rest of CMUD runs normally as expected. You can have as many background threads running as you want. The new #THREAD command can be used to display all of the running threads and their status.
Each thread is associated with a particular Window. If the script was entered from the command line, the thread is marked as a "user" thread. Otherwise it's a thread created by something else (like a trigger that fired). If you press the ESC key on the command line, all of the "user" threads associated with that window are terminated. If a window closes, all threads associated with that window are terminated.
This all works transparently. I'll eventually add some more features to allow you to control threads within a script (suspend, resume, terminate, etc). But for now you shouldn't notice this change, except for the fact that the #WAIT command will now WORK as expected (finally!) and you have the new #WAITFOR command that waits for a pattern from the MUD.
When using #WAITFOR, any named subpatterns will properly set local variables within the script. The #ABORT command can also be used to terminate a thread, just like you'd expect.
I think this is pretty cool stuff. CMUD now combines the best of both worlds: event-driven scripts, and sequential scripts. This should make it a lot easier for beginners to start scripting with CMUD, and it will probably offer a lot of ways for existing users to simplify their scripts.
I decided to implement this now because v2.0 will have a lot of other new features, and this is the time to beta test and debug big changes like this. I'm sure there will be problems and bugs with it. Synchronizing multiple threads is always tricky and complicated. But so far my testing is working pretty well.
I'm sure everyone wants me to release this right now, but I still have a lot more stuff to add to v2.0, so you'll just need to be patient for a few weeks. Try not to drool too much :) Between this new feature, the Lua support, and some of the other new stuff in v2.0, I'm hoping that this will finally give people some good reasons to switch from zMUD to CMUD. |
|
|
|
oldguy2 Wizard
Joined: 17 Jun 2006 Posts: 1201
|
Posted: Thu Jul 05, 2007 10:56 pm |
Holy cow...I will have to rewrite everything now. This is great! Do you realize how easy this will make everything? Well of course you do. Anyway, I am looking forward to this release for sure.
|
|
|
|
Fang Xianfu GURU
Joined: 26 Jan 2004 Posts: 5155 Location: United Kingdom
|
Posted: Thu Jul 05, 2007 11:03 pm |
You sure are packing the features into this version!
In a couple of subversions, when the bugs have been stamped out, people should be flocking to CMUD in droves! |
|
|
|
nexela Wizard
Joined: 15 Jan 2002 Posts: 1644 Location: USA
|
Posted: Thu Jul 05, 2007 11:06 pm |
*drool*
One small problem that comes to my mind (correct me if wrong) would be like in Zmud
Trigger A starts
Trigger B starts (but counts on stuff from trigger a which isn't done)
Trigger B stops
Trigger A stops
Which shouldn't be a problem if your using multistate triggers :p and this changed combined with multistates should be pretty powerfull |
|
|
|
oldguy2 Wizard
Joined: 17 Jun 2006 Posts: 1201
|
Posted: Thu Jul 05, 2007 11:08 pm |
By the way, I am confused. You said you decided to implement it now for beta testing but you aren't going to release it until the 2.0 version? Maybe I just misunderstood. I certainly would like to try this out right now.
|
|
|
|
Fang Xianfu GURU
Joined: 26 Jan 2004 Posts: 5155 Location: United Kingdom
|
Posted: Thu Jul 05, 2007 11:17 pm |
By "now" he means "in this version rather than in a future version". No doubt 2.0 will be a beta version.
|
|
|
|
oldguy2 Wizard
Joined: 17 Jun 2006 Posts: 1201
|
Posted: Thu Jul 05, 2007 11:19 pm |
Fang Xianfu wrote: |
By "now" he means "in this version rather than in a future version". No doubt 2.0 will be a beta version. |
Ahhh well darn I want to play with it right now! hehe |
|
|
|
Zugg MASTER
Joined: 25 Sep 2000 Posts: 23379 Location: Colorado, USA
|
Posted: Thu Jul 05, 2007 11:42 pm |
Nexela: as long as you don't use #WAIT for #WAITFOR (or any other command that is added to control the thread), then it will still work the same way as the current version. In other words, triggers still wait for themselves to finish before going to the next trigger (unless the trigger thread is suspended for some reason).
So yes, if you put #WAIT in Trigger A, then Trigger B is going to run before Trigger A is finished. This is usually how you want it to work. You never want a long wait in one trigger to stop all processing in all other triggers too...that just causes horrible lag.
Over time, I expect I'll add some features to allow synchronization. Most threading systems offer these kinds of features. Using some sort of synchronization commands, you could write Trigger B so that it waits for Trigger A to finish, even if Trigger A has a WAIT command in it. I just need to figure out what sort of syntax I want to use for that.
Since my last post, I went ahead and added some other commands:
- #THREAD name
- When the #THREAD command is given an argument, this becomes the "name" of the thread (instead of just having a unique number for the thread)
- #SUSPEND name
- In addition to suspending alarm triggers, this command also now looks for a thread of the specified name and will suspend the thread
- #RESUME name
- In addition to resuming alarm triggers, this command also now looks for a thread of the specified name and will resume that thread (if it was suspended)
- #STOP name
- When an argument is given to the #STOP command, the thread that matches the name is terminated. If no argument is given, then it stops the current speedwalk as normal
If anyone has ideas on the command names to use for synchronizing threads, let me know. |
|
|
|
Zugg MASTER
Joined: 25 Sep 2000 Posts: 23379 Location: Colorado, USA
|
Posted: Fri Jul 06, 2007 1:41 am |
Hmm, having a bit of trouble with #WAITFOR setting local variables. The whole concept of local variables is that they are defined at COMPILE time. The compiler reserves space on the local call stack for them, and generates code to set/retrieve these stack values.
Well, #WAITFOR happens in the middle of a script, and the compiler does not parse the pattern field (it doesn't have the cache to store the compiled data like a trigger does...it's just a script line). So, if you reference the local variables in the pattern later in the script, the script doesn't compile ("unknown local variable") because the compiler doesn't see the local variables in the #WAITFOR pattern.
A trigger is a bit different. A trigger creates a whole new code "block" and any local variables set in the trigger pattern are active within this code block. But #WAITFOR doesn't create a new code block...it just suspends the thread until the pattern is detected.
So, I'm going to have to think about the best way to handle this. I want to make it easy to extract data from a #WAITFOR trigger and need to think of something that is easy and makes sense syntax-wise, but that is also something relatively easy to implement. I think I'll sleep on it.
Except for that minor detail, everything else with sequential scripting is now implemented and seems to be working. |
|
|
|
Thinjon100 Apprentice
Joined: 12 Jul 2004 Posts: 190 Location: Canada
|
Posted: Fri Jul 06, 2007 1:44 am |
At compile time, couldn't you look through the #WAITFOR command for any variable references and place implied declarations for those variables into the compiled version of the script, just before the WAITFOR line itself?
As an aside... will there be an option for WATIFOR patterns to be pure regex, as opposed to zMud-style patterns? |
|
_________________ If you're ever around Aardwolf, I'm that invisible guy you can never see. Wizi ftw! :) |
|
|
|
shalimar GURU
Joined: 04 Aug 2002 Posts: 4692 Location: Pensacola, FL, USA
|
Posted: Fri Jul 06, 2007 2:11 am |
Wouldnt #WAITFOR just be a fancy name for #TEMP?
|
|
_________________ Discord: Shalimarwildcat |
|
|
|
Zugg MASTER
Joined: 25 Sep 2000 Posts: 23379 Location: Colorado, USA
|
Posted: Fri Jul 06, 2007 2:29 am |
shalimar: In a way...there was always a way to do this stuff before. Either using temp triggers (instead of #waitfor) and using alarms (instead of #wait). The whole point of this is that you don't need to do that anymore. No more extra temp triggers and alarms sitting around, and your code becomes easier to read and follow.
Old way with temp triggers:
Code: |
#EVENT onConnect {
#TEMP {Username:} {
#CHAR
#TEMP {Password:} {#PW}
}
} |
New way with sequential scripts:
Code: |
#EVENT onConnect {
#WAITFOR {Username:}
#CHAR
#WAITFOR {Password:}
#PW
} |
In the first case you ended up with 2 temporary triggers. In the second case you don't need any temp triggers. Since creating triggers is relatively time consuming, the second version runs much faster. Also, because this is implemented with true threads instead of just playing games with the Windows Message queue, it's much cleaner.
Thinjon100: I've been trying to think of a better way to handle pure regex. I really hate the current trend in CMUD to create a second version of every command/function just for RegEx. Seems like I need a better way to specify regular expression syntactically. For example, someone has previously suggested using a pattern of "/regex/" to specify a regular expression instead of a normal trigger pattern. With a syntax like that, you could just use the normal commands and functions. It's something I'm definitely considering. I do *not* want to create a "#WAITFORREGEX" command. That just seems messy.
Also, the problem that I mentioned above is that the pattern is not known at compile time. Imagine doing "#WAITFOR @Pattern" where @Pattern contains the pattern. There is no way to parse this at compile time. So that's why I need a different way to handle local variables in this case. |
|
|
|
MattLofton GURU
Joined: 23 Dec 2000 Posts: 4834 Location: USA
|
Posted: Fri Jul 06, 2007 2:32 am |
Sounds like this, coupled with the customized interface thing you mentioned, will make a lot of Simutronics users cry great crocodile tears of joy. All the power of ZMud-like scripting languages plus the builtin goodness of their custom FEs.
All that's left to capture the market is a plugin or customizable feature to process various plain-text script languages (more like VB, without the conditional compiler functions like #IF, than zscript). One that actually translates such things and builds the CMud equivalent automatically would be amazingly better (the steeper learning curve loses all importance, becoming a leisure activity rather than a required one). |
|
_________________ EDIT: I didn't like my old signature |
|
|
|
Fang Xianfu GURU
Joined: 26 Jan 2004 Posts: 5155 Location: United Kingdom
|
Posted: Fri Jul 06, 2007 2:45 am |
Do you mind clarifying that second point, Matt? I got a bit confused when you mentioned VB - from there it sounded like you were just talking about WSH. Thanks.
|
|
|
|
Thinjon100 Apprentice
Joined: 12 Jul 2004 Posts: 190 Location: Canada
|
Posted: Fri Jul 06, 2007 6:28 am |
Zugg: I like the idea of an inline Regex pattern... that would make a lot of things very nice to use, especially something like inline comparison "#IF (%i ~= /myregex/)".
Now, I'm a relative newbie in matters of actually designing compilers, but would it be plausible to modify the script compiler such that if the script contains an undeclared local variable AFTER an inline #WAITFOR, it doesn't trigger an undeclared variable at compile time, but instead "declares" the variable just before the #WAITFOR command... perhaps with a null value? Unless I'm mistaken, that would cover all possible instances of local variables being assigned (or not assigned) by #WAITFOR triggers, with the added benefit of a simple #IF (%null($myundeclaredlocalvar)) check to determine if the local variable was actually assigned to by the regex (great in instances where you /ARE/ using a @pattern variable, and you're not exactly sure WHAT it's going to be storing).
Again, I don't know much about compiler design, but theoretically I think it might help... just an idea :) |
|
_________________ If you're ever around Aardwolf, I'm that invisible guy you can never see. Wizi ftw! :) |
|
|
|
Thinjon100 Apprentice
Joined: 12 Jul 2004 Posts: 190 Location: Canada
|
Posted: Fri Jul 06, 2007 6:37 am |
Quick questions to toss into this, Zugg...
Assume the following trigger:
Code: |
#ONINPUT {^} {
#VAR myvar somevalue
#VAR dosome otherstuff
#WAIT @myduration
#VAR do somethingelse
}
|
A) Would the @myduration variable be allowed? (Can't see why not, but figured I'd ask anyway)
B) This is an oninput that fires on ANY input to the mud... assuming @myduration is something like 10 minutes, one could assume that sending a command and waiting 10 minutes would execute the remainder of the code (do somethingelse)... now, if I send another command (thus triggering this oninput) within that 10 minute timeframe, does this destroy the earlier-created #WAIT thread, as it's the same trigger initializing it, or are there now two duplicates resident in memory. If the latter, what options would be available for identifying and eliminating other threads originating from the same trigger.
I read your above post with the #THREAD and assorted commands, but I'm not sure how they would play into this... will there be optional second arguments for #WAIT and #WAITFOR that will be the thread names? Or will I have to use #THREAD somehow in order to find and rename those? If I create a #WAIT with a thread name, and then create another with the same name, does that #STOP and overwrite the earlier copy, or will my creation fail?
Sorry if I seem to pester, but I was considering some of my own scripting ideas that might use this (wonderful) feature, and would like to cover all the bases while it's still in development and... malleable, as it were. :) |
|
_________________ If you're ever around Aardwolf, I'm that invisible guy you can never see. Wizi ftw! :) |
|
|
|
Caled Sorcerer
Joined: 21 Oct 2000 Posts: 821 Location: Australia
|
Posted: Fri Jul 06, 2007 7:42 am |
Will it be possible to #WAITFOR expression ?
script
#WAITFOR @var>1
more script |
|
_________________ Athlon 64 3200+
Win XP Pro x64 |
|
|
|
Fang Xianfu GURU
Joined: 26 Jan 2004 Posts: 5155 Location: United Kingdom
|
Posted: Fri Jul 06, 2007 11:39 am |
Thinjon100 wrote: |
A) Would the @myduration variable be allowed? (Can't see why not, but figured I'd ask anyway) |
It is already, and I don't see why he'd remove something like that :)
Thinjon100 wrote: |
If the latter, what options would be available for identifying and eliminating other threads originating from the same trigger. |
If the answer to this isn't that the old thread is overwritten, this sounds like a job suited for the %alarm function anyway.
Thinjon100 wrote: |
I read your above post with the #THREAD and assorted commands, but I'm not sure how they would play into this... will there be optional second arguments for #WAIT and #WAITFOR that will be the thread names? Or will I have to use #THREAD somehow in order to find and rename those? If I create a #WAIT with a thread name, and then create another with the same name, does that #STOP and overwrite the earlier copy, or will my creation fail? |
The impression I got was that #wait and #waitfor don't create new threads - they just pause the execution of the current thread. It's the act of calling the trigger or alias that creates new threads.
The #thread command, I'm guessing, will name whatever thread calls the command. |
|
|
|
Zugg MASTER
Joined: 25 Sep 2000 Posts: 23379 Location: Colorado, USA
|
Posted: Fri Jul 06, 2007 4:51 pm |
Quote: |
Will it be possible to #WAITFOR expression ? |
No, not at this time. You can already do this with an expression trigger, or by creating a simple loop in your script:
Code: |
#UNTIL (@var>1) {#WAIT 10} |
Since CMUD v2.0 uses threads, this example works well and doesn't hang the system or cause much of a performance problem. You can adjust the #WAIT time to whatever make sense as a polling interval. Without the #WAIT command, this would be a tight loop that would use all of the CPU power. But with the #WAIT added, it works great. Even using "#WAIT 1" works pretty well.
This is also a good example of how using threads is a bit different than just using an alarm. An embedded #WAIT command in a loop has never worked well in CMUD before, and now it works just like you'd expect it to.
Thinjin: A) Yes, you can use @myduration just like you'd expect
B) A new thread is created each time. So when you enter another command, the #ONINPUT trigger runs in a new thread. So your previous thread is not disturbed and works just fine. You can use the #THREAD command to list all of the running threads. Even if you assign a name to a thread, it still has a unique number. You can use the "#STOP threadid" command to stop a thread (referenced by either unique id number or by name). You can assign the same name to multiple threads...that doesn't cause the thread to abort or anything. But if you have multiple threads with the same name, they you would need to reference them by ID number to stop them.
There is a new function called %threadid(name) that will return the ID number of a named thread. If the "name" is not give, it returns the unique ID number of the current thread. There is also a function called %threadname(id[,newname]) that returns the name of a given thread number, and optionally allows you to set a new name for that thread. This allows you to name a thread outside of the thread itself (the "#THREAD name" command only renames the currently running thread).
C) (About using /regex/ syntax): Starting a new topic about this so that we can keep this topic focused on threads. |
|
|
|
Thinjon100 Apprentice
Joined: 12 Jul 2004 Posts: 190 Location: Canada
|
Posted: Fri Jul 06, 2007 6:04 pm |
Thanks for the response, Fang and Zugg. That's awesome.
I feel silly now that I think about it. For some reason I thought #WAIT and #WAITFOR were responsible for creating the thread... didn't even cross my mind that the trigger code itself would be an independent thread.
This does, however, bring to mind an interesting "problem". Given a trigger set that executes over a list and has some cumulative effect (let's assume something simple like counting the instances of items in a room's object list)... you would want each trigger to run in series, rather than parallel, as you'd most-likely be processing and entering data into one variable, which might get corrupted or reflect bad data if it had 30+ threads trying to write to it simultaneously. Will there be some sort of syntax we can put into a trigger to tell it to effectively #WAIT until thread (id|name) is done executing, or to wait until all previous threads from the same trigger are done? I think this may play somehow into your thread synchronization commands, but I'm not sure.
Just tossing it out there, but if your thread has to wait for another to finish executing, how about "#WAITTHREAD {id|name}" ? Fits with the other two wait commands... perhaps with an optional argument to determine whether it fires simply on the target thread's STOP, or if it also fires on the target thread's call of #WAITTHREAD... which would make back&forth independent threads work quite well...
i.e.
A starts executing
B start executing
B, seeing A executing, suspends itself with #WAITTHREAD {a} {suspend}
A does some processing
A suspends itself with #WAITTHREAD {b} {suspend}
B "wakes up", does some processing
B suspends itself again with #WAITHTHREAD {a} {suspend}
....
repeat as desired. |
|
_________________ If you're ever around Aardwolf, I'm that invisible guy you can never see. Wizi ftw! :) |
|
|
|
Zugg MASTER
Joined: 25 Sep 2000 Posts: 23379 Location: Colorado, USA
|
Posted: Fri Jul 06, 2007 6:50 pm |
Yep, "#WAITTHREAD threadid" is exactly what I'm going to do. Also, keep in mind that triggers will still execute sequentially, just like in past versions, until you use a command that suspends the thread (like #WAIT, #WAITFOR, etc). When a thread is suspended, the trigger processing then continues to the next trigger in the list. So, for normal existing triggers, you won't see any change. You only need to worry about synchronization if you start using the new WAIT commands.
And yes, #WAITTHREAD has an argument to specify whether to wait for the thread to be finished, or just wait until it gets suspended again (via any #WAIT command, etc). So you can definitely do "back & forth" threads like this if you want.
The other synchronization method that I'm adding is something called "signals". You can do "#SIGNAL signalname" and in some other thread(s) you can do "#WAITSIGNAL signalname". This allows complete flexibility on synchronizing threads. You can have several threads waiting for a single signal, and then fire that signal from somewhere else. If multiple threads were waiting on the signal, then they will all start executing as simultaneous background threads. This will be one way that you can actually cause multiple threads to run simultaneously if you want that. Windows will be controlling the thread execution, so you won't be able to specify the order in which the threads run...Windows will timeslice between them all.
So, I think this gives you all of the power you could possibly want. It gives you ways to control threads and execute them in specific order (with the WAITTHREAD), but also methods to synchronize and run some threads in parallel. Pretty powerful stuff! |
|
|
|
Thinjon100 Apprentice
Joined: 12 Jul 2004 Posts: 190 Location: Canada
|
Posted: Fri Jul 06, 2007 7:29 pm |
Well, all I have left to say is:
*worship Zugg* |
|
_________________ If you're ever around Aardwolf, I'm that invisible guy you can never see. Wizi ftw! :) |
|
|
|
Zhiroc Adept
Joined: 04 Feb 2005 Posts: 246
|
Posted: Fri Jul 06, 2007 10:36 pm |
As for synchronization methods... you'll need a mutex (or semaphores) to protect critical regions. In fact, you can't reliably do signals without a mutex.
#if (something) {#WAITSIGNAL xxx}
is not safe in that "something" can become false (the other thread exits, for example) between the test and the #WAITSIGNAL leading to an infinite wait.
In fact, to deal with wait/wakeup code, you probably need to support the following construct:
Code: |
#MUTEX @var // waits until you are granted exclusive access to @var
#WHILE (test) {
#WAITSIGNAL sig @var // gives up the @var MUTEX, and when resumed, you are granted it again
}
#UNMUTEX @var |
The reason for the WHILE is that #SIGNAL generally wakes up all threads waiting on a signal, so you can't be sure whether you can assume that you can proceed without retesting your conditional.
In fact while I love threads, thread programing is an order of magnitude harder to get right. And when it goes wrong, the problems are an order of magnitude harder to find.... |
|
|
|
Tech GURU
Joined: 18 Oct 2000 Posts: 2733 Location: Atlanta, USA
|
Posted: Sat Jul 07, 2007 12:03 am |
Wow.. all this thread talk is making drool and forcing me to think of new ways to do all this cool stuff. Still waiting on functions though
Quick question can I have a SIGNAL be an EVENT or will I have have an EVENT that sends the SIGNAL? |
|
_________________ Asati di tempari! |
|
|
|
Zugg MASTER
Joined: 25 Sep 2000 Posts: 23379 Location: Colorado, USA
|
Posted: Sat Jul 07, 2007 12:13 am |
Zhiroc: That's why I'm trying to simplify things. I don't want CMUD users to need to worry about true synchronization. As you said, that gets really complicated. Yes, in theory people could end up with problems if they have multiple threads running, all changing the same data. But I'm not going to make this really messy by adding stuff like mutexes and semaphores. I'm just trying to provide the simple cases to allow the most common stuff to work properly. I'm not going to support full robust thread programming...it just doesn't make much sense to do that in any scripting language. CMUD already has the critical section protection so that if two threads try to access the same CMUD variable (or other setting), it takes proper turns.
Yes, I know that thread programming is complicated. Trust me, I've been pulling my hair out the last few days dealing with some obscure thread lockup problems. And when you get "stuck" threads in the background, it can be tough to get rid of them without crashes. But that's all my problem to solve. The end-user shouldn't need to worry about complicated stuff in most cases.
Tech: The Signals probably will not be events. But yes, you will be able to issue a #SIGNAL command from an event if you'd like (same as from anywhere else). Should have functions added next week. |
|
|
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|