 |
Zugg
MASTER
Joined: 25 Sep 2000
Posts: 23379
Location: Colorado, USA
|
 Posted: Tue Sep 26, 2006 12:15 am
CMUD 1.08 Development Blog
|
It's time for another Development Blog. I'm working in v1.08 now. In addition to fixing more bugs, this version is probably going to implement the Gauge stuff that many people have been waiting for.
But today the job was Local Variables.
It all started out well, then went to hell. To begin the day, I decided to create a "Trigger Hell" test script. The idea of this test script is to create a bunch of annoying trigger test cases, such as deleting the current trigger being run, adding a new trigger to the priority list to mess up the current position in the list being tested, and stuff like that. I know that CMUD currently has bugs regarding special trigger cases like this.
So, what I needed was a test alias that would display a text message to be processed by triggers and then check to see if the correct triggers fired the correct number of times. Here is the start of this script:
| Code: |
#ALIAS testtrig($mess,$expect) {
#VAR fired %null
; display message to cause triggers to fire
#SHOW $mess
; now check to see which triggers fired
$saved = $expect
#FORALL @fired {
#IF (!%ismember(%i,$expect)) {
#ECHO {%ansi(red)Trigger %i fired when it shouldnt}
}
$saved = %delitem( %i, $saved)
}
; now check for any triggers that didnt fire
#FORALL $saved {
#ECHO {%ansi(red)Trigger %i didnt fire}
}
}
#TRIGGER {test} {#ADDITEM fired 1}
testtrig test "1" |
Basically, we pass the message to be displayed, along with a string list of the triggers we expect to fire. Each trigger adds it's trigger number to the @fired list. So @fired contains the triggers that fired. Now we just loop through and determine if any fired that weren't supposed to, or if any that we expected to fire didn't.
The first step in playing with this was to make an improvement to the Text Editor in CMUD. Normally the Text Editor sends the text to CMUD line by line. This prevents you from putting multi-line scripts into the Text Editor for testing. So, I added an option called "Send Line by Line" that defaults to true, but if you turn it off, then the text (or selected text) is sent to the CMUD parser as one chunk, which allows the parser to handle the multiline scripts.
I plan to add a new command (maybe #SCRIPT or something like that) which will work like the #READ command but will send the entire file to the parser as a single block. Again, the #READ command only reads and processes scripts line by line. And to run my trigger test script, I want to store it in a *.TXT file and then type something like:
#SCRIPT trigger_hell.txt
and have it read the script and execute it. This is similar to how I already test the parser. I have a test script called "CMUDTEST.TXT" that I execute using the #READ command. This is an older test script and doesn't have any multiline scripts (although it would be a lot more readable if I made it multiline)
Anyway, I think people like edb will be really happy with this new feature. It makes it a lot easier to play with scripts in the Text Editor and run them.
OK, after getting this new feature added to the Editor, I ran into a couple more bugs. First, it was giving a syntax error on the } before the
$saved = %delitem( %i, $saved)
line. First I thought it was related to local variables, but I discovered that if I put *anything* on this line it caused an error.
Turns out there was a bug in the parser that would cause an error after any command like #IF, #WHILE, etc that has an expression as the first argument and a list of commands as the second argument. The parser was getting stuck in "expression mode" and was giving the syntax error trying to process a command statement.
I fixed that parser bug and now the script would at least compile and run. However, the "testtrig" alias that was created wasn't quite right. A minor annoyance is that it had an extra newline after each {. I fixed that pretty easily. The bigger problem was that the
$saved = %delitem( %i, $saved)
line was getting stored in the alias as:
$saved = %delitem( %i, )
In other words, it was trying to insert the value of $saved at alias creation time, thinking that $saved had a value outside the scope of the alias definition.
Parsing local variables gets tricky when you are doing stuff like creating an alias. And while I thought I had handled this already, In this case you'll notice that $saved is being accessing within the #FORALL block, which is nested within the main #ALIAS block. Turns out the parser was not handling local variables properly when you got deeper than one inner block level.
While I was fixing this stuff, I ran into a hell of a problem. It took me hours to figure out and caused me to start yelling and swearing at the computer again.
Somewhere in my changes, the parser had gotten totally screwed up, and even entering something simple like:
#ALIAS test {#SHOW hello}
was giving a syntax error. I couldn't track down the problem by single-stepping through the parser. It seemed that the parser was just giving up at the end and flagging it as a syntax error.
I started removing the code I had added bit by bit. Nothing would fix it.
Fine! I decided to restore the parser backup from yesterday. I ran Acronis True Image to restore the backup file. But apparently Acronis had stopped working last week sometime and when I ran Acronis to restore the file, it decided that it was past the scheduled time of making a backup, and started the backup task that is normally scheduled for 3AM.
The problem with this is that this caused my latest daily backup file to get deleted! If deleted the old weekly file and created a new one. My daily files are "differential" backups and depend upon the weekly backup. Since the weekly backup used to create the differentials was gone, it couldn't restore my file! Damn backup program! I don't know why it stopped working last week. Plenty of disk space on the backup volume. Looks like I need a better backup plan.
OK fine, I still have the CVS archive. Unfortunately I hadn't sent new versions to CVS in over a week. So the parser files were a bit older that I wanted. But restoring these files make the parser start working again.
So I started going through and adding my changed back in, using the CVS Diff utility to help me out. None of the changes that I thought would cause the problem seemed to be the cause.
All of a sudden the parser stopped working again! What the hell was going on!
Well, it turns out that YACC has a limit on the maximum length of a line. And one of the changes I was making that had nothing to do with any of the bugs I was fixing was causing one of the lines to get too large.
But instead of giving me an error message, YACC was just truncating the line and generating bogus parsing code. But through some fluke, the generated code would still compile in Delphi. Just a coincidence that it was truncating the line right after a ; character so the generated Delphi code was just missing some statements after the ; but the stuff generated was still valid Pascal code and compiled without error.
Took forever to figure this one out. Stupid YACC should have generated an error instead of just truncating the input lines. Sigh.
Well, once I figured out the problem, I was able to restore the files from CVS that I created this morning, and then just fix the long line in the yacc language file. Then things started working and I finally got my "testtrig" alias to work.
The next step is to add the #TRIGGER lines that will really test all of the wierd cases in the trigger processor. I'll work on this tomorrow. When I have it working I'll post the test script over in the CMUD forum and let other people suggest any improvements. Then I'll fix the trigger processor so that all of the triggers work properly.
Having this test script will make it a lot easier to avoid future trigger bugs. The current CMUDTEST.TXT script tests a lot of basic trigger stuff, but not wierd stuff like deleting the current trigger as it's running and stuff like that. Anything that currently adds or deletes a trigger while a trigger is running will cause problems, and that's what I need to fix in the trigger processor.
Anyway, just another day that didn't go nearly as smoothly as I was hoping. But at least all of the local variable stuff seems to be working now. As you can see in the above test script, the Named Arguments and Local Variables really make scripts a lot easier to understand and doesn't leave a lot of junk variables lying around. |
|
|
|
|
slicertool
Magician
Joined: 09 Oct 2003
Posts: 459
Location: USA
|
Posted: Tue Sep 26, 2006 5:35 am
|
Yay for local variables working right
I hope I'll have some spare time when 1.08 gets released. Don't feel bad when you resort to cursing and yelling at the computer, it knows you still care. |
|
|
|
|
edb6377
Magician
Joined: 29 Nov 2005
Posts: 482
|
Posted: Tue Sep 26, 2006 7:39 am
|
^^ yum.. i can only imagine the abilities of all of this when its working right. WOOHOO!
|
|
_________________
Confucious say "Bugs in Programs need Hammer" |
 |
|
|
saet
Apprentice
Joined: 12 Jun 2006
Posts: 106
|
Posted: Tue Sep 26, 2006 10:47 pm
|
Poor odd numbered releases, they never get their own blog entries :(
|
|
|
|
|
Zugg
MASTER
Joined: 25 Sep 2000
Posts: 23379
Location: Colorado, USA
|
Posted: Wed Sep 27, 2006 2:07 am
|
Wow, I keep finding some pretty amazing bugs in CMUD!
First, I found the bug that was resetting the trigger options. Turns out that it was effecting *all* settings options! Every setting read from the database was getting overwritten with the default options. This was really bad! Hard to believe that more problems didn't get reported because of this.
I fixed that and then started looking at some parser bugs. Someone had a script similar to this in their MUD file:
| Code: |
#WHILE ((@HP < @MxHP/2) AND (@M > 10)) {
hp=@hp+100
#SHOW @hp
} |
Well, the first problem was that this generated a syntax error. Removing the #SHOW command removed the syntax error...a wierd problem like some of the stuff Rorso has been reporting. But even without the #SHOW command, the #WHILE command wasn't working. Looking at the compiled code for this showed some wierd stuff. Looked like the compiled code was getting corrupted.
In CMUD, there is some optimization at compile time. For example, if you enter this:
| Code: |
| #IF (2+3 > 1) {...} |
then CMUD actually evaluates the expression (2+3>1) at *compile-time*. This saved time later at runtime. CMUD 1.07 performs optimizations like this with literal strings and integer math. However, there appeared to be a bug in the code for integers. This bug was corrupting the code from the previous script.
The way CMUD was performing optimizations was to look at the code stack when adding an operator (like + or / etc) to see if there were two literal numbers already on the stack. For example, if it saw the code:
then it could evaluate this at compile-time and replace it with just "INT 5".
So, when adding an operator, it checks the top two items on the stack. But how does it know that the last two items on the stacks are "INT" commands? In the compiled byte code, the INT operator has the value of 1. So, "INT 2" is implemented as two integers in the code stream: 1 2. 1 is the INT command, and 2 is the value. "INT 3" looks like 1 3.
In order to check the stack, it was looking at the next-to-last item to see if it was a 1. Since 1 is the INT command, this tells it that the last thing on the stack is an INT.
Do you already see the problem with this method? Yeah...what about other commands that put data on the stack. For example, a function call is represented by the FUNC command, then an integer giving the function ID number, then an integer giving the number of arguments passed to the function.
So imagine the case where the function ID is 1. Now we have a 1 followed by another integer (the number of arguments) which might look like an INT command when scanning backwards.
Or what if a STRING value is placed in the code stream. Normally a string doesn't contain a ASCII(1) character, but you can't count on this. And what if the value of the INT command changed from 1 to something else in the future, like 65, which is a common character value.
Basically, you can't just scan the code stream backwards to determine what is on the code stack. It just doesn't work reliably. So I added additional code to keep track of the stack position where new commands are added. This removes the uncertainty about where a command begins in the code stream. It's a bit more work to manage this new stack pointer list, but it's not too bad.
Once this was implemented, then the original script started working properly. And the syntax error with the #SHOW command also went away. So the parser is looking much better now.
While I was working on this, I also added similar optimization for INT64 and FLOAT datatypes (and also a mix of these). I also fixed a few issues with floating-point math comparisons.
So, lots of stuff was fixed today. Not stuff that I expected to be broken at this point. I just hope I didn't add too many new bugs with this kind of low-level parser work (gee, I probably shouldn't have said that  ) |
|
|
|
|
MattLofton
GURU
Joined: 23 Dec 2000
Posts: 4834
Location: USA
|
Posted: Wed Sep 27, 2006 3:08 am
|
Sounds like a pretty good day. A few more of those and Chiara will have to start distracting you with her own codes  .
|
|
_________________
EDIT: I didn't like my old signature |
|
|
|
Zugg
MASTER
Joined: 25 Sep 2000
Posts: 23379
Location: Colorado, USA
|
Posted: Thu Sep 28, 2006 12:49 am
|
I made some more good progress today. I changed how the trigger list is processed. In previous versions, the trigger processing routine was using a database index (list of triggers ordered by priority number) to loop through the triggers.
The problem with using this index directly is that if the script of the trigger creates a new trigger or deletes a trigger, this changes the index while CMUD is in the middle of scanning it. This can cause triggers to get skipped or to fire more than once since the position in the trigger index essentially becomes lost.
What CMUD 1.08 does now is to process the index before running through the trigger loop and create a linked-list of trigger "nodes" in the correct order. Adding new triggers doesn't effect this local linked list, and deleting a trigger causes it's node to be removed from the list without disturbing the position in the list.
Tomorrow I'll be finishing the "Trigger Hell" test script to see if this change fixes all of the wierd cases that CMUD wasn't handling.
While I was working on this stuff, I was also able to make a few performance improvements. I think I've gotten about another factor of 2 speed improvement in the trigger loop routine, and another speed improvement when setting the value of CMUD variables. Because of the database updates (indexes, etc), setting the value of a variable in CMUD is slower than in zMUD, and probably always will be. But I think I've got it running at a pretty decent speed now.
If all goes well tomorrow, then I'll try to release 1.08 for people to play with. We have to drive down to Santa Fe, NM on Friday to attend a friend's wedding and will be gone until Monday, so if I can release 1.08 tomorrow then everyone can mess with it while I'm gone again. |
|
|
|
|
Vijilante
SubAdmin
Joined: 18 Nov 2001
Posts: 5182
|
Posted: Thu Sep 28, 2006 1:40 am
|
I was thinking more about the problem you had with local variables and it occurred to me you might yet have a problem in the design. The problem is in the autotyping. Let's imagine the following:
#ALIAS bad
#VAR $local1 {0}
so far that would appear to be an INT type, I would guess you allowed for having the INT, FLOAT, etc. numeric types taking the same amount of space, but what happens when somewhere much further in the alias you get this?
#VAR $local1 {%concat($local1,"|abd|def|",$local2,"|",$local3,"|sdk|lgh|e34")}
Is the amount of stack space your assigning fixed by the type or do you handle them all as pointers into a heap that can overcome the problem of type assumptions? |
|
_________________
The only good questions are the ones we have never answered before.
Search the Forums |
|
|
|
slicertool
Magician
Joined: 09 Oct 2003
Posts: 459
Location: USA
|
Posted: Thu Sep 28, 2006 3:00 am
|
It sucks that I'm going to be travelling from thursday until monday, so I probably won't be able to play with 1.08 until after most of the bugs have been found :(
|
|
|
|
|
Zugg
MASTER
Joined: 25 Sep 2000
Posts: 23379
Location: Colorado, USA
|
Posted: Thu Sep 28, 2006 4:25 am
|
Vijilante: Variables are compiled into the code stream as pointers to existing database records. Only constant values are embedded into the byte stream. So there are not any problems with different type lengths or anything like that. Also, the runtime stack that hold intermediate values, such as local variables, also are stored as pointers. INTs and FLOATs are not stored in the same space and are not assumed to be the same size. In fact, that's why CMUD can be so flexible and internally handles Boolean, Bytes, 32-bit Integers, 64-bit Integers, double-precision floating point, strings, COM references, etc. CMUD can really handle any data type in the way it's designed.
|
|
|
|
|
Zugg
MASTER
Joined: 25 Sep 2000
Posts: 23379
Location: Colorado, USA
|
Posted: Fri Sep 29, 2006 4:01 am
|
OK, I have released v1.08. As I mentioned, I'll be gone until Monday at a friend's wedding. With all of the parser fixes in this version, I expect there to be some new bugs. Also, I started making some performance improvements and sometimes those have nasty side effects. I did some basic testing with some MUD files that people have sent me, but otherwise didn't have much time to do full testing today. But I'll continue to fix bugs when I get back next week. So let me know how 1.08 is working (or not) over in the CMUD Beta forum.
|
|
|
|
|
Tornhelm
Beginner
Joined: 24 Nov 2002
Posts: 20
Location: Australia
|
Posted: Tue Oct 03, 2006 2:44 am
|
Are you still looking for mud files? I have this one at the moment (besides the ones I have already sent you) that cmud really doesn't like trying to import.
|
|
|
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
