Register to post in forums, or Log in to your existing account
 

Post new topic  Reply to topic     Home » Forums » Zugg's Blog Goto page Previous  1, 2, 3  Next
Zugg Posted: Mon Mar 19, 2007 5:47 pm
CMUD v1.27 Blog
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23377
Location: Colorado, USA

PostPosted: Fri Mar 30, 2007 11:51 pm   
 
This is hard work! I think my brain is going to explode again.

OK, the success of this new Lexer method is all dependent upon the ability to recover from parser errors. I had added a property to ignore errors in the parser and kludged it so that when the parser failed, the syntax highlighter would try to run the parser starting on the next line break.

But why was the parser stopping in the first place? Yacc parsers are supposed to have error recovery to allow it to find multiple syntax errors. This isn't something that I have ever played with before. In the error routine used in CMUD, it was calling the Yacc yyabort method to end the parser whenever it got an error. So CMUD wasn't using any of Yacc's error recovery system.

I decided to do this right and not kludge it. Rather than just ignoring errors or restarting the parser at a new line, why didn't I use Yacc's error trapping and recovery? This would produce a better parser with better error messages. And it would open up the possibility in the future of reporting multiple syntax errors, just like other modern compilers.

Well, it's probably still a good idea, but boy has it been hard. The Yacc error recovery is pretty finicky. It's really easy to get it wrong. And to recover from an error also requires some work in the lexer to get it to the point in the script where it can continue parsing.

The reason I started to care more about proper error recovery is because of the "Reformat Script" function that I added. If there is a syntax error in a script, then the Reformat option doesn't work very well, and it can actually drop text around the error that didn't parse correctly. People probably saw this happen sometimes in zMUD too. The Pretty Printer really only worked properly if there were zero syntax errors. And I don't want it to just pretty-print the first part of a script and ignore the text after a syntax error. Once again, I want the parser to recover from and error and continue formatting the rest of the script as much as possible.

This gets very tricky. I need to be very careful with making parser changes or else I might introduce new parser errors. So it's going slow as I try to learn how to properly add the proper error rules to the Yacc grammer.

But, this is important work. Better error messages and the ability to get the cursor on the *correct* spot containing the error, and being able to reformat old zMUD scripts is all stuff that everyone needs. As usual, it just takes longer than I'd like. But it should be worth the effort. I'll continue this parser work on Monday.

Oh...Delphi 2007 finally arrived today (I didn't get the electronic download, so I had to wait for them to burn the DVDs and ship them). Not sure when I'm going to get the time to play with it yet. My brain is too tired today and this work on the CMUD parser is much more important.
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23377
Location: Colorado, USA

PostPosted: Sat Mar 31, 2007 5:06 am   
 
Cool, I figured out a better approach for the "Reformat Script" that worked *much* better when dealing with syntax errors.

The way it worked in zMUD is the Yacc parser would "reconstruct" the script as it went along, changing certain tokens to create a re-formatted script. As an example, one of the Yacc actions was something like this:
Code:
CommandList:
   ComLine                                         
        { $$ := $1; }
 | CommandList COM_SEP ComLine   
        { if PrettyPrint then $$ := $1+CRLF+$3 else $$ := $1+';'+$3}

If you are familiar with Yacc, you'll recognize this as a grammer rule, with an action that returns the parsing result. It's a recursive rule that appends each command separated by a COM_SEP (the ; character). If PrettyPrint is True, then instead of using the ';', it uses a CRLF character to put each command on a new line.

Now, the actual Yacc rule is much more complicated than this (it contains the compiler code generation, etc). But I've stripped out the essence of how the Reformatter worked. The Reformatter relied upon the parser to reconstruct the original script and to change whatever was needed to improve the script formatting.

The problem with this approach is the error recovery that I've been talking about. When you use the 'error' rule in Yacc to recover from an error, there is no way to reconstruct the part of the script that caused the error. So when the parser tries to put the script together, it is missing the piece from the error. This is a real pain to deal with. I've been working all night on trying to reconstruct the part of the script that caused the error so that it can be passed up the parser chain. But it just doesn't work.

So, I finally tried a different approach. Rather than re-constructing the script as it is parsed, I start with the original script value and simply insert and replace text in this copy to improve the formatting. Since it is now *adding* or changing data, rather than building it from scratch, errors are easily handled (it just ignores the error text and doesn't change it). So, the above rule now looks like this:

Code:
CommandList:
   ComLine 
 | CommandList COM_SEP ComLine   
        { if PrettyPrint then Replace(2, CRLF); }

where the Replace method takes the index of the grammer item (COM_SEP is the 2nd item in this case) and replaces the text of that item with the new text that is given. So, this replaces the ';' in the original script with a CRLF.

It takes a bit of work to keep track of the changed position of existing text as new text gets added to it. But this was just some fairly simple bookkeeping. After a couple of hours of playing with this new method, I have finally got it working. And it works *much* better when dealing with errors. This also makes the Reformat method faster since it's just changing a copy of the existing script rather than building a new script string from scratch. I just tried it on some of the scripts in the Finished Scripts forum and it works great! And no worry about it accidentally changing part of your script that might contain an error.

So this is starting to look really great now. I still have a bit of tweaking of the syntax highlighting to do. And then I need to handle the code folding. The Code folding is a bit weird...I thought I understood the various API calls to handle it, but it doesn't seem to work as documented. So on Monday I'll be digging into the Scintilla code to figure out how the API really works.

Anyway, I'm really happy that the Lexer/Parser are working so well now. It's actually been fun to work on this stuff again!
Reply with quote
Seb
Wizard


Joined: 14 Aug 2004
Posts: 1241

PostPosted: Sat Mar 31, 2007 2:18 pm   
 
This all sounds really positive. Well done. Smile
Reply with quote
bortaS
Magician


Joined: 10 Oct 2000
Posts: 320
Location: Springville, UT

PostPosted: Sat Mar 31, 2007 2:58 pm   
 
This has been quite a fascinating read. Even better is that you are having fun doing this stuff. I'm guessing many of us that program for a living, vicariously share your enjoyment. Mr. Green
_________________
bortaS
~~ Crusty Klingon Programmer ~~
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23377
Location: Colorado, USA

PostPosted: Tue Apr 03, 2007 3:16 am   
 
Today was a bit more annoying. It was one of those days when I didn't really get to work on what I wanted to.

I started fixing a few bugs in the syntax highlighting, and then ran into a bug in the Package Editor where it wouldn't create variables correctly. However, this only happened in weird cases. I had to use Ctrl-A to create a new alias (which also filters the package editor for aliases-only). Then I select New/Variable and add a new variable. Now it's filtered for both aliases and variables. Now I select the Alias and then use New/Variable to create a second variable. Now, click on the alias in the treeview, then click on a variable in the treeview, then click on the alias again. Then select New/Variable to try and create a 3rd variable. Now, instead of selecting the created variable, the window object is shown in the editor instead of the variable.

This was a really annoying bug to track down. Of course, it was all the fault of the TreeView component (again). It was trying to update itself because of the filter and causing the current database record to get changed in the middle of the New operation. I truly hate this tree view. It updates when I don't want it to, and doesn't update when I want it to. Someday I'll replace it, but that's a lot more work than I want to get sidetracked with right now. So I just added another patch to the code to take care of this situation. But that was a couple of hours down the drain.

Then, I was playing with the syntax highlighter for some more complex scripts and ran into a case that seemed to hang the editor. After debugging this for a while, it seemed that it wasn't really hanging the editor...it was just calling the Reformat routine to recompute the syntax highlighting many many times. The Scintilla editor component seems to reload it's properties whenever the window handle gets recreated. And when working with the editor within a frame, the window handle gets deleted when the frame gets hidden, and recreated when the frame gets displayed again. So this happens a lot. When the editor reloads it's properties, it calls the routine to update the syntax coloring whenever *any* style property is set.

For example, if you change the foreground color of "comments", the editor gets re-colored. If you change the background color of "comments", it gets re-colored again. If you change the font style of "comments", it gets re-colored again. So when the window handle is created and *all* properties are reloaded, the editor gets re-colored hundreds of times! Guess they really didn't test this component much with external lexers and their associated styles. Anyway, once I figured this out, I was able to add some update locking so that during the reloading of properties it doesn't re-color the text until it's done loading everything. Delphi components often have a "BeginUpdate" and "EndUpdate" methods to do this kind of stuff, so I just needed to add this to the property loading routines.

OK, once that was done I was able to mostly finish the various syntax "styles" for zScript. Each of these "styles" gives the colors, fonts, and other properties of a particular kind of syntax, such as "comments", or "commands", or "functions". One of my big jobs later this week is to write the user interface for allowing people to change these styles. At the same time I will be using "styles" elsewhere in CMUD to make it more consistent on how to set various colors and fonts.

But today I was just focused on the syntax highlighting itself. I think I've uncovered a few parser bugs in this. The syntax highlighter works so well that it has shown the wrong color in some cases. And in these cases, the syntax highlighter was working perfectly and was simply indicating that the parser itself was returning the wrong symbol values. So this is turning out to be a good tool to help debug the parser.

Right now, the syntax highlighter will color valid/invalid commands and functions in different colors. Also, when you start typing the first two letters of a command or function, it displays a popup list of commands/functions that start with those letters. So it has auto-completion for command and function names. In addition, it will perform auto-completion for any other word used in the same script. For example, if you have already used the word "zugg" somewhere in your script, then whenever you press "zu" a popup list will be displayed with all of the words within the current script that start with "zu". You can press Enter to select "zugg" from the list to auto-complete it. All of these auto-completion popups can be turned on/off in the editor preferences.

I also added different colors for defined/undefined variables and aliases. Also, when a variable/alias is defined, you can hover the mouse over the name and an underline will appear indicating that this is a hyperlink. When you double-click on the variable/alias name, it jumps to that variable/alias in the Package Editor. I will eventually implement a Back/Forward history in the package editor so that you can easily go back to the script that you double-clicked in.

I've also got the mouse-over value hint for variables working now. It simply displays the value of a variable in the mouse-over hint. It doesn't try to execute or evaluate anything. It just displays the simple value.

I also got the "Call Tips" feature implemented. When you enter the name of a function and press the ( key to start entering the arguments, you get a popup hint describing the arguments to the functions and a brief description of what the function does. I haven't figured out how to make this work for commands yet since commands use a space between arguments, so there isn't a unique key (like the "(" key) to indicate the start of the argument popup help. And you don't want the popup whenever you press the space key.

So, that was actually a lot of work today. Tomorrow I will work on the Code Folding to try and get it working. I'll also be working on some of those parser bugs. I hope to start working on the UI for the Style Preferences on Thursday.

At this rate, I don't think I'm going to implement the trigger tester yet in this version. I think I'll leave that till v1.28. I'd really like to get 1.27 released within a week or so if I can so that people can start playing with this new editor.
Reply with quote
Fang Xianfu
GURU


Joined: 26 Jan 2004
Posts: 5155
Location: United Kingdom

PostPosted: Tue Apr 03, 2007 12:07 pm   
 
Zugg wrote:
when you start typing the first two letters of a command or function, it displays a popup list of commands/functions that start with those letters... In addition, it will perform auto-completion for any other word used in the same script... When you enter the name of a function and press the ( key to start entering the arguments, you get a popup hint describing the arguments to the functions and a brief description of what the function does.

Surprised Shocked Very Happy Very Happy
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23377
Location: Colorado, USA

PostPosted: Tue Apr 03, 2007 10:50 pm   
 
Today I got the Code Folding to work. The API was a bit tricky and a bunch of the flags that were confusing me have apparently been deprecated. Once I took a close look at the source code for the C and Pascal lexers then it wasn't too hard to figure out.

The idea is that each line has a "Level". When the Level increases, a fold point is added. When the Level decreases, the fold point ends. There is also a flag to mark the beginning of a new fold section. So, in theory you just count { and } characters, increasing or decreasing the Level of the line.

The CMUD lexer/parser already returns a LBRACE or RBRACE symbol and handles stuff like comments, { within a string, etc. So I started with that. What caused me some trouble was how the lexer handles incrementing the line number. There was a special case when the } was following a newline. I'm not sure why the parser does the special case the way it does (the RBRACE gets generated *before* the line number is incremented for the newline even though the newline comes before the RBRACE). But I was afraid of introducing a bug in the parser so I left the parser alone and just fixed my folding code to detect this situation and handle it.

The other tricky part of all of this syntax highlighting that I tackled today was optimizing the performance. For external lexers, when you insert a character at position N in the text buffer, it calls the NeedStyling routine for the current line. But then it also calls the NeedStyling routine again and asks for the entire buffer from N to the end of the text to be restyled. This second call kills the performance when editing near the top of large scripts. And it really isn't necessary. Changes on a line rarely effect the syntax highlighting of the text on the lines after it (at least with zScript syntax). The only thing that can change after the current line is the code folding, which doesn't really need to be updated right away.

So, I added a background timer and queue and when NeedStyling is called to refresh the entire rest of the text buffer, I delay it until CMUD is idle. After CMUD has been idle for one second, then the background syntax highlighting is performed for the entire text buffer as requested. This really helps improve the text typing performance.

The other trick was modifying the initial call to NeedStyling. Normally it only refreshes the syntax for the current line. However, the CMUD parser/lexer has a lot of trouble starting at any line in your script. When you are within a {} block, the parser/lexer doesn't know what command you are within, and therefore can't determine the proper type for arguments. For example, if you are entering an #IF statement like this:
Code:
#IF (@a > 1) {
  #SHOW hello
  } {#SHOW else}

Imagine if the cursor is in the "else" clause and you start typing. If it just started at the beginning of the current line for syntax highlighting, it would just see the "} {#SHOW else" text and the parser wouldn't know that this was the else clause of an #IF statement. So the parser wouldn't know if #SHOW should be highlighted as a command, or as just normal text. For example, consider the command:
Code:
#SHOW {#SHOW}

In this case, the second "#SHOW" is a plain text string, and not a command. This is the most complex part of the CMUD parser...it needs to know the correct argument type for each command and function. The parser knows that the third argument of #IF is a list of commands (the else clause), and it knows that the arguments for #SHOW are strings.

In order to handle this all correctly, I adjust the starting position of the syntax highlighting to the beginning of the current block. This allows it to properly determine the command or function that we are typing in and to correctly identify the argument types that are expected. This slows down typing only slightly and would only have a big effect if typing at the end of a very large script block.

Most of the weird things that I thought were parser problems went away after these changes. Before, the parser just couldn't determine the correct syntax when only looking at a single line. Now it's all working much better.

So, I think the main details of the new editor are complete. Now I just need to write the user interface for setting all of the various styles and other options. This will probably take a couple of days.
Reply with quote
Seb
Wizard


Joined: 14 Aug 2004
Posts: 1241

PostPosted: Wed Apr 04, 2007 1:54 am   
 
All these editor enhancements sound very cool and will make writing scripts a lot easier and quicker. The Back and Forward buttons will be a nice touch in the future too. Keep up the good work! Very Happy
Reply with quote
Guinn
Wizard


Joined: 03 Mar 2001
Posts: 1127
Location: London

PostPosted: Wed Apr 04, 2007 5:30 pm   
 
Looks like a lot of progress made overall, congrats.
Got a quick question - with the improvements to the syntax, lexer etc and the way it now looks further back to see where commands start, would this fix up problems like
Code:
#SWITCH (@test == 1) {#say 1}
  (@test == 0) {#say 0}

working but
Code:
#SWITCH
  (@test == 1) {#say 1}
  (@test == 0) {#say 0}

failing (as it does currently) because CMUD expects the first argument to follow on the same line as the #SWITCH command?
_________________
CMUD Pro, Windows Vista x64
Core2 Q6600, 4GB RAM, GeForce 8800GT
Because you need it for text... ;)
Reply with quote
Fang Xianfu
GURU


Joined: 26 Jan 2004
Posts: 5155
Location: United Kingdom

PostPosted: Wed Apr 04, 2007 6:20 pm   
 
That'd be nice, but in situations like that I just use the alternate syntax. Works out nicely.
_________________
Rorso's syntax colouriser.

- Happy bunny is happy! (1/25)
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23377
Location: Colorado, USA

PostPosted: Wed Apr 04, 2007 9:20 pm   
 
Guinn: No, it won't have any effect on this. The issue that you raise is because of how Yacc does it's lookahead. Right after a command name, it isn't actually prepared to check for the argument list yet and can't handle a newline before the arguments (it conflicts with commands that don't have any arguments). Supporting the second syntax that you mentioned is much more difficult. It's on the list, but not something that will be fixed in the next version.
Reply with quote
nexela
Wizard


Joined: 15 Jan 2002
Posts: 1644
Location: USA

PostPosted: Wed Apr 04, 2007 11:32 pm   
 
To work around the first line limitiation just use 1 as the first argument since a true expression would return 1, or If I am looking for the first condition to fail I do #SWITCH 0
Code:
#SWITCH 1
  (@test == 1) {#say 1}
  (@test == 0) {#say 0}

OR
#SWITCH @test
 (1) {#SAY 1}
 (0) {#SAY 0}


If I am looking to match a string I do something like
Code:
#SWITCH $stringvar
 "IsItThis" {#SAY Yes}
 "OrIsItThis" {#SAY OR!}
 {#SAY ELSE!}


#SWITCH is the best cmud addition ever!
_________________
Zmud Support Library
Zmud Knowledge Base
Reply with quote
shalimar
GURU


Joined: 04 Aug 2002
Posts: 3515
Location: Pensacola, FL, USA

PostPosted: Thu Apr 05, 2007 12:48 am   
 
Well... if the indentation of the following arguments lined up with the first argument... it would look better
_________________
Windows 10 Home Premium 64-bit
AMD Phenom II x6 1055T 2.8GHz 16GB
CMUD Pro v3.34
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23377
Location: Colorado, USA

PostPosted: Fri Apr 06, 2007 4:44 am   
 
Wow, what a long day this has been. As usual, the last 10% of work seems to always take 90% of the time.

I had made really good progress on putting together the UI for editing the Style preferences. It replaced the "Fonts" tab in the Preferences, since the previous font entries are now just system Styles.

The basic editor panel allows you to edit the name of the style, the foreground color, background color, font name, font size, bold, italic, underline, etc. The only thing I still need to add is a small "sample text" area at the bottom to show the style in context. For example, zScript styles will show some zScript sample text so that you can immediately see how your change to a particular style will look.

All of this was going really nicely until I started to get annoyed at the ColorComboBox component from Developer Express. This is the same color combo box that is used in the Color Picker in the CMUD Tools menu.

This color picker has two big problems:
  • Can't enter hex color values
  • Can't copy/paste color values between controls

The first one is a killer in the Style editor, because the colors of styles are often some arbitrary #RRGGBB value and not some nice web-safe color name.

Now, this ColorComboBox inherits from the normal DevExpress ComboBox. The ComboBox has a property called DropDownListStyle which controls whether or not you can edit the value directly or only pick from a fixed drop-down list. This property is not exposed by the ColorComboBox...it is set to FixedList in the creator. I tried just changing this to the EditList value (which allows direct editing of the text field), but it had all sorts of problems.

I started digging into the DevExpress source code, and it seems that they almost went out of their way to make this all not work. They took a really lazy approach and always assumed that the text value matches something in the list, and when it isn't in the list of colors, it always sets the control to the default color (which displays "Color not selected" in the text field).

So, I started working on fixing this. Wow, what a pain it was. First, I fixed it so that when an RGB color value is entered, it gets added to the color list as a "custom color". This gets the color into the list, so now the control is happy. I had to fix this for both raw text entry, and for pasting a value into the control (which are handled in two different routines in their source code).

This mostly worked, but there was still an annoying problem. The incremental search feature in the combo box always returns the first matching string in the list. But when the color list is sorted by color, a color such as "BlueViolet" appears *before* the color "Blue" in the dropdown list. So, whenever you'd enter "blue", it would select the "BlueViolet" color. And selecting the "Violet" part and pressing the DEL key would just delete the entire entry and change it to "Color not selected"!

So first I had to change the behavior of the DEL key so that it would work properly and not just clear the entire color value. How when "Violet" was selected and DEL was pressed, the "Violet" would be momentarily deleted, and then it would do a lookup of "Blue" and return the first matching "BlueViolet" item again. Grrr.

OK, so I had to go into the routine that matches items in the incremental search (just for the color combo box) so that it would look for the shortest matching item in the list. Now when you enter "Blue", it correctly finds the "Blue" color. If you enter "Bluev" then it will find the "BlueViolet" color as the closest matching color.

Fixing the DevExpress source code to handle all of this took about nine hours! That's almost as much time as I've spent on the entire rest of the Style editor! But at least now it works. This will also give the same improvements to the Color Picker tool, so it was worth the effort I think.

What's left of the new editor and Style system is to add the "sample text" to the styles, and then add some other system-defined styles. I'd like to go ahead and finish the whole style system and add the new #STYLE command, MXP STYLE tags, and some other features to make styles really useful. The idea is that you'll be able to use a Style name in place of any color name in CMUD. So, for example, if you had a style called "MyStyle", you could do this:
Code:
#TRIGGER {Zugg} {#CW MyStyle}

and it would apply the entire style (foreground color, background color, font name, size, bold, italic, underline, etc) to the matching words. This could really end up being a very powerful feature in CMUD and is something I've been wanting to implement for a long time.

But I'll have to see how much more effort it is to fully implement the style system, or whether I'll release the next version with just the styles for the syntax highlighter.
Reply with quote
Larkin
Wizard


Joined: 25 Mar 2003
Posts: 1113
Location: USA

PostPosted: Fri Apr 06, 2007 12:13 pm   
 
Sweet! I got my new Vista PC last week, and I've been holding my breath for this 1.27 version as the first one I put on my new machine. I hope it's ready soon because I'm feeling a little lightheaded. Very Happy
Reply with quote
Tech
GURU


Joined: 18 Oct 2000
Posts: 2733
Location: Atlanta, USA

PostPosted: Fri Apr 06, 2007 2:51 pm   
 
Can we get 1.27 today... It could be my early birthday gift! Smile

For those of you who spend too much time in the forums you'll recall I made the same request for the BETA release of CMUD. What a great mudding year it's been. Thanks for all the hard work Zugg, and Happy Easter to all who celebrate it.
_________________
Asati di tempari!
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23377
Location: Colorado, USA

PostPosted: Fri Apr 06, 2007 3:37 pm   
 
Tech: I'd love to give you a birthday present, but I'm afraid it will be next week. I don't really want to ship something that has the new stuff just partially implemented. Hopefully next week though.

Heh...at least you won't have to wait as long as you did last year. Since the first beta version didn't actually get released until last June, you were waiting for a couple of months back then. But I remember the blog and our discussions about it. It was a pretty stressful time if I remember correctly (I've blocked most of it from my mind :)
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23377
Location: Colorado, USA

PostPosted: Sat Apr 07, 2007 2:16 am   
 
Today was another busy, but good and productive day. Worked on more stuff that had slipped my mind (that seems to happen more and more as I get older ;)

One of the problems with the Delphi wrapper around the Scintilla editor is that they are storing *all* of the language information (syntax styles, keywords, etc) for *all* available languages with the properties of *each* editor component (stored in the Delphi form file). So, each editor component is completely separate from each other, and the language information gets duplicated all over the place.

There is a property of the editor called the "LanguageManager" that contains the list of all available languages and their corresponding styles and keyword lists. It looks like the author was in the middle of trying to make the Language Manager a separate component so that multiple editors could share the same language information. However, in the source code, there is the following comment:
Code:
   31/01/2006 Made some preparations for TSciLanguageManager standalone component
              NOTE!! NOT COMPLETE.
   14/09/2006 Really should get on with the item above, but I am somewhat stuck on how to do
              it without too much restructuring of the code.
              To me it seems like this would require quite a bit of rewriting with the following
              breakage of code.

That didn't sound very good to me. But the more I looked at the code, the more I understood a way to make it work. Of course, I didn't have to worry about compatibility and any "breakage of code". I didn't care if the new version worked with old code. I just needed to make it work for new code. It involved some tricky work, but I eventually got it pulled apart. So now CMUD has a single LanguageManager that all of the Scintilla editors point to for their language information.

I'm going to send my changes into the author listed on the SourceForge project to see if he finds it useful. But I have to admit that it probably isn't "production quality". Still, it works fine for my application.

Once I got a single LanguageManager in place, I was then able to add all of the styles for all of the other scripting languages (VBScript, JScript, Perl, Python, Ruby, Lua, etc) to my master style list so that they all show up in the new Styles Preferences editor page.

My CMUD "Style" component is a wrapper around several other types of "styles". It can wrap the Scintilla Style component, and I added the ability to wrap the RichView Style that is used for the RichEdit components (the Text Editor screen, the Command Line, etc). So now the master Style list can contain any mix of different style types. I added the styles for the Command Line, Status Window, and Text Editor windows today.

The one remaining "style" that I need to wrap is the ANSI color styles used in the MUD Output window. That will be a bit tricky and might require some changes to my core ANSI Terminal Window component to allow more general text colors. I'm still trying to decide if I will go ahead with this for this version, or leave it for a future version. Since the current Style system is complete and working, I'm probably going to wait on implementing the styles in the main MUD window so that I can get this version done next week.

The last thing that I worked on and completed was all of the saving and loading of the styles to the INI file (for global styles) and the PKG files (for local styles). I'm using a text format that is also used in the Scintilla LanguageManager to save a style to a single formatted text string. It's nicely human readable, so you can easily edit the style information in the CMUD.INI file if you want. It should also allow me to load style file "overrides" for the various scripting languages, allowing you to specify your own language file (custom keywords, etc) if you need that.

I really like how this is coming together! It's been a pretty good week. Hopefully I can finish up the loose ends next week and get this version released at the end of the week.
Reply with quote
edb6377
Magician


Joined: 29 Nov 2005
Posts: 482

PostPosted: Sat Apr 07, 2007 10:04 pm   
 
sounds great as always zugg.. hard at work and lots of things to test :P
_________________
Confucious say "Bugs in Programs need Hammer"
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23377
Location: Colorado, USA

PostPosted: Tue Apr 10, 2007 2:26 am   
 
I just love it when stuff works. Today was fun. Today I integrated the zApp scripting host into CMUD.

If you are one of the few people who ever followed my zApp project development, you might recall that I developed my own Scripting Host (following the published interfaces from Microsoft). This Scripting Host is similar to the normal Microsoft MSSCRIPT.OCX scripting object that is used by most applications (including zMUD, and all of the other MUD clients that use the Microsoft Scripting engine to execute scripts in VBScript, JScript, etc).

The problem with the Microsoft scripting host, and the main reason I wrote my own, is that the Microsoft scripting host can only host a single scripting language at any given time. You might have noticed this in zMUD. If you tried the following code in zMUD:
Code:
#MSS "a=123" "vbscript"
#SHOW %mss("a","vbscript")
-> 123
#MSS "a=456;" "jscript"
#SHOW %mss("a","jscript")
-> 456
#SHOW %mss("a","vbscript")
-> nothing

then you would notice that as soon as you changed languages from VBScript to JScript, all VBScript variables and functions were removed. So there was no way to use multiple scripting languages. You could use a *single* different scripting language, but not a mix.

The scripting host that I wrote for zApp fixed all of this. It allows multiple scripting languages to be used at the same time, and even allows some basic communication between the scripting language (for those languages that support the full scripting host model properly).

This scripting host is now used in CMUD, so when you execute the above code, the last line properly shows "123" since the VBScript variables are still defined.

The other benefit of the zApp scripting host is that I was able to implement "dynamic COM objects" in zApp. This allowed scripts in zApp to reference the various components that were created at runtime (buttons, edit boxes, etc). Using this dynamic technology, I have added a nifty new feature to CMUD that allows you to *easily* access CMUD variables from within other scripting languages. Just use the "var.varname" syntax, where "varname" is the name of the CMUD variable you want to access.

For example:
Code:
A = 123
#MSS "msgbox var.a"

will display a popup msgbox from VBScript that displays "123". In the past, you needed to use the complex COM API to retrieve a variable object and then get it's value. With this new feature, you can use CMUD variables from other scripting languages as easily as you can access their own local variables.

I'll probably eventually implement easy access to other objects, such as aliases or triggers. But for now there is just the variables using the "var.varname" syntax. (Note, in Perl this would be $var.a instead of var.a since Perl requires a $ before variable names).

Today I also finished off the syntax editor and defined all of the built-in languages that it recognizes (VBScript, JavaScript, Perl, Python, Lua, Ruby). I also added a mechanism where you can define a "language.sty" file which contains the definition of a language (based upon any lexer that is contained within the Scintilla editor DLL), along with the keyword list and style list for the language. So, this allows you to install *any* scripting language and then add a file to allow CMUD to perform syntax highlighting for that language. This should make the CMUD script editor really versatile.

I integrated all of this into the Package Editor so that when you select a different scripting language from the drop-down list in the script editor, it automatically changes the syntax highlighting to reflect the selected language. Any scripting language that you have installed into Windows that supports the Microsoft scripting engine should be automatically recognized and available in the drop-down language list.

The beauty of this new system is that you will be able to implement your scripts in any scripting language that you want, and then upload these scripts to the Shared Script library. And people will be able to download your scripts and use them, without having to worry about whether they are mixing different scripting languages. Of course, the end user will need to install the scripting language itself if it's not one of the languages that comes with Windows (VBScript, JScript). However, I am considering adding a built-in LUA scripting engine, and possibly RubyScript in the future (currently you have to install RubyScript separately)

I'll try to document this stuff in the help file right before I release the new version. It's really a huge improvement to the alternate scripting language support that zMUD had, and should make scripting in CMUD better than any other MUD client.

For the rest of this week, I'm going to work on some of the high priority bugs on my bug list. That means I'm mostly done with any big new features for the next version now. I'll let you know how the bug fixing goes during the rest of this week. Looking good for a release later this week.
Reply with quote
Fang Xianfu
GURU


Joined: 26 Jan 2004
Posts: 5155
Location: United Kingdom

PostPosted: Tue Apr 10, 2007 4:09 am   
 
It's nice that you've been able to add something this powerful with ease. It's human nature to remember times when things go wrong over when they go write - commit this to memory :)

While I'm thinking about it, did the bug where CMUD didn't transmit/the library didn't receive your catagory/mud etc settings ever get fixed? It'd be nice if there were some sort of Languages Used descriptor too.
Reply with quote
NowhereMan
Beginner


Joined: 09 Dec 2004
Posts: 10

PostPosted: Tue Apr 10, 2007 6:01 pm   
 
Zugg wrote:

The other benefit of the zApp scripting host is that I was able to implement "dynamic COM objects" in zApp. This allowed scripts in zApp to reference the various components that were created at runtime (buttons, edit boxes, etc). Using this dynamic technology, I have added a nifty new feature to CMUD that allows you to *easily* access CMUD variables from within other scripting languages. Just use the "var.varname" syntax, where "varname" is the name of the CMUD variable you want to access.


Maybe it doesn't matter, but "var" is a keyword in javascript/JScript. You might want to consider a different naming. (if you are speaking literally about a 'var' object, that is. )
Reply with quote
Zugg
MASTER


Joined: 25 Sep 2000
Posts: 23377
Location: Colorado, USA

PostPosted: Wed Apr 11, 2007 1:13 am   
 
Hmm, yes, you are correct. Using "var.varname" in JScript gives an error. Any suggestions on a better name that isn't going to be a reserved work and yet is short and easy to remember? Keep in mind that "cmud" is already used to refer to the main CMUD COM application object. I was also going to use "func.functionname" and "cmd.commandname" to reference CMUD functions and commands, but "func" and "cmd" might also be reserved words. (actually, it looks like "func" and "cmd" are ok. So it's just "var" that needs to be changed). Suggestions are welcome.

Today I just worked on fixing various bugs. I updated some of the Beta forum posts for some of them. Fixed a bunch of misc stuff, including some parser bugs, and some new bugs that were introduced by the new syntax editor component. Also fixed a problem with the new Reformat routine that could cause a problem if you tried to reformat a script that was already formatted. Also fixed various bugs dealing with ; and // comments (especially following a { or at the beginning of a script block).

I'll continue working on more bug fixes tomorrow and for most of the rest of this week.
Reply with quote
MattLofton
GURU


Joined: 23 Dec 2000
Posts: 4834
Location: USA

PostPosted: Wed Apr 11, 2007 1:39 am   
 
you call them entities in MXP. How about "ent"? What about "dat" or "data"? What about doing something "the cheap way"? Instead of "var", just go with "va" or "vr" or go a little longer like "vari".
_________________
EDIT: I didn't like my old signature
Reply with quote
nexela
Wizard


Joined: 15 Jan 2002
Posts: 1644
Location: USA

PostPosted: Wed Apr 11, 2007 2:50 am   
 
How about cmudvar.varname this would allow for cmudcmd for commands and cmudfunc for functions and 10 months down the road you can look at it and remember that it is cmud related
_________________
Zmud Support Library
Zmud Knowledge Base
Reply with quote
Display posts from previous:   
Post new topic   Reply to topic     Home » Forums » Zugg's Blog All times are GMT
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
© 2009 Zugg Software. Hosted on Wolfpaw.net