Zugg Software :: View Entry

SubAdmin Joined: 18 Nov 2001 Posts: 5182

I definetely agree with Rainchild about memory use. My programming projects consitently have comments in them when allocating memory; that memory is cheap, allocate high...reallocation takes longer.

The only problem I see in the the system your using, for creating unique id's, is the 11 bit limit it imposes. I am not sure quite sure if that leaves me 2047 unique settings, or 2047 unique settings of each setting type. In either case lets suppose MUDs become so sophiscated, over the next ten years, that using 2 million triggers is not uncommon; how will anyone manage under the constraint of having to wade through over 1000 packages to find an errant trigger? I would suggest planning for CMud to be around for another 10 years and making that id field bigger to start with.

Posted: Mon Dec 12, 2005 5:27 pm

Vijilante, I think you mis-read the post. I'm using 12 bits for the package number, and 20 bits for the settings ID. The 20 bits for the settings ID gives you about 1 million triggers per package. And the 12 bits gives you 2048 packages. I can't imagine using more than 1 million settings per package. I know this sounds like "famous last words", but really, when would you *ever* need so much stuff for a single MUD game in a single package?

Sure, I hate to design memory constraints like this, but making these fields larger just slows down CMUD *now*. 32-bit indexing is very fast. Multiple-field indexing is slower. Why slow down the entire thing just because one person might somehow need more than 1 million aliases/triggers in the future? Really just doesn't make sense.

As far as the updates, keep in mind that the update doesn't do anything unless something has changed that needs to be sent to the database. If you ignore Variables for a minute, few of your other settings are changing. Most triggers don't change their definitions and most aliases don't change definitions. From what I have seen on my test system here, the update process is *very fast*, at least for SQLite databases.

This gets back to your other question about caching the database (map in memory, etc). SQLite *does* load the entire database into memory. That is how it gets it's speed. Sure, updates are still sent to the disk file, and honestly I have no idea how SQLite does this, whether it uses background threads or not. That kind of low-level detail is beyond the scope of what I'm doing. I'm using the higher-level ZeosLib database library and letting it handle these details.

But it is *not* rewriting the entire file, or anything like that as far as I can tell. If I load a couple hundred settings into memory and mark and then save everything to a new database file (so it has to update every record), it's nearly instant when I click the Save button. I'll do more precise timing benchmarks in the future, but I don't expect this to cause any lag.

And, of course, you can set it just like in the current zMUD to only "save on exit". Then you won't get any lag at all and the settings will only be updated to disk when you exit, manually save, or open the Package/Settings Editor.

Now, back to Variables. Variables are the things that change the most. One of the optimizations that I'm planning to add are "local variables". These are variables that get loaded from your settings file (or created on the fly), but are *not* updated back to the disk file when they change. Any changes to a "local variable" are just kept in memory. A large percentage of variables are only needed within a specific session. The number of variables that need to "persist across sessions" is much smaller. So by marking the variables that change all the time as "local", you can save a lot of updates.

As far as the "rotating package saving"...that doesn't work in the current design. Remember there is only *one* database in memory. When I tell that database to save it's changes to disk, it loops through the database and sends the appropriate SQL UPDATE to the proper SQL database for each record. There is no way to tell the database to "only save those records for package xx". Sure, I can put something in my "SaveRecord" code that just skips any records that don't belong to the current package being saved, but since the SQL database is already open, and already in memory at that point, I might as well go ahead and save the record. But this kind of detailed optimizing will probably wait until the beta period to tweak.

All I can say is that SQLite is *extremely* fast. It *might* be possible to use different database formats for different packages, and I'm designing it with that ability in mind. So if you attach a package to a remote SQL database, then it might be slower. But in that case, I'd just turn off the autosave option and only save the settings manually as needed.