Zugg
MASTER
Joined: 25 Sep 2000
Posts: 23379
Location: Colorado, USA
|
 Posted: Fri Jun 04, 2010 9:09 pm
Benchmark results for v3.18d - AMAZING!
|
I'll be posting the 3.18d version later today. But I just completed the benchmarking results for the new SuperObject tables and the results are really amazing and just what I was hoping for when I started all of this work.
For those who want to play with this themselves, here are my simple test scripts:
| Code: |
#ALIAS makelist {list="";StartTime=%secs;#LOOP %1 {#ADDITEM list %i};#SHOW "Time:" %eval(%secs-@StartTime)}
#ALIAS makedb {db="";StartTime=%secs;#LOOP %1 {#ADDKEY db %i %i};#SHOW "Time:" %eval(%secs-@StartTime)}
#ALIAS querylist {StartTime=%secs;#LOOP %1 {#CALL %ismember(%i,@list)};#SHOW "Time:" %eval(%secs-@StartTime)}
#ALIAS querydb {StartTime=%secs;#LOOP %1 {#CALL %db(@db,%i)};#SHOW "Time:" %eval(%secs-@StartTime)} |
I'm not testing any nested tables here because I want scripts that I can also run in zMUD.
For each test routine, you pass a number to it indicating how many times to loop. Notice that I'm not generating any random keys or values, but am just using the loop counter as the key and data. Again, wanted to keep this simple for now.
The columns in the following results indicate the different values of "n" (the loop counter argument) used. When doing the "query" operations, I have two rows: the query(100) means that the original list was built with "makelist 100" and the query(10000) means the list was built with "makelist 10000". This allows me to see if the query results depend upon the size of the list being queried.
Here are the raw results (all values in milliseconds):
| Code: |
CMUD 3 10 100 1000 10000
makelist 0 4 38 377
makedb 0 4 44 440
querylist(100) 0 2 13 133
querylist(10000) 0 2 14 137
querydb(100) 0 1 13 136
querydb(10000) 0 2 14 136
CMUD 2 10 100 1000 10000
makelist 0 5 374 36685
makedb 0 2 20 200
querylist(100) 0 4 49 508
querylist(10000) 0 3 194 18426
querydb(100) 1 2 69 775
querydb(10000) 1 1 15 164
zMUD 10 100 1000 10000
makelist 0 9 368 35643
makedb 0 14 1100 114966
querylist(100) 1 9 106 1078
querylist(10000) 6 64 863 33833
querydb(100) 1 10 124 1274
querydb(10000) 13 124 1552 48770 |
Here are the highlights to note:
- In CMUD 3 it doesn't matter whether you are using a string list or a database table. It doesn't matter what the size of the table is when you query it. All of the times are *very* fast and scale properly with N. Results for sorted lists and tables are not shown but are the same. CMUD 3 doesn't care if anything is sorted.
- CMUD 3 doesn't care if anything is nested. Doing these tests on nested lists and tables give the same results.
- In CMUD 2 (and v3.17 and earlier) the string lists are much slower than the database tables. This is because the tables use hash tables. The string lists become faster when using #SORT. Creating the hash tables is very fast, even slightly faster than CMUD3. However, the querydb is still slower than CMUD 3 and depends upon the size of the list. While sorting the string lists make them about as fast as the hash tables, most people are probably not using sorted lists in their scripts.
- In CMUD 2, any "nested" strings or tables are treated as plain strings, which end up giving about the same times as the unsorted string list. Again, very slow.
- The zMUD results are pretty laughable. Everything is slow. Nothing scales properly.
So yes, I am *VERY* happy with these results. If you have scripts that use string lists or database variables you'll definitely want to be using CMUD v3.18 and later. |
|
