Zugg Software :: View topic

Posted: Thu Jun 07, 2007 10:18 pm

Well, now you know what I've been working on for the past several weeks! Sorry I had to keep this a secret, but I wasn't even sure that this would be possible and I didn't want to get anyone's hope up.

For those that have used zExplorer or AC Explorer in the past, you know that I had figured out how to extract the game terrain from Turbine's data files. When LOTRO was released last month, I wasn't expecting much. Turbine's track record after their wonderful Asheron's Call game wasn't very good. I was really pleasantly surprised to discover what a wonderful game Lord of the Rings Online is! It was a refreshing break from WoW, and is getting excellent reviews.

I wasn't able to add zExplorer support for WoW, because WoW is actually a zone-based game. The coordinates for locations are relative to the zone you are in, rather than being a single world-wide coordinate system. Also, there wasn't any way within the game to determine your location, which was important for adding new locations to the zExplorer database.

Overtime, many WoW MODS have added location information, and there are several other good map programs for WoW. So, the time passed zExplorer by and it never became worthwhile to add WoW support. Also, the game world of WoW was just too large to handle in zExplorer.

When LOTRO was released, I dusted off my tools for extracting map data and started exploring the LOTRO data files. Turbine had changed many things in the data files, and for a while I wasn't sure it was going to be possible to extract the terrain data.

But I made several breakthroughs in decoding the map files that led me to finally create a huge map of the entire LOTRO world. Even areas outside of those within the game were viewable as raw terrain data. I found more detailed terrain data for the accessible areas of the game that I was able to overlay on top of the raw terrain.

The problem was the size of the image. zExplorer previously could handle an image file 2048x2048 with 8-bit color. The LOTRO map was about 16000x10000 with 24-bit color. That was larger than Windows could display as a single bitmap image. Even when I reduced the color to 16-bit, the resulting BMP file was over 400MB in size. When loaded into zExplorer, zExplorer would require 400MB of memory, which didn't leave enough left on most computers to run the actual game!

So the next challenge was to figure out a more memory efficient way to handle panning and zooming extremely large bitmaps. After a lot of work, I finally came up with some algorithms to handle this. The result is seen in the new zExplorer, which can quickly pan and zoom across a huge bitmap image. This image was compressed and optimized into my own DAT file format so that zExplorer only requires 20MB of memory, instead of 400MB, to handle the full LOTRO map. Yeah, I was even impressed myself!

The other thing that made all of this possible was the inclusion of the /loc command in the game (thanks Turbine!). Unlike the human-readable ;loc command that displays stuff like 50.1S, 60.2W, the /loc command displays a bunch of data that is harder to figure out. After some experimenting, I was able to develop an algorithm to convert the full /loc output to a location with *two* decimals of accuracy instead of one. So the output of /loc can be converted to something like 50.12S. 60.23W.

With this more accurate algorithm, I was able to create a much more accurate database of locations within the LOTRO game. I started with a combination of various databases and then explored most of the game myself to adjust locations of NPCs and objects to this higher accuracy (and remove any bad data, fix spelling, etc). I also added links to the lotroquests.com site for the quests of each NPC.

The result is pretty impressive. I mainly made this for myself, but I hope other LOTRO players like it too. And since I can generate the map from the actual game data, I should be able to easily update the map whenever Turbine releases a new update (like they are planning next week already!)

If you play LOTRO, I hope you give zExplorer a test drive. It has a free 30-day trial. I have plans to add real-time Tracking in the future, and hope to learn how to extract some building data from the game files too. If you are a previous registered user of AC Explorer Pro or zExplorer, this is a free upgrade.

For those CMUD users who are reading this, sorry to disappoint you by not announcing a new CMUD version today. I really had a limited-time opportunity to update zExplorer to work with LOTRO, and I decided it was important to take the few weeks that this required. Now that zExplorer 5 has been released, I should be able to get back to CMUD. I hope to release a new version of CMUD by the end of June with the SSH code in it (along with some other new features).

Posted: Thu Jun 07, 2007 10:56 pm

Congrats Zugg!!! The map compression and display algorithms do sound impressive. If I played LOTRO I would check it out.

I am one of the folks that wish this was a CMUD release but I know it helps to mix things up a bit.

Plus the silver lining a see is that you have now refreshed you skills and should be ready let zMapper/cMapper code fly real soon.

I may check out zExplorer just to look at the maps. Congrats again. And to all you LOTRO folks out there... lucky buggers!!! Smile

Posted: Thu Jun 07, 2007 10:59 pm

You know, I'd been living in a bubble when it comes to LOTRO - I remember one of my friends being invited to the beta a while ago, and after that I'd heard nothing. I didn't even know it had released. If Zugg's playing it, though, they might get me to give it a peek ;)

Posted: Fri Jun 08, 2007 1:36 am

Yeah, it hasn't been marketed very strongly. It's getting great reviews and sales, but they seem to be depending upon word-of-mouth. I think that's a good thing. If it had been hyped (like most bad games these days), then I probably would have skipped it. I also think it was smart to let them start smaller and build over time since they can keep up with the server demand a bit better that way.

In fact, it was probably the smoothest MMO launch that I have ever seen (and I've pretty much tried them all). It feels very polished for a brand new game. And while WoW certainly has more content, I'm still impressed with the amount of content LOTRO has at it's release.

Of course, everyone should still be using CMUD to play MUDs and shouldn't be spending time with MMOs Razz

But at least with zExplorer I can diversify business a bit more. Now that zExplorer can handle huge maps, I might even go ahead with WoW map support for it sometime in the future if I get some time.

And yes, Tech, I think cMapper will definitely benefit from the work I have just done on zExplorer. As you probably know, the core map routines between zMapper/zExplorer/zMUD/CMUD are all the same. And the work on zExplorer has gotten me more familiar with a lot of the code again. While zExplorer isn't "themed" like CMUD is, I *did* convert it to use all of the DevExpress controls. So it gave me a good look on how much work might be involved with cMapper. And if I had a couple more weeks of time, I *would* have gone ahead with the conversion from ADO/MDAC to SQLite. Working on zExplorer really reminded me how much I hate the ADO stuff. I still have lots of stuff I want to do in CMUD first, but the cMapper rewrite is definitely still in the works for this fall.

Posted: Fri Jun 08, 2007 2:43 am

Posted: Fri Jun 08, 2007 2:49 am

Yeah, that's exactly why I hadn't bothered with adding WoW support. I already use MetaMap, but I'd rather have an actual terrain-based map instead of their "artist map". But there is already a program called WoW Cartographer that does most everything zExplorer would do, and it's already free (or donation-ware) too. And there is no way I could compete with the Thottbot database.

That's why I felt I had a limited window of opportunity to support LOTRO early. With Asheron's Call, AC Explorer became the best map and database resource very quickly, even though it was released a year after the game went live. But these days, there are more fan sites that are much more organized and quick to respond. There are already attempts to do some mapping sites, but they either use the ingame "artist" maps and have horrible resolution, or the Google-Map project uses ingame screen captures of radar images, which doesn't cover most of the world, and certainly can't keep up with new expansion regions.

So with LOTRO I felt there was really a good need for something better. I wanted something better myself. I doubt a lot of people will pay for it, but then again, plenty of people bought AC Explorer Pro many years ago. So there is always a small percentage of people willing to pay for high-quality software like this. Hopefully the combination of increased accuracy for the location database, and the ability to easily update the map as new expansions are added will help zExplorer keep it's value for LOTRO.

Posted: Fri Jun 08, 2007 8:40 am

Out of curiosity, how are you marketing it to LOTRO users? Now that you've built it, how will you get them to come?

Newbie Joined: 08 Jun 2007 Posts: 2

I'm sorry if this is already answered in the documentation or of this is is the wrong forum, but I'm too excited just hearing about this. Can you please tell me if this will allow players to log into a "guild" server and track their pips on the map like you had for AC Explorer? That was the coolest feature to me.

edit: Okay, I played around with it some now. I don't see tracking yet (and I noticed its on the wish list). I can guarantee 1 sale once tracking is working. Very Happy

Posted: Fri Jun 08, 2007 5:22 pm

Tracking is definitely something I want to investigate. It's something I've really wanted myself. Not really sure how Turbine will feel about that though...it requires hacking into the client memory image, which might be against the EULA.

In terms of marketing, I've already posted the news to several LOTRO fan sites. So far the reception has been *very* positive. I also posted to the official LOTRO forum, but my post was deleted because their forum policy doesn't allow discussion of 3rd party software (a policy I wasn't aware of). So, I'll have to stick to the fan sites for now.

If anyone has any other ideas on marketing this, let me know.

Wizard Joined: 14 Oct 2000 Posts: 1368

Newbie Joined: 08 Jun 2007 Posts: 2

Not sure if you got my PM on the LOTRO forum, but I had offered to promote zExplorer on my guild site. Its a stepping stone between the official forum and your site so they really can say anything about it. Very Happy

Just let me know if you're cool with it and I'll set up a link and ad for you.

edit: ok, scratch that, just got the PM, hehe

All set at www.guilduniverse.com/P_B Very Happy

Posted: Fri Jun 08, 2007 9:53 pm

Rorso: Using a third-party program to hack into an MMORPG's memory is normally very, VERY bad. Instant permanent ban on all your accounts bad. The reason they have that policy is because once you've read the memory, you have enough information to build a bot that can click on the screen to play the game for you. If you were to write to the game's memory yourself, you could trick it into doing many things - teleporting you across the game, all sorts. So no, that won't be supported because it'd make zExplorer against the rules.

Which is probably why they have a policy against third-party software on their forums. Perhaps if you emailed them and explained what exactly zExplorer is and how it works, they'll let you post about it. They may be working under the impression that it's a bot.

Wizard Joined: 14 Oct 2000 Posts: 1368

Posted: Fri Jun 08, 2007 10:41 pm

You'd be surprised what MMORPGs let the client do actually. It's entirely possible to teleport around Ragnarok Online by hacking the memory, I know this for a fact. As bad an idea as it is to trust the client, I imagine that MMORPGs have to - when you have a couple of thousand users logged onto the server, each user has a limited amount of updates to the server. Perhaps more modern MMOs have found ways around this, but most just scan your memory for known bots that're running and report you if they are.

And the difference between zMUD playing MUDs and an zExplorer playing MMORPGs is that MMORPGs insist you use their client and no third-party, memory-scanning software, for fairness. MUDs normally let the user choose their own client. That and, as we've already established, using third-party software that reads the game's memory is against the EULAs of every single MMO I can think of, whereas MUDs normally have no rules about what client you can use.

Imagine a world where MUDs allow no client-side scripting or insist via an EULA that you use their own proprietary client, and then you can draw a comparison between the two.

Wizard Joined: 14 Oct 2000 Posts: 1368

Posted: Sat Jun 09, 2007 2:13 am

Back to the "how to market this" issue...I remembered that I had some mail credits left over in my MailChimp account from when I announced CMUD to all zMUD users last October. So, since the new zExplorer is a free upgrade for all existing AC Explorer Pro and zExplorer users, and because I need as much positive word-of-mouth as possible for the new version, I have just sent an email to all existing AC Explorer Pro/zExplorer customers to tell them about the new version. Hopefully this will generate some good buzz within the game.

Next week I will also resume work on the Zugg Software Affiliate program, where people can earn money by generating sales for CMUD, zMUD, zExplorer, etc from their web site. I think this will be very popular for LOTRO Guilds who want to promote the new zExplorer (as well as helping with CMUD from various MUD sites). I'm hoping I can finish the affiliate system within a couple of days and get it released quickly. Then I'll get back to CMUD programming.

Newbie Joined: 06 Jul 2007 Posts: 1

If you are interested in getting the *actual* location data to the EXACT position (and the location format they utilise) and tracking facilities for personal use with ZExplorer please drop me a line.

I have extensive knowledge of the lotroclient since I took it upon myself as a reversing project over the last couple of months and can obtain all the World entities within client visible area (approx 100.0f radius from your current position), including attributes about them.

I haven't had an opportunity to reverse the file format for the .dat files yet but was about to look into that when a search brought up your website with reference to ZExplorer's ability to read them, hence I would be willing to trade an eye for an eye so to speak to read these files saving me the time and hassle of reversing them.

It should be noted however that my methods involve client memory hooks / detours and so on to obtain all the information using injection and have implemented memory protection layer for any future detection schemes (currently they have none..)

You can have a nosey here if you like, www.isxlotro.com , should you be interested.

Keep up the good work Zugg.

Posted: Fri Jul 06, 2007 5:16 pm

Thanks for the offer. But I'm really not that interested in client hacking.

What I'm planning to do in the future is implement a "poor man's" tracking system where zExplorer can watch the contents of the clipboard, and in the game you just do a "/loc" command and then copy the result to the clipboard. zExplorer then detects the data in the clipboard and adds a tracking marker (cookie trail) to the map. This method would allow tracking in zExplorer to work with any game.

The tracking in AC Explorer where you needed the memory location in the client was a pain to support. It changed with every little patch. It eventually stopped working altogether when Turbine made big changes to their code. I don't really want to get into this kind of support headache for every game that zExplorer supports. Not to mention the fact that tracking would violate most EULA's these days.

Good luck with the .dat files...I'm sure someone else will eventually hack them too. I just wanted to be clear that I didn't support the kind of extensive hacking that you are doing in LOTRO. I think that kind of work leads to bots and macroers, and can end up ruining the game. And Turbine has made it pretty clear that this kind of client hacking is a violation of the EULA and can get you banned from the game...even if you do implement a "memory protection layer". It's not something I personally want to take a chance with. I try very hard to keep Zugg Software on the legal side of the EULA line with zExplorer.