Monday, March 18, 2013

More Data Hacking...

Although LayOut will handle the obvious visual mechanics of creating an OutRun track, things don't and can't stop there. In fact, OutRun's engine utilises an additional lookup table of pre-calculated values, required by the car physics and attract mode AI code.

This table, stored in the master CPU code, corresponds to the road path data, stored in the slave CPU code. Here is how three typical segments of road might be represented:


This means that at position 161, there's a left bend with property '121', until we reach the straight at position 277. This straight lasts until we reach the right curve at road position 359. This data is stored for every single curve and straight in every track in the game. 

Note, that this table does not define the path of the track at all from a visual point of view; this is purely used by the game's logic. If we get this stuff wrong, the car behaviour won't feel right when cornering. The value is used to adjust both the x position of the Ferrari as it enters a bend and to adjust your speed when you move through the corner. Without this, it would feel as if you drifted around corners too quickly. 

Understanding this table is all very well, but the problem is how to reverse engineer a curve back into this magical 'curve info' value required by the engine. Without knowing this formula, the editor project would be doomed!

I spent a long time experimenting with different ideas, largely aided by Excel and a bit of trial and error. In the end the solution was this:

For each curve in the game, calculate the average of the distances between the points on that curve. Therefore, you can use the standard distance formula for every pair of points:

Once you have the average distance between the points on a curve, you can do:

Curve Info = (1 / AverageDistance) * 4096

Where 4096 represents a 'one' in fixed point format. And finally we can spit out a curve info value in the format required by the game engine. 

Hopefully this gives you an idea of some of the less visually interesting work that I'm doing. 

37 comments:

Adrian Smethurst said...

This is truly fantastic news yt!

I've been digging through the Super Hang-On code for the past few weeks and had found the equivalent lookup table in amongst the static data for each pair of stages and was struggling to understand how the second word in each group of three words worked. The table in SHO for the first two stages of beginner class is at $16560 in the main cpu program for 70 words (23 triplets plus a terminating $ffff word). I'd figured out the first and third words of each triplet and assumed that the second word was some kind of a properties word but hadn't looked much beyond that.

Your time and effort on reversing this very difficult piece of the puzzle is much appreciated by me. It should now be possible for me to hand craft a new bend into a track and it'll hopefully feel correct as a ride around it.

I appreciate that LayOut is intended to be solely an Out Run track editor but will you at any point in the future consider adding SHO track support into it? It should be quite easy to do as an SHO track is simply a Out Run track with just a single road instead of two roads.

Thanks once again for your continued hard work on all this. I for one know just how tough some of this can be.

yt said...

I'd expect it to be trivial to add a Super Hang-On exporter for the roads, providing you know the location of the various bits in ROM.

And it sounds like you're making some good progress tracking down the relevant locations.

It might not support all the features (i.e. sprite placement), but shouldn't be too much work.

To be honest, the most time consuming part of this is actually just coding an editor that is easy to use.

Adrian Smethurst said...

It would be awesome if LayOut could one day support both OR and SHO stages!

I've not looked at the static data structure for an OR stage but for a pair of SHO stages the static data starts with 9 pointers which point to the following :-

1) palette for gradient filled sky
2) primary/secondary colour for the line that runs down the centre of the road
3) primary/secondary colour for the wider road edge marking
4) primary/secondary colour for the narrower road edge marking
5) primary/secondary colour for the road itself
6) primary/secondary colour for the ground plane
7) bend characteristics (thanks to you :-) )
8) height map info
9) trackside sprite placements

Is that similar to (or the same as) an OR stage? Hopefully!

What's the base address of the static data for Coconut Beach in rom? I'll have a quick look and compare for myself?

Thanks





yt said...

Coconut Beach can be found at 0x1DD36 in Revision B.

The format you detail in your post is identical. Note that address 8 also contains width info, at least for OutRun.

I wonder how similar the sprite storage formats are.

yt said...

So yes, looks like we'll be able to support both.

How is your C++? Maybe you want to write the exporter for Super HangOn. ;)

It should be relatively easy if we already have an OutRun exporter working.

Adrian Smethurst said...

Ok, I've had a look at Coconut Beach and compared it a SHO paired stage. There's a few differences, mainly due to the fact that OR runs two roads together versus the single road of SHO.

Here's a few things I noted :-

1) The first pointer (gradient filled sky) in a SHO paired stage always points to 32 words of palette data. In Coconut Beach the first pointer points to just a single word. This is presumably because Coco Beach uses a solid filled sky colour rather than a gradient filled sky (whereas all SHO paired stages use a gradient filled sky). It would be good to compare an OR stage like Gateway (which DOES use a gradient filled sky) to see if the first pointer on that stage points to a single word or 32 words. What's the start address of the Gateway static data?

2) Second, third, fourth and fifth pointers for Coco Beach all point to 4 words of data whereas those same pointers on a SHO paired stage only point to 2 words of data. This is presumably due to 2 roads on OR versus 1 road on SHO. This also allows some pretty neat stuff such as dual-coloured roads, as can be seen here :-

http://homepage.ntlworld.com/cmonkey/two_coloured_roads.png

3) Sixth pointer in both cases points to two words for the ground plane (although strangely enough both words are the same for Coco Beach, even though the ground plane is two-coloured!)

4) Data structure the for seventh pointer is identical (assuming the properties word operates the same in both game engines)

5) Eighth pointer - SHO simply uses two zero words for the road width words as the width never changes. Other than that the data structure is identical. How many height maps are defined in OR? SHO has 21 different height maps. Are the height maps stored in the slave cpu code in OR? (they are SHO)

6) The data structure for the final pointer (trackside object placement) seems identical except OR uses $7fff to terminate and SHO uses $ffff (but what's 1 bit between friends eh!). Other than it's just a word for object location and a word for object index (the second word does look rather different in OR but then again there's an AWFUL lot more trackside scenery in OR than there is in SHO so that may explain it). It always seems like I'm riding through a barren wasteland when I play SHO immediately after a blast on OR!

I haven't studied the sprite storage format in any depth yet so can't say how it compares to OR at the moment.

Unfortunately my C++ skills are slim to none! I can just about follow the Cannonball source but I'm an old school standard C kinda guy and even then my C skills aren't too hot. This obviously doesn't bode well in an object oriented world these days! I'm more of a reverse engineer than I am a forward engineer so I think it would be better to leave the coding to a professional such as yourself rather than destroy LayOut with my shabby efforts at coding! I'm more than happy to offer all the knowledge I've picked up in looking through the SHO code but you're the man when it comes to coding! :-)

Cheers

yt said...

I'll reply to this one in detail tonight when I have access to my disassembly :-)

Howard Casto said...

I've been busy with my own projects, and haven't had time to chime in but the progress you've made is amazing yt.

About the road format. I'm currently hacking outrun 2006 in my spare time and while the courses themselves are obviously 3d models, a lot of the collision data and re-usable objects apparently are placed via script files and for some of them the format you guys just described is awfully familiar.

For some of them there appears to be sections of data containing pointers and generic x/y/z values terminated by either a 0xff or 0xffff.

I haven't looked into it yet closely, my focus has been more on fixing 2k6's controls and hacking in force feedback but if you guys have a copy you might want to take a look.

Right now I can swap tracks out, but I can't find the script that controls where the road split model is positioned or the script that places the traffic, so when you get to the end of the stage you are faced with an invisible wall.

Unfortunately it might not be possible. Most of the files in 2k6
have some variant of sz (lz77) compression applied to them and without a point of reference I don't know how to properly decompress them. But that's yet another hurdle I haven't looked into yet.

yt said...

Howard, I have some random questions for you:

- Am I right in thinking the OutRun bonus minigame didn't make it into 2006? It would have been good to disassemble this and see what approach was used.

It was obviously in OutRun 2 as an unlockable, but that's only available on XBox AFAIK.

I doubt it helps you, but the road split in OutRun is completely hard-coded. It's as crude as:

if (road_pos >= ROAD_SPLIT_POS)
load_split_segment();

Have you posted your findings online anywhere? Would be fun to see them!

Howard Casto said...

Your going to have to refresh my memory in regards to the mini game. It's been 10 years. ;)

The game has a singular model for the road split. Somewhere, hopefully in one of the script files, there is some data that tells the game the y,x,z and rotational values needed to position it properly at the end of each individual stage. I know this because with certain swap-outs I can see the damn thing floating up in the sky!

It's actually rather clever from a programming standpoint. The split has that long corridor you go through with the high walls. The game is actually loading the next stage as you go through it.

In regards to outrun 2, I've got a copy of the xbox version and previous issues aside I should be able to swap the bonus stages (Daytona and Scud Race tracks ect..) except they are using a different form of compression than 2k6. 2k6 as I mentioned is using a variant of szip, while the xbox games use a variant of gzip. It might just be gzip without a header, but I'm not a compression expert, so I'm sorta stuck.

Online Arcade (the xbla/psn port) uses szip as well, but the files inside the container must be in another format because when I try to swap out the sprites folder (to get higher-res sprites) the game crashes.


The game is quite interesting. Much of it is completely modular and controlled by external files. Also there are strange bits of weirdness like the fact that the "outrun 2" selection from the main menu essentially loads a pseudo-emulated version of 2sp... all the usually memory locations go dead and they are moved.

Yeah my incoherent ramblings on the subject can be found here:

http://forum.arcadecontrols.com/index.php/topic,130891.0.html

I'm pretty close to my goals for the project.. I just need to find out the texture format/compression format so those 480i sprites can be swapped out, but I've got nearly all of the FF data at this point.

yt said...

That's really great progress. I had no idea that the 2k6 port was such a dog.

By the mini-game, OutRun 2 has the original OutRun embedded in it as an unlockable extra which you can play. It's a similar version to Shenmue 2 / Dreamcast GameWorks in terms of the engine. (i.e. not great, with no config options and some conversion quirks).

I was curious as to whether this version existed on the PC disk in any shape or form.



Howard Casto said...

Nah that's definitely not in there.

The sad thing is it, and the outrun2 bonus tracks SHOULD be on the pc version.

They removed it from the xbox/ps2 ports due to size restrictions, which is understandable, 2k6 has all the outrun 2 tracks and the sp tracks... they take up a lot of room.

But seeing as the pc doesn't have size restrictions I don't understand why the bells and whistles weren't added back in.

If I can get the pc port upgraded somewhat it really is the best version to work with. The car models were done with shaders and mip-maps instead of a lot of textures, so even today they look just fine even at 1080p. The only thing that shows the game's age are the sprites and road textures. Apparently the game auto-loads shaders though, so somebody might be able to upgrade the level textures a bit using shader tricks.

yt said...

Adrian:

1/ Gateway Data can be found at 0x1DB5E.
An alternate, unused version of Gateway can be found at 0x1C2B6. This version is the original layout found in the Japanese ROM.
Those sky values index into a palette table at 0x17590 in the master CPU code.

5/ OutRun has, wait for it, 225 height maps! Take that Super HangOn! You can find the height table at 0x220A in the sub CPU code. I'm working on that stuff at the moment.

6/ All that stuff (0x7FFF vs 0xFFFF) will be easy to handle in the exporter. The editor itself doesn't care about this stuff internally. It uses its own format, which is much nicer to work with.
Check out osprites.cpp and olevelobjs.cpp for details on the track sprite format in OutRun.
Otherwise, sit tight and I'll eventually document it in plain English :-)

What I'll do with the SHO exporter is send you a template to fill in when the time comes. Might not be for a couple of months.

As you can probably guess from my work on Cannonball, I have every line of the OutRun 68k code commented. So if you have questions, ask, or look at the C++ code as you can work back from that to the assembler easily :-)

bluepillnation said...

Taking an educated guess, I'd suspect that the versions of OutRun in Shenmue and XBox OutRun2 would have been tweaked emulation layers. The System16 DOS emulation by Thierry Lescot and JoseQ was around in the public domain by 1997, and the Dreamcast hardware was plenty capable of running it. Prior to the code being subsumed into the MAME project, there was a DirectX version as well - so porting to Dreamcast and XBox would have been relatively trivial.

The interesting port code-wise is probably the Sega Saturn "Sega Ages" version, as that would have not only been required to run natively, but would have had to undergo some serious reworking.

Great stuff by the way, yt - once more I doff my cap!

Adrian Smethurst said...

Ah, the sky data makes more sense when you realise that it's indexing a palette table rather than having the palette entries hard coded into the static data itself! So I presume, being as though there are 15 palettes (64 words to a palette) defined at $17590, that each stage has its own sky palette. I'm unsure whether a palette defines 32 colours or 64 colours as I don't know whether the palette data is word referenced or long word referenced but it's easier to see now why Coco Beach has a solid filled sky colour as all 64 words for it's sky palette use the same value whereas the sky palette for Gateway @ $17a4c defines multiple colours for its gradient filled sky.

225 height maps!!! At 15 stages for the entire game that means each stage can have 15 different height maps and you'd still never get height map repetition! SHO bows in the superiority of OR!!

Still can't understand how the ground plane colours work in OR as both values are same for Coco Beach (0001 0001) and Gateway (0000 0000). Ah well, I wont worry about that too much as you're the OR expert and I'm concentrating my efforts on SHO.

More than happy to help with a SHO exported when the time comes. First things first though, get it perfected for OR :-)

My ultimate aim with SHO is to do the same as you've done with Out Run by understanding and commenting every line of the 68k code. At the moment I'd say I'm about 30-40% of the way through it. Unlike you however I have no plans to rewrite the game in C++!! That must have taken a momentous effort on your part, one which very few people truly understand. Am I correct in thinking that you now have the OR source in a format that allows you to reassemble it to recreate a rom image which is an exact binary duplicate of the original rom? That's my aim with my work on SHO. I presume this is how you made the changes for the OR enhanced roms (unless you hand patched the changes into the originals).

I must say I think the addition in the Cannonball source of annotating where each routine can be found in the 68k code was an extremely helpful touch. One of the original drivers for me in starting to reverse SHO was because I wanted to add an enhancement of my own to Out Run in the way that the initial entry is handled when you make the top 20 (the other driver was that, after Out Run, it always was my favourite racer!). It always annoyed me that you couldn't erase the 3rd initial so that if you pressed the accelerator on the wrong 3rd initial you were stuck with the wrong initials on the high score table. I don't know whether this was a bug by the design team or it was always intended to be that way but nearly every other Sega game of the 80's always allowed you to backspace over the final initial should you enter it incorrectly. So I decided to reverse the initial entry routine of SHO (which allows you erase the third initial) and compare it to, initially the source of Cannonball, and then ultimately to the code in OR with a view to submitting a feature request to you to add to the enhanced roms one day.

However once I got started on reversing SHO I found myself saying "well I'll just reverse this one more routine and then I'll call it a day..." and before you know it I'm nearly halfway through the code! I've reversed quite a few Spectrum and C64 games in my time but this is by far the biggest game I've undertaken. One of the biggest challenges is figuring out which areas of memory are shared by the main/slave/z80 cpus and which areas are private to each cpu. It's an incredibly satisfying experience though when you finally fully understand a chunk of code that's puzzled you for a few days! It's also cool when, just as you did with OR, you find unused lap timer code in the game, as can be seen here :-

http://homepage.ntlworld.com/cmonkey/sho_laptimer.png

;-)






James said...

yt, I'm playing with the code to add an additional shifter mode, specifically that supports two separate buttons for H and L gear. That part works fine, but I can't find the code that displays the gear next to the speed. There's obviously something conditional about it, in that it will display in GEAR_BUTTON mode but not in GEAR_PRESS mode, and I need to enable the display for the additional mode I've added. Just not sure where or what to look for. I'd have guessed it was in ohud.cpp but nothing stands out.

Thanks,
Endaar

Howard Casto said...

I would be curious to see how many others games use a similar format as well.

Turbo Outrun would probably fall into this area as would many of the less popular sequels building up to Outrunners.

yt said...

Howard: I would expect Hang-On and Enduro to be similar at a guess too. I'm not sure how similar the Turbo OutRun codebase is, as I haven't analysed it in detail. Turbo OutRun isn't one of my favourite AM2 offerings. I really wish they'd retained the car handling and camera viewpoint of the original.

James: The code you are after can be found here:
https://github.com/djyt/cannonball/blob/master/src/main/engine/oinitengine.cpp#L295

Why is it located at this point of the codebase? Who really knows. The original codebase is not very modular and does stuff like this a lot. Refactoring the game is possible, but also likely to add bizarre bugs.

bluepillnation: I'm not sure its System 16. That emulator used a heap of x86 assembly, and the code was a real mess. Not to knock the amazing work those guys did on being the first to emulator the System 16 and OutRun platforms. Looking at the Dreamcast binary, I suspect it's a custom rewrite from the original source code.

The Saturn hardware is essentially an evolution of the OutRun -> XBoard arcade series, so I believe that port would have been facilitated with automated conversion tools. I still haven't got to the bottom of why the Saturn 60fps mode feels so strange (like you're driving in slow motion). Obviously Cannonball doesn't feel like that and maintains the same sense of speed as the original 30/60 arcade hybrid. Answers on a postcard please.

yt said...

Adrian: For ground and road palette enquiries I refer you to:
https://github.com/djyt/cannonball/blob/master/src/main/engine/opalette.cpp#L522

Also check the references to the palette tables in ROM here:
https://github.com/djyt/cannonball/blob/master/src/main/engine/oaddresses.hpp#L163

Those chunks of code should answer all your queries on palettes.

Regarding memory usage, the best reference is the MAME memory map for the game which will answer all your questions.

I have a full IDA based decompile of the main CPU, sub CPU and Z80 CPU code for OutRun. I have also disassembled all the revisions of the game to understand what changed between versions. OutRun Enhanced Edition came about as a result of Darren Finck writing to me with an idea to add high-score saving to the game. He'd been studying the schematics and realised the PCB had battery backed up RAM. I said "Aha, guess what I have an entire disassembly and can tell you where to the insert the routines you need". I worked in parallel with him to fix bugs and add a proper Freeplay mode. Amusingly, and by coincidence, I had ported one of Darren's emulators to the Dreamcast over 10 years earlier!

I do not have the current IDA code recompiling. In fact, if you check Enhanced Edition you'll see I wrote a Java tool that reads a config file and inserts chunks of assembler directly into the ROM and patches the checksums. This approach works well for simply hacks, but obviously for something more involved I would need to do a full recompile. It's not something I've attempted yet, although I would like to. I was so focused on Cannonball at that point, I couldn't find time to do that as well. I also came to the realisation that a lot of the enhancements I wanted to make needed a full rewrite (60 FPS, widescreen support, extra game modes). I was also more interested in documenting and preserving OutRun for future generations, as opposed to writing a limited version that would only run on a brittle and increasingly rare hardware platform! Plus the testing involved when adding new features to Enhanced Edition is somewhat involved, as you need to program the ROMs and run on a PCB etc.

Whilst Cannonball would not be possible without emulation and the efforts of the MAME team, emulation itself doesn't really document the game, only the hardware it runs on. The game remains a mystery, a black box of sorts, and much of the history and development story is locked in the game code itself. Part of the reason for this entire project was to generate more technical interest in a game I loved, and hopefully inspire other people to do the same thing, like you are doing with Super HangOn.

James said...

Thanks yt, much appreciated. James

James said...

yt, couple more questions if you don't mind...

First, what's the rationale behind disabling scanlines unless the scale is proportionate? I forced scanlines enabled in full-screen mode, and while I'm not a fan of the effect, it seems to work fine.

Second, would you mind elaborating a bit on the force-feedback implementation? I have a Logitech Momo which is showing some odd behavior. For one, the centering spring apparently won't disable. I think that's a glitch with the DirectX sample code because I've played with it previously and never managed to get the spring off. But it's hard to tell because of the other problem I'm having...even when the car is not moving, the Momo constantly moves left and right, perhaps 20 degrees each direction. The vibration effects themselves seem fine if you go offroad, etc., but there's something quirky causing the wheel to move when it shouldn't. Anything you think I should look for?

Much appreciated.

yt said...

Details of the scanlines can be found in the faq.

They "might" work ok in full-screen mode, dependent on what the final scale factor is, which depends on your screen resolution. But it's not a safe approach to enable them.

For the FFB stuff: The centre spring is supposed to be enabled as the original machine had spring based centering.

Sounds like you need to reduce the overall strength of the FFB as outlined in the manual. The default settings work out the box for me, and some others. But you'll need to play around to find the optimal setup for your hardware.

Kungfu Steve said...

The wheel, and the FFB unit are actually 2 separate devices. The wheel on outrun is always center sprung. It uses a pot to collect values. The ffb device on the non cockpit games, is simple.. they just use a piston crank arm, attached to a one way motor. The entire wheel unit sits on a sliding platform. The ffb motors arm is attached to the platfrom, and violently slides the entire unit about an inch each direction...at a rate of speed determined by the game events.

Kungfu Steve said...

This is all really cool. I also am a big Super Hangon fan, so seeing a mod for that would be awesome.

Im also currious to see a mod for a 4 way shifter in outrun... with real tach responses. That would really add some awesome challenge to an already great and challenging game.

The new pc outruns look nice.. but they feel so slow, and are just so boring.

Hurrah for the original feel. Theres not many games like it. Not even close.

Kungfu Steve said...

Its interesting to see that they separated the road grip from the actual corner data. One would simply use a pre-fabbed speed to grip formula.. however, this method does open up some more interesting possibilities.

For example, if the terrain is loose gravel, sand, snow...ect, the grip can be effected. Also, to make a unique and increased difficulty challenge on certain curves.

James said...

Thanks yt. I've added code for an additional shifter mode (separate buttons for H and L) as well as the appropriate XML and menu options. I've also added the ability to select the joystick # via the config file.

Do you have any interest in this, and if so, how should I get it to you.

Thanks again,
James

Adam said...

I know you're busy with the track editor at the moment, but I've had an interesting idea for a future release of Cannonball, providing it can be done.

Would it be possible to put in some kind of endurance mode, where you go through all 15 stages together? ie Coconut Beach, then Gateway, Devils Canyon, Desert, Alps, Cloudy Mountain etc all the way through every stage, ending with Lakeside.

I think if this could be done it would be a great addition, to try and beat the time limit through all 15 stages in one go.

Not in any hurry to eee it implemented if it can be done of coure, but just an idea for looking at for a future release.

yt said...

James: You could create a diff of your code from the master branch of my git repository and send it to me. Then I can integrate it :)

Adam: Yep, certainly possible to do that in the future I would think!

James said...

yt, if you could point me in the direction of exactly what you need (on Windows), I'll be happy to. I haven't worked with source control before (I'm a Network Admin) so while I know what a diff file is, I don't know how to create one particuarly in the format you're looking for.

Thanks, James

yt said...

Try the info here.

Essentially you just want to create a patch that records the differences between your local source, and the master branch of the remote git repository.

Alternatively, if you know what files you changed, you could just send me the new files so I can make a comparison.

James said...

Thanks yt. I was able to get git working via the link you provided but unfortunately I think Visual Studio nuked your indents, so the diff between the master and my changes is misleading. If you let me know how to send the changed files to you, I'll do so. It's six files total, probably not more than a few dozen changes, and they're marked as such.

Thanks,
James

yuriks said...

yt,

Just finished reading through your entire blog archive. A very inspiring tale! 3 years of hard work that produced a perfect reproduction in the end, with more to come...

And I have a similar dream. As someone who has done it before and lived to tell the tale, what would be a few hints you could give to someone who wants to attempt a similar REing project?

The main question I have is: do you think it's better to do the C port as you go along discovering the original assembly, or do the entired REing work upfront before beginning a port. It seems to me you did the latter. Do you think that's the best option?

Good job and congratulations on your hard work.

yt said...

yuriks: There is no right or wrong answer to your question. It probably depends on the scale of the project you're working on. Certainly both routes are possible and I'm still not sure which approach would have been better in my case. You learn a lot through decompiling everything. But you also learn a lot through rewriting the code. It's an incremental process, and the increase in knowledge is proportional to the amount of time spent.

The main tool at your disposal is being able to edit and tweak the original code to observe the effects. A solid knowledge of the mame debugger is one way of achieving this, and rewriting the code into an easily changeable format is another.

If you let me know what project you are planning to embark on, maybe I can provide some ideas.

yt said...

James: Sorry for the delay. See this page for details on sending the changes to me. I'll review them in due course.

yuriks said...

yt: My project is porting the DOS version of The Lost Vikings, one of Blizzard's first published games.

I'm using IDA and also the DosBox debugger. While it works well, it has a few weird behaviours that make is somewhat annoying, but by far the most annoying part is not integrating with IDA in any way to be able to read variable and code labels. I don't know how the MAME debugger compares.

I had actually already started on this a several months ago, but lost motivation when the work got boring and grueling. This time I've been focusing more on understanding the game code completely rather than porting things over even if I don't know how they work. It seems to be working out better for me so far, and I'm instead writing tools to inspect the format of the data and levels. And I agree, playing with the variables or execution flow of the game while it executes is invaluable.

The game code is very complex. Most of the game logic is in scripts interpreted by a custom interpreter, supporting about 200 different opcodes. The kicker is that the scripts (which are in external data files) directly reference variables in the game by RAM address, making them highly unportable. I'm still thinking about how to handle this. (The number of addresses accessed is large enough that an address -> variable mapping, which was my original approach, won't be viable.)

yt said...

yuriks:Yes, the most difficult aspect with a hobbyist project of this nature is simply ploughing on for what can be months if not years, often with little in the way of satisfying results.

I found choosing a game I really loved helped. I certainly wouldn't do a project of this scale again, for fun due to the amount of free time needed. Maybe commercially. I guess that's why you don't see many hobby projects of this nature; coding an emulator is far easier!

I had no way of linking IDA with the MAME debugger. It's possible I could have written some kind of script to convert the IDA comments to MAME debugger comments. But I never tried that.

I don't really understand why you couldn't just use some kind of quick and dirty hashmap to map addresses to variables. But I suspect there's something I've missed there.

matt said...

Another guy considering RE'ing my favourite game. For me that's another SEGA game: Flicky (1984).

I have more modest goals, in that all I want to do is find the logic that governs when a diamond is dropped by a cat when it is knocked out. No rewriting, or disassembly... yet!

So, I needed to know how to use the MAME debugger. I recently found a couple of tutorials for using the MAME debugger for these purposes:

Ikari Warriors
http://www.paladingrp.com/brianb/rom_hacks/ikari.html

Robotron 2084
http://dorkbotpdx.org/blog/skinny/use_mames_debugger_to_reverse_engineer_and_extend_old_games