Ripping Sega System 16 Sprites & Palettes: A Brief Guide

I recently updated and released the source code to Sega Sprite Viewer. This tool allows you to view the contents of sprite roms from many popular Sega games of the era. 

Hardware that (should) be supported includes:

• Sega Hang-On Hardware (including Enduro Racer & Space Harrier)
• Sega System 16 (Golden Axe, Altered Beast, Shinobi etc.)
• Sega System 18 (Moonwalker, Shadowdancer etc.)
• Sega OutRun  (including Super Hang-On)
• Sega X-Board (AfterBurner, Thunderblade etc.)
• Sega Y-Board (Power Drift, G-Loc etc.)

Part 1: Viewing Sprites

Let's take an example System 16 game, in this case Altered Beast. First up, we can establish the configuration of the sprite roms by looking at the MAME System 16 driver here. 

We can see the sprite roms listed above in the highlighted yellow section. We can also see how these are configured in memory. The first step is to copy the sprite roms into their own directory. These should be referenced by creating an XML configuration file for the sprite viewer called altbeast.xml.

• Format 0 specifies System 16 sprite format. 
• This example places the sprite roms in a sub-directory named roms/altbeast/ 
• The offset and length of the files are configured to match the MAME driver.
• You can view the final configuration file here for reference. 

Now we can load the sprites into the viewer by passing the name of our XML file as a command line argument:
s16_viewer altbeast.xml

Everything looks good. And you could stop here if you wanted. But you might be curious as to why the graphics are greyscale...

Part 2: Extracting Palettes

The sprite roms themselves don't contain colour palette information, or in fact anything particularly useful beyond the raw pixels. Palettes are contained in the actual 68k CPU program code. The process for extracting the palette is different on a per-game basis. But the general principles remain the same. Here, I'll discuss how I extracted the palette data for Altered Beast.

Establish the location of Palette RAM

Altered Beast is a System 16 based game. For System 16 games, the location of palette RAM is allocated dynamically at run-time by the Sega 315-5195 Memory Mapper. (Note, that other Sega hardware titles will configure the location of palette RAM differently - the point is that it's a sensible first step to determine its location).

Using the MAME debugger we can view the configuration of the memory mapper once the game is running:

• View the 315-5195 Memory Mapper m_regs configuration.
• The yellow highlighted value ($84) at offset $1d contains the details we are interested in.
• This value is left shifted by $10, giving us the location of palette RAM: $840000
• Often the sprite palette is at offset +$800 within the palette RAM. Therefore $840800. (I know this part through pattern recognition, but it can also be established from decompiling the code further, as we shall see later.)

I can sanity check $840800 is the right location for sprite palettes by filling its contents with garbage whilst the game is running.

Here, I've filled the fast part of RAM with the value "$1111" repeatedly. As you can see from the circled sprites, these have lost their palette information and are rendered as black silhouettes.

Now, we could just save the contents of palette RAM at $840800 into a file and load that directly in the sprite viewer. However, this wouldn't contain all the palettes the game uses - only the ones loaded into memory at this moment in time. Therefore, it wouldn't be particularly comprehensive. We would find that many palettes were missing.

Establish where Palette RAM is populated from

If we can establish where the palette RAM is populated from within the code, then we are highly likely to locate the data table containing all the palette information. We can set a memory watch point at $840800. This will informs us when palette RAM is written to, and the part of code responsible for this.

I've set a watchpoint at $840800-$840804. Whenever a value is written to this address program execution halts. 

We can now view this section of code that copies data into sprite palette RAM in more detail in the MAME disassembler. I've annotated this with some comments, in green, on the right hand side. 

The main takeaways from this section of code are as follows:

• Address register A2 points to a table of address information containing the source address in ROM and destination address in palette RAM to move the values between.
• Address register A0 is loaded with the source address. (a location in the program ROM)
• Address register A1 is loaded with the destination address. (sprite palette RAM)
• It should be noted that 28 bytes are copied, not the full 32 bytes required by sprite hardware. This is important palette format information we will need later.

Now we can step through this routine to get some clues as to where the palette data is originally located. However, as all addresses are setup from address register A2, we need to mine deeper into the code still, to establish where this table of pointers is first configured. 

By watching the memory location specified by A2, we can work our way back to the following program code:

From this section of code we can not only establish where the first palette entry is stored in rom, but also the total number of entries. This key points are as follows:

• A byte is used (0-255) as an index into the table of entries. So we can assume that the palette table contains 255 entries max (highlighted in yellow and loaded to D0).
• This routine specifies $242a0 as the first sprite palette entry (highlighted in yellow and loaded to A1).   
• We established each entry was 28 bytes previously. This means the table is $1be4 in length (28 bytes * 255 entries).
  

This gives us the final solution to our puzzle. We know where all the palette entries are first stored, before palette RAM is even accessed. We also know how many entries are in the table. 

Save the Palette table to a Binary file

We can now save the palette as follows using the MAME debugger:

save altbeast.pal, 0x242a0, 0x1be4, 0

This can be added to our sprite viewer XML configuration as follows:

We specify the number of bytes per entry, and the fact that the palette data is offset by 1 word when displayed by hardware. Essentially, the sprite hardware expects 32 bytes per entry, but the first and last words are not populated in the palette data.

Viewing the Palette

Finally, we can load the sprite data and view it in glorious colour by cycling through our 255 palettes. One limitation is that we still have no way of automatically connecting a palette to a sprite. This information is also contained in the 68k program code and is unique on a per-game basis.

I hope this has been informative! If you successfully create both a palette and configuration file, let me know and I'll add them to the source code repository. 

CannonBall V0.31 - Maintenance Release

This release focuses on upgrading the libraries and compilation tools CannonBall uses, as I hadn't maintained the codebase in a number of years. Most of these changes will be invisible to most users. Right now, I'm trying to get the house in order as opposed to add lots of wild new features! :)

The most exciting news is the upcoming SmartyPi support, but until the hardware is released, that's kind of a mute point!

Changes:

[audio] Audio updates at the correct rate and resolves the longstanding issue with music and sound being very slightly 'off'

[roms] ROMs are now read by CRC 32 value. Filenames no longer matter - so long as they are present they can be renamed to anything.

[roms] Fixed expected Z80 rom file length

[controls] Start Button behaviour less 'sticky' and buggy

[controls] Analog axis for accelerate and brake can now be configured via the in-built menu system

[menu] Reduced delay when scrolling through menu with analog controls

[config] ROMs and save data can be relocated to separate locations

[bug fix] Time Trial mode no longer crashes if used as the first mode played

[timing] Code tries to use V-Sync for timing OR internal timing, as opposed to both at once

[source] SDL 2 used by default. SDL 1 removed from codebase. This appears to have fixed compatibility bugs for some people

[source] Added compatibility for upcoming SmartyPi hardware (Pi 4 based) to run on original arcade hardware

[source] A general clean-up in many areas

Sega X-Board Memory Test Software

A set of ROM images to test the RAM ICs and custom chips on Sega X-Board hardware (AfterBurner, Thunderblade etc.). It is more robust than the on-board tests and stands a better chance of running on a dead boardset.

• It does not require working main RAM to actually run the main RAM test.
• Remove all sub CPU EPROMs when installing (IC 20, IC 29 etc), as these interfere with the results.
• It requires a vanilla 68K CPU to be installed, not the FD1094 security processor present on some X-Board games.
• The palette will be incorrect when used on games other than AfterBurner. But it should still operate correctly. 

This was not previously released, because I hadn't verified the IC labeling on hardware. However, a number of people have already used this software to successfully fix PCBs. Therefore, I figured I should get this out there and address problems as they are reported.

This is based on the OutRun Memory Test. The modified (and messy) source code is available here. 

The compiled ROM images can be downloaded here. 

OutRun: Enhanced Edition 2.02

OUTRUN: ENHANCED EDITION V2.02

OutRun: Enhanced Edition is a set of 7 replacement EPROMs intended for use on original OutRun arcade hardware. 

It fixes many bugs present in the final official codebase (Rev. B), and introduces new features to extend the life of the game, including: 

• Working Free Play Mode
• High Score Saving
• Additional High Score tables
• 3 additional in-game audio tracks
• Best Track Time (aka ‘Lap Time’) records
• New and old course layouts
• Software DIP Switch support
• Cheats - including infinite time and the ability to disable traffic
• Optional car handling modifications

Full documentation and installation instructions

To register:
1/ Complete this form

2/ Consider an optional charity donation

Space Harrier: The protection strikes back!

This is a guest post from Adrian Smethurst. 

It was back in January 2015 when I first started looking into the Space Harrier code on reports of a ‘bug’ which gave the player ‘extra' lives when they lost a life.  This ‘bug' only affected the game when played in either MAME, on an Enduro Racer converted board or a bootleg Space Harrier board.  It turns out that it wasn’t a bug but rather a time delayed protection mechanism created by Sega to try and hit the arcade operators, who bought bootleg Space Harrier boards back in the mid 80’s, in the pocket.  You can read more about that here.

Space Harrier PCB with Intel 8751 Microcontroller

Fast forward 5 years and a conversation I was having with respected indie game dev and creator of Fortress Craft, Adam Sawkins (ex. Criterion and Codemasters) about Space Harrier, at Arcade Club Leeds.  He asked me why, after the game had been powered up for a while, the enemy shots would start to come at the player faster and faster, to the point where you’d need lightning quick reflexes to simply avoid them.  He also told me that a reboot of the game would reset the enemy shot speed back to normal again, for a short while.  Hearing this news immediately sent my mind drifting off to a time 5 years previously when I’d investigated (and presumably fully defeated) the time delayed protection in Space Harrier.

So that night I went home and stated looking further into the Space Harrier code…

The heart of the protection is built around the ‘in-game’ timer.  This is a 6-byte timer located at address $40020 in main CPU address space.  The first 4 bytes of the timer represent the ‘seconds’ of in-game time played and the final 2 bytes represent the sub-second (frame) timer.  The timer is reset to zero at the start of each new game.  The timer is only incremented during normal gameplay, attract mode gameplay doesn’t increment the timer.  Every vertical blank the sub-second timer is incremented and compared with a value of 61 (yes, it’s a bug!).  If the sub-second timer is greater than 61 then it’s reset to zero and the ‘seconds’ part of the timer is incremented.  This means that, due to the bug, 1 second on the ‘in-game’ timer is actually 62 frames of gameplay, rather than 60.  The code that increments the ‘in-game’ timer is at address $1514 (it also updates the power-on timer located at memory address $40000 at the same time).

During every frame of normal gameplay code at address $4aae checks if the ‘in-game’ timer is at a multiple of $200 (this should, in theory, have been every 512 seconds but due to the bug mentioned above is actually every 529 seconds) and if it is the following code is executed :-

004ACC: move.w  $12444e.l, D0   
004AD2: move.w  $404ee.l, D1
; d0 and d1 never seems to contain a value other than 0 after many hours of gameplay testing
004AD8: eor.w   D1, D0          ; d0 = d1 XOR d0               
004ADA: cmp.w   $40090.l, D0    ; compare d0 with value at $40090.w
004AE0: blt     $4af0           ; branch if d0 < contents of $40090.w

If the 'branch less than’ condition is false (i.e. IF d0 < contents of the word memory @ $40090) then

004AE4: addq.w  #1, $4008e.l    ; trigger ‘previously unknown’ protection
004AEA: addq.w  #1, $400f0.l    ; trigger the 'increase lives' protection

When I was investigating the protection routine 5 years ago I completely failed to consider the first of these 2 instructions and only concentrated my efforts on investigating the second one.  This is because, at the time, I’d only been made aware of the ‘increasing lives’ issue.  So at the time I worked backwards from the ‘number of lives' counter to find out what was increasing the number of lives.  From there my investigations led me back to the instruction at address $4aea.  At that point I wrongly assumed that the ‘increasing lives’ issue was the only side effect of the protection routine.

How wrong I was…

The word value at address $4008e is directly related to the game difficulty.  It’s a value that's set initially by the boot-up code reading the ‘difficulty’ settings from dip switch B and is set as follows :-

EASY/MEDIUM     0
HARD            1
HARDEST         2

The code which does this is at address $2d7c.  This value is then subsequently used as part of the calculations in the routine which handles the enemy shots at address $b9d2.  It’s easy to tell that that routine handles enemy shots as if you change the first word of the routine from $08ed to $4e75 (RTS) the enemies will no longer fire shots at the player during gameplay.  The higher the value at address $4008e, the faster the enemy shots head towards the player.

Ordinarily the difficulty value at address $4008e would NEVER change after boot, assuming the game code is running on a genuine Space Harrier board with the 8751 MCU present.

However, as you can see from above, the instruction that I completely ignored 5 years ago, INCREMENTS that value (and hence the difficulty level) every 529 seconds of in-game play.

What this effectively means is this - if you power up the board from cold, ensure the difficulty setting on dip switch B is set to EASY and complete the game (assuming roughly 18 minutes for a full playthrough, although I have seen people complete the game in around 17 minutes) then the difficulty will be at the HARDEST level by the time the game completes.

And it will keep getting harder for each 529 seconds of completed ‘in-game’ time.  This is because the value at $4008e is only reset by either rebooting the game or dropping in and out of service mode.  It ISN’T reset at the start of each new game.

It’s very easy to see the results of a higher value at address $4008e by simply changing it via the MAME debugger and playing the game.  Values of 6 and above (which would represent just 3 or 4 full playthroughs from starting on EASY difficulty) make the game almost impossible to play as the enemy shots head towards the player with such high velocity..

Taking a step back, I feel it’s likely that the 8751 MCU probably exposes a value of 1 at address $40090 (it could in theory be any value between 1 and $7fff for the conditional branch instruction at address $4ae0 to pass but 1 is the most likely value, IMHO).  The updated patch has been added to the Sega Enhanced package here.

I hope that this finally lays to rest the protection in Space Harrier.

Build Your Own HangOn Cab: Dorimaga Papercraft Model

Here's another model from Dorimaga's Papercraft series. This time, Vol 2 - HangOn!

High resolution versions available here

Bringing Turbo OutRun Audio to OutRun: Rush A Difficulty

Following Camino’s optimization, I performed a similar treatment on Cruising Line, the remaining 3DS track. I reduced the track’s filesize from 24K to 9K using a similar set of techniques. Cruising Line does not suffer from the quantization issues that plagued Camino, which made the process a little easier and yielded even better results. So far so good and there was plenty of ROM space left to stuff with additional music! The next Enhanced Edition will contain three new audio tracks, which is an amazing result.

Originally, I considered bringing the new Switch music into the fold: Radiation and Step On Beat. However, from a subjective point of view, neither of these tracks are particularly great. I felt like they didn’t sit harmoniously with the existing music, and I wasn’t prepared to spend many weeks optimizing music I didn’t love.

Turbo OutRun - A prime example of what happens when you don't understand your own product.

Instead, I turned to another reference point in the OutRun universe - Turbo OutRun. Whilst Turbo OutRun is arguably a disappointing sequel, the soundtrack is impressive. In particular, Yasuhiro Takagi’s ‘Rush a difficulty’, which is an upbeat number that wouldn’t sound out of place in the original game. As an aside, Takagi went on to become sound director for Shenmue II, before moving to the Yakuza series.

Listen: Rush A Difficulty

Terrible Name. Amazing Track.

Turbo OutRun runs on the same hardware as its predecessor so, on the surface, the idea of converting the music might appear simple. Being a hand-crafted piece of MML, we wouldn’t need to worry about the rigorous optimization process required by the 3DS audio. However, the audio engine embedded in the Z80 program code isn’t identical. Between OutRun and Turbo OutRun Sega added a number of improvements to the engine. Firstly, an extra 3 PCM channels can be utilized by music, bringing the overall number of simultaneous samples to 8, bolstered by the usual 8 FM channels. (On OutRun, these 3 channels are strictly reserved for sound effects and can’t be used by music.) Secondly, samples can be played at different pitches. Let’s say the composer took a sample of an electric guitar chord, this could be triggered at different pitches and replayed like an instrument. AfterBurner used this functionality to great effect with its guitar-laden riffs. Whilst the samples are 8-bit, and relatively lo-fi compared with clean Yamaha FM patches, they add depth and grit to the overall mixdown when used wisely. In order to backport the music to OutRun, considerable changes would be needed.

So, the Turbo OutRun engine uses additional channels and manipulates sample pitch intelligently. It was time to decompile the necessary sections of Turbo OutRun’s Z80 code to start analyzing the raw music data. A starting point was the PCM channels, as we potentially needed to remove or remap the extra ones. It was immediately clear that 2 PCM channels were permanently disabled. Interestingly, the disabled channels contained an early draft or a guitar riff for the tune that sounded unfinished when reactivated. This was good news, as it meant there was only one extra channel of audio to worry about. The extra channel contained a sampled driven slap-bass line. Converting this back to OutRun would be problematic. It would involve finding space for the slap-bass sample in the, almost full, sample ROMs and backporting the pitch manipulation code. Plus there wasn’t a spare PCM channel to use anyway, so this was a non-starter.

The bass line was an essential ingredient of the track - it sounded sparse without it. I decided to recreate the bassline as a YM patch/instrument. CMonkey had the great idea of sourcing a patch from a Megadrive rendition of Rush a Difficulty. The patch wasn’t perfect, but proved a good starting point for further manipulation. I used the VOPM plugin, which emulates the Yamaha 2151 chip, to modify the patch further, before converting the data back to the format required by the OutRun engine.

VOPM Plugin. Spend ages fiddling with knobs

A YM patch will never sound as beefy as a sample, but it’s not a bad compromise. I replaced a, sparsely used, existing YM channel that didn’t contain a strong lead with the bassline.

Audio Comparison

The next hurdle was remapping the track’s percussion. The Turbo OutRun music utilises a different set of drum samples. Now, we could theoretically replace all six sample EPROMs on the PCB with larger ones to include these new drums, and solder the corresponding jumper. But at a practical level, this seems like a big ask on the poor user just for the drums on a single track! Most of the Turbo OutRun drums have an equivalent in OutRun - kick drum, snare, hi-hats, tom-toms etc. Whilst the OutRun drumset doesn’t contain as much reverb, this seemed like a sensible compromise for now.The only one that’s missing is the cowbell, which I mapped to a wood rim instead.

One final change was needed. The entire Turbo OutRun engine runs at a different timing value to OutRun. To work in OutRun, the engine needs to be temporarily patched to the Turbo value, but only whilst the music is playing. I have a temporary fix for now, which will need to be improved before release. So finally, the track is successfully converted. The main differences are: remapped drum samples, the sampled bassline replaced with a YM patch, with the resulting loss of a single YM channel.

Listen: Sunset Rush (The Enhanced Edition Remix)

So there we have it. A different challenge to optimizing the 3DS music that entailed rewriting existing tooling, decompiling the Turbo OutRun audio engine and converting the MML data and commands to an older format.

I’d also like to thank cmonkey, without his assistance this would have taken much longer. When working on a project of this nature it's invaluable to have someone to bounce ideas off, challenge your assumptions, and sometimes make you feel (unintentionally) ridiculous. I've been incredibly lucky to find someone who understands the Sega audio engine as well as he does, and I wish I could say more than, "thanks buddy!"