Monday, April 12, 2021

Ripping Sega System 16 Sprites & Palettes: A Brief Guide

I recently updated and released the source code to Sega Sprite Viewer. This tool allows you to view the contents of sprite roms from many popular Sega games of the era. 

Hardware that (should) be supported includes:

  • Sega Hang-On Hardware (including Enduro Racer & Space Harrier)
  • Sega System 16 (Golden Axe, Altered Beast, Shinobi etc.)
  • Sega System 18 (Moonwalker, Shadowdancer etc.)
  • Sega OutRun  (including Super Hang-On)
  • Sega X-Board (AfterBurner, Thunderblade etc.)
  • Sega Y-Board (Power Drift, G-Loc etc.)

Part 1: Viewing Sprites

Let's take an example System 16 game, in this case Altered Beast. First up, we can establish the configuration of the sprite roms by looking at the MAME System 16 driver here

We can see the sprite roms listed above in the highlighted yellow section. We can also see how these are configured in memory. The first step is to copy the sprite roms into their own directory. These should be referenced by creating an XML configuration file for the sprite viewer called altbeast.xml.

  • Format 0 specifies System 16 sprite format. 
  • This example places the sprite roms in a sub-directory named roms/altbeast/ 
  • The offset and length of the files are configured to match the MAME driver.
  • You can view the final configuration file here for reference. 
Now we can load the sprites into the viewer by passing the name of our XML file as a command line argument:
s16_viewer altbeast.xml

Everything looks good. And you could stop here if you wanted. But you might be curious as to why the graphics are greyscale...

Part 2: Extracting Palettes

The sprite roms themselves don't contain colour palette information, or in fact anything particularly useful beyond the raw pixels. Palettes are contained in the actual 68k CPU program code. The process for extracting the palette is different on a per-game basis. But the general principles remain the same. Here, I'll discuss how I extracted the palette data for Altered Beast.

Establish the location of Palette RAM

Altered Beast is a System 16 based game. For System 16 games, the location of palette RAM is allocated dynamically at run-time by the Sega 315-5195 Memory Mapper. (Note, that other Sega hardware titles will configure the location of palette RAM differently - the point is that it's a sensible first step to determine its location).

Using the MAME debugger we can view the configuration of the memory mapper once the game is running:

  • View the 315-5195 Memory Mapper m_regs configuration.
  • The yellow highlighted value ($84) at offset $1d contains the details we are interested in.
  • This value is left shifted by $10, giving us the location of palette RAM: $840000
  • Often the sprite palette is at offset +$800 within the palette RAM. Therefore $840800. (I know this part through pattern recognition, but it can also be established from decompiling the code further, as we shall see later.)
I can sanity check $840800 is the right location for sprite palettes by filling its contents with garbage whilst the game is running.

Here, I've filled the fast part of RAM with the value "$1111" repeatedly. As you can see from the circled sprites, these have lost their palette information and are rendered as black silhouettes.

Now, we could just save the contents of palette RAM at $840800 into a file and load that directly in the sprite viewer. However, this wouldn't contain all the palettes the game uses - only the ones loaded into memory at this moment in time. Therefore, it wouldn't be particularly comprehensive. We would find that many palettes were missing.

Establish where Palette RAM is populated from

If we can establish where the palette RAM is populated from within the code, then we are highly likely to locate the data table containing all the palette information. We can set a memory watch point at $840800. This will informs us when palette RAM is written to, and the part of code responsible for this.

I've set a watchpoint at $840800-$840804. Whenever a value is written to this address program execution halts. 

We can now view this section of code that copies data into sprite palette RAM in more detail in the MAME disassembler. I've annotated this with some comments, in green, on the right hand side. 

The main takeaways from this section of code are as follows:
  • Address register A2 points to a table of address information containing the source address in ROM and destination address in palette RAM to move the values between.
  • Address register A0 is loaded with the source address. (a location in the program ROM)
  • Address register A1 is loaded with the destination address. (sprite palette RAM)
  • It should be noted that 28 bytes are copied, not the full 32 bytes required by sprite hardware. This is important palette format information we will need later.
Now we can step through this routine to get some clues as to where the palette data is originally located. However, as all addresses are setup from address register A2, we need to mine deeper into the code still, to establish where this table of pointers is first configured. 

By watching the memory location specified by A2, we can work our way back to the following program code:

From this section of code we can not only establish where the first palette entry is stored in rom, but also the total number of entries. This key points are as follows:
  • A byte is used (0-255) as an index into the table of entries. So we can assume that the palette table contains 255 entries max (highlighted in yellow and loaded to D0).
  • This routine specifies $242a0 as the first sprite palette entry (highlighted in yellow and loaded to A1).   
  • We established each entry was 28 bytes previously. This means the table is $1be4 in length (28 bytes * 255 entries).

This gives us the final solution to our puzzle. We know where all the palette entries are first stored, before palette RAM is even accessed. We also know how many entries are in the table. 

Save the Palette table to a Binary file

We can now save the palette as follows using the MAME debugger:
save altbeast.pal, 0x242a0, 0x1be4, 0

This can be added to our sprite viewer XML configuration as follows:

We specify the number of bytes per entry, and the fact that the palette data is offset by 1 word when displayed by hardware. Essentially, the sprite hardware expects 32 bytes per entry, but the first and last words are not populated in the palette data.

Viewing the Palette

Finally, we can load the sprite data and view it in glorious colour by cycling through our 255 palettes. One limitation is that we still have no way of automatically connecting a palette to a sprite. This information is also contained in the 68k program code and is unique on a per-game basis.

I hope this has been informative! If you successfully create both a palette and configuration file, let me know and I'll add them to the source code repository. 


yt said...

Looks like I made a wrong assumption about Hang-On! Presumed it was similar to Space Harrier and didn't check.

Still, at least it works. If you find anything fun, post your results!

yt said...

Cool. I should try to rip a palette for Hang-On at some point. About to add one for Super Hang On that's been contributed.

Adrian said...

The sprite palette data for Hang-On is located starting at $ca4e in ROM and extends for 1920 bytes. The code which copies that data to sprite palette RAM is at $c918.

yt said...

Thanks Adrian. I've added the Hang On palette and xml on github:

cli said...

Any chance to add the Shinobi palette and xml? As far as I know, the main character sprite and hands sprite from the bonus stage had a different sleeves colour (red instead of blue) in the early versions as shown in these arcade flyers...
Somehow they forgot to change this red colour from the square magic icon in the bottom right and in the life icons. Even the drawing on the flyer features the red colour in the clothes, what could have happened?

yt said...

I've added a Shinobi palette and config file. No sign of the palette that colours the arms differently in the version of the game code I used mind you... (I used the 'shinobi' set in MAME, which is also referred to as Set 6)

The instructions I provided above for Altered Beast were near identical for Shinobi palette extraction.

cli said...

Yay! thanks a lot! Great work!

DH said...

Hey, this is a really great tool.

I was looking at the Altered Beast sprites and I noticed the stage 3 boss is nowhere to be found. Is it stored somewhere else or am I missing something obvious? Either way, thanks for your work on this, it's so good.

yt said...

@DH - Are you sure the Stage 3 boss isn't using tiles, as opposed to sprites? An easy way to check is to use the MAME tile viewer whilst the game is running.

DH said...

@yt I had the same thought and did check the gfx and tilemap viewer but didn't see it. Maybe I need to look again, it's entirely possible it's buried somewhere in the 8x8 gfx tiles. If you can look into it at all I'd appreciate it, but you've already helped me out so much just by releasing this tool. In any case, thanks again.

SL42 said...

That's cool, I was beginning to investigate about how to extract the graphics from Shinobi, then found your sprite viewer. I have to check if there is something significantly different about the tiles.

Sam32 said...

Hello! Is there any way this possibly support System 24 games? This would also be useful for Model 2 as it uses the system 24 tilemap chip as well for it's tilemaps

dlfrsilver said...

hello, i see that only the win64 is available, anyone can compile the source for win32 ? thanks :)

Antony said...

Do you happen to have any advice on how to locate sound effects, analogous to how you've managed to identify the offset and length of sprites?

In particular I'm interested in silencing or "zeroing" most of the sound effects in Space Harrier (I read somewhere that there are hidden configuration settings intended only for the cabinet's owner, but that sound effect volume isn't one of them).

While these sound effects provide useful feedback to whoever is playing, I feel they also detract from the viewing pleasure of anyone who might be watching. I'd personally rather enjoy the music than the sound of the main characters gun firing!

Which might be a little too overspecific/pedantic/opinionated, but I really would like to know how to go about locating sound effects, if possible, please :-)

yt said...

@Antony: Your best bet to remove sound effects would be to remove them either from where they are called in the 68k code. Or to edit the Z80 sound program itself. Or to remove the sample data from the PCM ROMs. I'd choose the former myself, maybe replace the sound playing function in the 68k code with a custom routine based on some settings somewhere so you can have effects on/off.

Antony said...

Sorry, that was actually one of your suggestions, wasn't it

> Or to remove the sample data from the PCM ROMs.

I'm afraid I'm new to most of this terminology and wasn't able to unpack what you'd said until I'd figured it out myself. Oops. Well I guess I guess I'll have to study MAME the source code some more.

Vic said...

Great post, very interesting. I'm reading these, in late 2023. Just wanted you to know that people still find your old blog posts interesting. I wish you would do deep breakdowns of other Sega arcade games, even if only limited to super scalers (Power Drift, Thunder Blade, Galaxy Force 2, etc). You could maybe even some System 16 regulars - E-Swat, Shinobi, Alex Kidd: The Lost Stars, etc. I would love to read about all the little secrets - hidden messages in the code, how you extract the sprite and palette data, so forth. Thanks very much for all you do!