Thursday, November 28, 2019

OutRun's Music Macro Language & Tooling

My recent post, which covered Sega's audio creation process in the 1980s, inspired fellow coder cmonkey to create a nifty tool to facilitate composing new music for OutRun.

The music in OutRun is, at the most basic level, a simple stream of MIDI note values and durations. The tunes were originally transcribed from sheet music into MML (Macro Music Language). Presumably, after transcription, Sega had tooling to compile the MML data into the format used by the game's audio engine. Cmonkey's tool, SiMMpLified, essentially recreates this tooling.

The format used by OutRun is compact and efficient. Tunes are comprised of up-to 8 FM tracks, played by the Yamaha YM2151 sound chip and up-to 5 PCM tracks, played by the custom Sega 315-5218 PCM sample playback chip. Technically, it's possible to have a tune that's purely 16 channels of PCM samples, but practically sample space is limited. In addition, some PCM channels are reserved for the game's sound effects including engine revs, tire screeches and voice samples. In some respects, the format can be considered a more advanced version of the MOD format, popularised by the Amiga home computers.

The power of Sega's MML format is a result of a relatively advanced looping and nested sub-routine structure. This enables the data for a tune like Passing Breeze to be squeezed into under 5K of EPROM space, excluding the percussion samples! If you're interested in the technical structure and command format of MML, it's worth reading the SiMMpLified documentation.

The SiMMpLified tooling inspired me to consider what could be achieved with OutRun's audio in the forthcoming Enhanced Edition, currently slated for release in 2020. Ideally, I'd love to include additional music, as per AfterBurner: Enhanced Edition. There are already candidates for the tracks. M2's 3DS version of OutRun introduced two new tunes, Camino and Crusing Line. The later Switch version introduced Step on Beat, which first premiered on the Megadrive and Hiro's Radiation. The rest of the music in the Switch version is streamed audio and doesn't use MML format.

However, whilst extracting the new music reveals it to be in the correct format, it's bulky in terms of file-size. Each new song is as large as all four of the original OutRun compositions combined! There was an impressive project to get the music running on original hardware, but it required a custom manufactured PCB. This allowed the Z80 audio processor to access the additional address space.

I'd like to achieve this without the need for custom hardware. The existing OutRun audio EPROM is 32K. The maximum size supported by the PCB is 64K, with a jumper swap. However, part of the 64K memory map is mapped to RAM. Therefore, there's only 60K of accessible EPROM space. Any new music would need to fit into ~28K of free space.

At this point I realised the new music would require analysis and optimization. Reversing the music by hand would be error-prone and time consuming. Instead, I coded a tool that would read a binary and output an MML file, compatible with the SiMMpLified tooling. The end result is that we can instantly produce a formatted MML file containing the notes and commands for any OutRun tune. This file can be recompiled with a Z80 assembler and inserted back into the audio EPROM.

For the sake of completeness, you can think of the flow through these tools as follows, although clearly this example is an extremity:

OutRun Z80 EPROM -> MML Reassembler -> ASM file + siMMpLified Libs -> Z80 Assembler -> MML Injector -> OutRun Z80 EPROM

The MML Reassembler suggests that the original music is hand-crafted, whereas the new tracks appear to have been produced using some kind of tooling developed by M2. M2 did not need to be concerned about file size, and as such the output is bulky and contains repeated data.

I started work hand-optimizing Camino last night and shaved 2.5K off the filesize within a few hours (or closer to 9K if you take into account the unused data which my MML Reassembler automatically strips out). I don't know whether this project will be possible, but it's certainly worth a serious attempt!

Why not write a tool to optimize the tracks automatically? At this stage, I don't think it would achieve the results of a manual analysis where every single byte counts. Why not ZLIB compress the tunes in EPROM? The Z80 only has access to 2K of Work RAM, much of which is already in use by the program code.

Bear in mind that this is a work in process, and I'm unsure whether this will be of use to anyone right now. Nevertheless, the tooling can be found here.


Unknown said...

Excellent work!

matt said...

Great to see you still going with all this!

tred said...

Playing outrun to an instrumental version of a-ha’s Take on Me would be legendary...

xga said...

Amazing work as always, Chris!

Looking forward to seeing what else you have in store for the 2020 enhanced release. :-)