The Accidental Emulator Part One

Recently, I accidentally started to make a Spectrum emulator.

The original intention was to disassemble some of my favourite Spectrum games to figure out how they’d been put together, and then perhaps even write my own game in assembly language. But what’s resulted so far is a partially functioning early-stages emulator, built in Unity, complete with basic debugging capabilities and display rendering. It’s been quite an interesting, challenging, and often diverted ride to get to this point so far, so I thought it’d be nice to share the learnings and source.

First, a little context is probably useful.

In the early 80s - probably around 1983-84 - my Dad was gifted, by his Dad I seem to recall, a ZX Spectrum, the affordable 48k rubber key home computer that took the UK by storm. I was in the early years of primary school, and was having limited access to the school BBC Micros at the time, and I was absolutely hooked. We bought the Software Projects tape version of Manic Miner to play with, plus the Horizons tape it shipped with, Hungry Horace, and rapidly added to that little collection with the help of the dodgy TDK-90 trading down the local cricket club. The games were great, but I was also enchanted with the built-in BASIC interpreter. I started writing “design documents” for games I never finished coding in half-understood BASIC, and I knew wholeheartedly, even at that age, that this was exactly what I wanted to spend my working life doing.

Fast-forward 35 years or so, a degree in Computer Science, a lot of mucking around and not paying enough attention in everything else, and recently passing 20 years as a professional video game developer, during which time I’ve been fortunate to have worked on some of the biggest games of their time (and many that weren’t that big, either!), and had the opportunity to work for and with studios both big and small, I finally felt like it was about time I went full circle, and understood what those early pioneers had to do to get a game running on my first beloved machine.
I now, supposedly, have the full armoury to glimpse with some clarity into the genius thought processes of a 17 year old (argh!) Matthew Smith. What he had to think about when working out how to put Manic Miner together. Maybe be able to walk some of the footsteps of heroes from my youth, like Mike Singleton, the Stampers, the Pickfords, and do it from completely the wrong angle, as a comparatively old man.

I mentioned that as a kid, I dabbled in BASIC from the Spectrum command-line.
BASIC is an interpreted language, which means that there’s a program running on the machine, from a ROM (we’ll come back to this shortly), which ‘reads’ the code you type into it and ‘interprets’ it into lower-level code - eventually becoming reams of sequential binary numbers, ‘machine code’, which the computer can understand.
BASIC is pseudo-english, so it’s much easier for folks to read and understand than assembly language, or machine code. I bet even if you’ve never written any code for a computer in your life, you could have a good guess that if you booted up a Spectrum, or indeed pretty much any home computer from the 80’s 8-bit era, and typed the following:

10 PRINT “HELLO WORLD!”
20 GOTO 10
> RUN

it will fill your computer’s display with “HELLO WORLD!” over and over again.

Back in the day, people did write games in BASIC. Some pretty good games were developed like that too, notably text adventures and text-heavy strategy games. But there’s the rub, an arcade game like Manic Miner, with smoothly-animated sprites (graphics) just isn’t going to run fast enough if developed in BASIC because of the processing overhead of the interpreter.

The lowest level language the computer understands is binary, but it’s probably a bit hardcore even for those 80’s pioneers to be expected to work everything out into sequential zeros and ones. Also, completely unnecessary, because the ZX Spectrum contains a Z80 CPU chip, and this means we can write Z80 assembly language for the machine, and an assembler can then turn our code into binary which the hardware, or an emulator, will understand.

First of all though, it’s worth visualising how this binary data that our computer/emulator ultimately needs to read is laid out.

The ZX Spectrum is an 8-bit machine. There are 8 bits (zeroes and ones, “on” or “off”) in a byte, and 1024 bytes in a kilobyte (or ‘K’ for short). Our trusty old rubber key Speccy has 48k of memory, or 49152 bytes, or 393216 bits. This memory is laid out sequentially, but ‘bookmarked’ at fixed points for various purposes, and much of which we can safely poke with binary values which are understood as machine code for the computer to process.
The 48k of memory that the Spectrum advertises as available though, actually starts - in terms of its place in the sequential memory map - not at ‘byte zero’, but after an initial 16k of stuff. That’s our first ‘bookmark’, then. So really, the 48k Spectrum has a 64k memory map, and that first 16k of stuff is taken up by the ROM (Read-only memory) we mentioned earlier. This read-only memory already has a program on it, and that program is the thing that displays the following screen when you boot it up, and holds and executes a BASIC interpreter for you to play with.

Even though we talk about the Spectrum being 8-bit, because our memory address space goes up to 65536 bytes (64k memory map, remember? So (1k = 1024 bytes) X 64), we’re obviously going to need to work with numbers larger than 255, which is the largest value we can represent in a byte. Otherwise, how would we read or write the memory address value 65535, for example? To achieve this, we can put two sequential bytes together to make a ‘Word’ - a 16-bit number, and that lets us represent numbers between zero and 65535 inclusive. Because of how we read binary, most significant bit to least, left to right, it’s easier to think of the high-byte being the first byte (so the byte representing the higher numbers), and the next byte being the low byte (our 0 to 255 part). This is known as a big-endian system. If the two bytes were stored the other way round sequentially in memory, it’d be little-endian, and just for a bit of extra exciting confusion that’s exactly how 16-bit numbers are stored on the Spectrum. Developers of cross-platform videogames love talking about system endian-ness. Just ask one, they love it.

Anyway, the Spectrum, and its Z80 chip, will do lots of its magic without much effort with 16-bit values, so that’s good to know.

Where to even start

So, I have a copy of the TAP file (a representation of the audio cassette tape data) of one of my all-time favourite games for the Spectrum - The Lords of Midnight.
https://worldofspectrum.org/archive/software/games/the-lords-of-midnight-beyond-software

But where do I begin, really? I want to dissect this game, even try and somehow extract and trace the original Z80 source code, game logic and graphic data from it. But I’m still thinking that I need to learn a little bit more about the Spectrum, first. And this is where Epic Diversion Number 1 happened.
I stumbled across this fabulous book:
https://worldofspectrum.org/archive/books/complete-spectrum-rom-disassembly-the
in which two wonderful chaps in 1983, listed and documented the entire 16k ROM program, and published it in a paperback.

This informs me that when I power up the Spectrum, and knowing that the ‘Program Counter’ (a register holding the value of the memory address that the computer is currently reading), is zero, it all begins with the Z80 instruction DI. Disable the ‘keyboard interrupt’, apparently.

I get out my trusty old “Programming the Z80” book by Rodnay Zaks, https://en.wikipedia.org/wiki/Programming_the_Z80, and discover that DI is a single byte instruction, which converts to F3 in machine code. That means that byte zero in my memory map is also F3 in Hexadecimal (which we’ll continue to use as our numeric base for clarity, so 243 in decimal, and 11110011 in binary).
When that byte is parsed by the powered-up computer or emulator, ‘keyboard interrupt’ is disabled, the program counter is incremented by 1 byte (the total size of this whole instruction), and we’re now looking at address 1, which my ROM disassembly documentation tells me is XOR A, 1 byte again, and AF in machine code.
I can do something with this! We’re off! On a complete tangent! To emulate the ROM and “boot up a Spectrum”!

First thing we need to do is to emulate the memory. We’ve established that the entire memory map can be represented as a sequential 64k buffer, so let’s just do that:

public class MemoryMap
{
   public static int Size = (1024 * 64);

   public MemoryMap()
   {
      Values = new byte[Size];
   }

   public byte ReadValue(ushort address)
   {
      return Values[address];
   }

   public void WriteValue(ushort address, byte value)
   {
      Values[address] = value;
   }

   private byte[] Values { get; set; }
}

We can start to fill this buffer with the initial 16k of bytes (or at least, the first few to get started with!) presented to us by our ROM disassembly book. So F3, then AF, etc. Eventually I’ll find myself a binary copy of the 16k Spectrum ROM that I can just file read in, and populate this buffer with automatically, to save my fingers from having to type 16384 hex codes.

The other thing we’re going to need to emulate before we can start implementing instruction logic, are the registers. The Spectrum has quite a few of these. Many of them are of course 8-bit values, like the Accumulator, used in mathematical CPU operations, but some - for example, the program counter (PC), storing the currently executing memory address, are 16-bit.
The Spectrum also utilises some of the 8-bit registers as register pairs for the purposes of some instructions. For example, the B and C registers individually hold bytes, but BC can also be treated as a 16-bit value. In our emulation of these, we’ll hold B and C as separate bytes, but have a property which combines them together to return a 16-bit value.
It’s also worth noting that the Spectrum has a bank of ‘alternate’ registers, allowing fast snapshots of current register state to be buffered and restored. We’ll call these A_, B_, etc.
Finally, the ‘Flags’ register is an interesting one, in that each bit has a specific purpose (on or off state), rather than being regarded as a complete value in its own right. We’ll supply a set of properties to query this single byte register, for readability and convenience.

public class Registers
{  
   public ushort PC { get; set; } // Program Counter
   public ushort SP { get; set; } // Stack Pointer
    
   public ushort IX { get; set; }
   public ushort IY { get; set; }
    
   public byte A { get; set; } // Accumulator

   public byte B { get; set; }
   public byte C { get; set; }

   public ushort BC { get { return (ushort)((B << 8) + C); } }

   public byte D { get; set; }
   public byte E { get; set; }

   public ushort DE { get { return (ushort)((D << 8) + E); } }

   public byte H { get; set; }
   public byte L { get; set; }

   public ushort HL { get { return (ushort)((H << 8) + L); } }

   public ushort IR { get; set; }

   public byte Flags { get; set; }
    
   // Alternate registers
   public byte A_ { get; set; }
   public byte F_ { get; set; }
   public byte B_ { get; set; }
   public byte C_ { get; set; }
   public byte D_ { get; set; }
   public byte E_ { get; set; }
   public byte H_ { get; set; }
   public byte L_ { get; set; }

   public bool IsFlagCarry()
   {
      return GetFlag(0x01);
   }

   public void SetFlagCarry(bool on)
   {
      SetFlag(0x01, on);
   }
    
   public void SetFlagSubtract(bool on)
   {
      SetFlag(0x02, on);
   }

   public bool IsFlagParity()
   {
      return GetFlag(0x04);
   }

   public void SetFlagParityOverflow(bool on)
   {
      SetFlag(0x04, on);
   }

   public void UpdateFlagOverflow(byte before, byte after)
   {
      if ( (before == 0) || (after == 0) )
      {
         SetFlag(0x04, false);
         return;
      }

      int signBefore = before >> 7;
      int signAfter = after >> 7;
      SetFlag(0x04, signBefore != signAfter);
   }

   public void UpdateFlagParity(byte test)
   {
      int onesCount = 0;
      for (int iByte = 0; iByte < 7; ++iByte)
      {
         if ((test & (1 << iByte)) != 0)
         {
            ++onesCount;
         }
      }

      SetFlag(0x04, (onesCount % 2) == 0);
   }

   public bool IsFlagHalfCarry()
   {
      return GetFlag(0x10);
   }

   public void SetFlagHalfCarry(bool on)
   {
      SetFlag(0x10, on);
   }

   public void UpdateFlagHalfCarry(byte before, byte after)
   {
      bool bit4Before = (before & 0x10) != 0;
      bool bit4After = (after & 0x10) != 0;

      bool bit3Before = (before & 0x0F) != 0;
      bool bit3After = (after & 0x0F) != 0;
        
      if (bit4Before && !bit4After && bit3After)
      {
         SetFlag(0x10, true);
      }
      else if (bit3Before && !bit3After && bit4After)
      {
         SetFlag(0x10, true);
      }
      else if (!bit4Before && bit4After)
      {
         SetFlag(0x10, true);
      }
      else
      {
         SetFlag(0x10, false);
      }
   }

   public bool IsFlagZero()
   {
      return GetFlag(0x40);
   }

   public void SetFlagZero(bool on)
   {
      SetFlag(0x40, on);
   }

   public void UpdateFlagZero(byte test)
   {
      SetFlag(0x40, test == 0);
   }
    
   public bool IsFlagSign()
   {
      return GetFlag(0x80);
   }

   public void UpdateFlagSign(byte test)
   {
      SetFlag(0x80, (test & 0x80) != 0);
   }

   private bool GetFlag(byte value)
   {
      return (Flags & value) != 0;
   }

   private void SetFlag(byte value, bool on)
   {
      if (on)
      {
         Flags |= value;
      }
      else
      {
         int flags = Flags;
         flags &= ~value;
         Flags = (byte)flags;
      }
   }
}

Now the crucial bit for this initial stage of development, I need to think about how I’m going to represent an instruction in Unity, in C#. Something like this:

public interface Instruction
{
    /// <summary>
    /// The human readable name, for displaying the assembly source.
    /// </summary>
    /// <param name="memory">Current memory map buffer state</param>
    /// <param name="offset">Starting address to read from</param>
    /// <returns>
    /// Complete instruction including any variants or parameters, or
    /// empty string if the value(s) from the given address do not match this instruction.
    /// </returns>
    string GetName(MemoryMap memory, ushort offset);

    /// <summary>
    /// Returns the total bytes comprising the instruction,
    /// the amount of bytes to progress the program counter after executing an instruction
    /// which doesn’t jump.
    /// </summary>
    /// <returns>
    /// Instruction size in bytes
    /// </returns>
    ushort GetBytesCount();
    
    /// <summary>
    /// Given the memory map, and offset (the address), is this our instruction?
    /// We can use this to ‘interpret’ the assembly.
    /// </summary>
    /// <param name="memory">Current memory map buffer state</param>
    /// <param name="offset">Starting address to read from</param>
    /// <returns>
    /// true if this instruction is a match for the given memory values, false if not.
    /// </returns>
    bool Parse(MemoryMap memory, ushort offset);

    /// <summary>
    /// Run this instruction.
    /// Supplied memory and register state may be modified by its purpose and effects.
    /// </summary>
    /// <param name="memory">Current memory map buffer state</param>
    /// <param name="registers">Current register state</param>
    /// <returns>
    /// The new memory address that the program counter should now point at
    /// </returns>
    ushort Execute(MemoryMap memory, Registers registers);
}

So now that we’ve got that simple interface template defined, let’s take a look at what the first instruction DI might look like, given that I don’t have keyboards or interrupts yet in my Unity emulator project, so executing is currently redundant:

public class DI : Instruction
{
    public string GetName(MemoryMap memory, ushort offset)
    {
        return IsDI(memory, offset) ? "DI" : string.Empty;
    }
    
    public ushort GetBytesCount()
    {
        return 1;
    }

    public bool Parse(MemoryMap memory, ushort offset)
    {
        return IsDI(memory, offset);
    }
    
    public ushort Execute(MemoryMap memory, Registers registers)
    {
        UnityEngine.Debug.LogWarning("DI instruction not implemented!");
        return (ushort)(registers.PC + GetBytesCount());
    }

    private bool IsDI(MemoryMap memory, ushort offset)
    {
        var value = memory.ReadValue(offset);
        return value == 0xF3;
    }
}

Even though this instruction isn’t actually performing any effect, we can see the interface at work. GetName returns the human-readable instruction only if the memory address supplied is F3, and Parse only returns true on the same check, so this provides the foundation for our interpreter logic. Execute currently has no effect other than warning that’s the case, but it does return the new address that the program counter should now be pointing to, after execution. We could, for sanity, validate that this is a DI instruction at the top of the Execute method, and throw an exception or something, but we’ll assume our interpreter logic is going to call Parse first.

The second instruction that our ROM needs to execute, XOR A, introduces a few new complexities, plus it can actually do something in execution with what we’ve set up so far, so we’ll take a closer look at this. The purpose of the instruction is to perform an exclusive OR operation with the value in the accumulator and ‘something else’, and write the result back into the accumulator. So XOR A is an interesting one, because it’s always going to write zero back into the accumulator (because the bits in A are always going to match the bits in A!), so it’s just a fast way of clearing it.

// 1-byte variant
public class XORs : Instruction
{
    public string GetName(MemoryMap memory, ushort offset)
    {
        var value = memory.ReadValue(offset);
        if (IsXORs(value))
        {
            return $"XOR {GetRegisterLabel(value)}";
        }
        else if (IsXORHL(value))
        {
            return "XOR (HL)";
        }
        else
        {
            return string.Empty;
        }        
    }
    
    public ushort GetBytesCount()
    {
        return 1;
    }

    public bool Parse(MemoryMap memory, ushort offset)
    {
        var value = memory.ReadValue(offset);
        return IsXORs(value) || IsXORHL(value);
    }
    
    public ushort Execute(MemoryMap memory, Registers registers)
    {
        byte aBefore = registers.A;
    
        var instruction = memory.ReadValue(registers.PC);
        if (IsXORHL(instruction))
        {
            var addressedValue = memory.ReadValue(registers.HL);
            registers.A ^= addressedValue;
        }
        else
        {
            var label = GetRegisterLabel(instruction);
            switch (label)
            {
            case "A":
                registers.A ^= registers.A;
                break;
            case "B":
                registers.A ^= registers.B;
                break;
            case "C":
                registers.A ^= registers.C;
                break;
            case "D":
                registers.A ^= registers.D;
                break;
            case "E":
                registers.A ^= registers.E;
                break;
            case "H":
                registers.A ^= registers.H;
                break;
            case "L":
                registers.A ^= registers.L;
                break;
            default:
                throw new System.Exception($"XOR s : Unhandled register label '{label}'");
            }
        }

        registers.SetFlagCarry(false);
        registers.SetFlagSubtract(false);
        registers.UpdateFlagParity(registers.A);
        registers.UpdateFlagHalfCarry(aBefore, registers.A);
        registers.UpdateFlagZero(registers.A);
        registers.UpdateFlagSign(registers.A);

        return (ushort)(registers.PC + GetBytesCount());
    }
    
    private static bool IsXORs(byte value)
    {
        var instructionPart = value & ~7;
        return (instructionPart == 168) && !string.IsNullOrEmpty(GetRegisterLabel(value));
    }
    
    private static bool IsXORHL(byte value)
    {
        return value == 0xAE;
    }
    
    private static string GetRegisterLabel(byte value)
    {
        byte registerPart = (byte)(value & ~248);
        return Registers.GetLabel(registerPart);
    }
}

The first thing you’ll notice, from the GetName method, is that we’re not just handling XOR A within this instruction implementation, we’re handling all XOR 1-byte variant instructions - i.e. XOR B, XOR C, XOR D, XOR E, XOR H, XOR L and XOR (HL).

We can identify one of the register value XOR instructions by the fact that the instruction value, with the first three bits masked out, will be A8 (168 decimal), which you can see being checked for in the private method IsXORs. We can then check those bottom three bits to see if they equal a value which represents a register - and this register value id check is a standard used across lots of instructions, so we can factor this into a static method in our Registers class, calling it GetLabel.

public static string GetLabel(byte value)
{
    foreach (var definition in sDefinitions)
    {
        if (definition.Value == value)
        {
            return definition.Label;
        }
    }

    return string.Empty;
}

private static List<Register> sDefinitions = new List<Register>()
{
    new Register("A", 7),
    new Register("B", 0),
    new Register("C", 1),
    new Register("D", 2),
    new Register("E", 3),
    new Register("H", 4),
    new Register("L", 5)
};

private class Register
{
    public string Label { get; private set; }
    public byte Value { get; private set; }
    
    public Register(string label, byte value)
    {
        Label = label;
        Value = value;
    }
}

XOR (HL) is an instruction which performs an exclusive OR operation on and with the value of the accumulator with the value pointed at by address HL, rather than the contents of one of the registers. The instruction has a value of AE, so that’s an easy thing to check for separately.

Finally, you’ll notice that we modify some of the flags register bits at the end of the execution method, depending on the result of the operation.

Let’s finish this round of work, and tie together what we’ve built so far.

We need somewhere to hold this library of instructions we’re going to start to build, so we’ll create a static Interpreter class, with the calls to the parsing and assembling methods, and execution of the correct instructions to interpret from the program counter memory address:

public static class Interpreter
{
    /// <summary>
    /// Disassemble N bytes-worth of instructions and buffer their human-readable names
    /// </summary>
    /// <param name="memory">Current memory map buffer state</param>
    /// <param name="pc">Starting address to read from</param>
    /// <param name="instructions">
    ///     The string buffer to store the disassembly back into.
    ///     The size of this buffer is used implicitly as the maximum disassembly size.
    ///     Each element corresponds by index to the starting memory address,
    ///     and instructions held in the first byte/index where they exist.
    ///     E.g. if the string array contains 8 elements,
    ///     then 8 1-byte instructions would be disassembled into each element by line,
    ///     or 4 2-byte instructions, where array elements 1, 3, 5 and 7 would be empty, etc.
    /// </param>
    public static void Disassemble(MemoryMap memory, ushort pc, string[] instructions)
    {
        int disassemblySize = instructions.Length;
        for (ushort iInstruction = pc; iInstruction < (pc + disassemblySize); ++iInstruction)
        {
            string instruction;
            ushort instructionSize = Parse(memory, iInstruction, out instruction);
            instructions[iInstruction - pc] = instruction;

            if (instructionSize > 1)
            {
                // Increment the memory read point by the additional instruction size over 1 byte
                iInstruction += (ushort)(instructionSize - 1);
            }
            else if (instructionSize == 0)
            {
                UnityEngine.Debug.LogWarning($"Syntax error at address {iInstruction:X4}");
                break;
            }
            // else instructionSize==1 which means memory read can be left to increment by 1
        }
    }

    private static readonly List<Instruction> sInstructions = new List<Instruction>()
    {
        new DI(),
        new XORs()
    };

    private static ushort Parse(MemoryMap memory, ushort offset, out string name)
    {
        var instruction = GetInstruction(memory, offset);
        if (instruction == null)
        {
            name = string.Empty;
            return 0;
        }
        else
        {
            name = instruction.GetName(memory, offset);
            return instruction.GetBytesCount();   
        }
    }

    private static Instruction GetInstruction(MemoryMap memory, ushort offset)
    {
        foreach (var instruction in sInstructions)
        {
            if (instruction.Parse(memory, offset))
            {
                return instruction;
            }
        }

        return null;
    }
}

Finally, for part one of our ROM emulation, we’ll make a Debugger MonoBehaviour, which will hold our entire ‘CPU processing loop’:

public class Debugger : MonoBehaviour
{
    private void Start()
    {
        mMemoryMap = new MemoryMap();
        SetupRegisters();

        StartCoroutine(ProcessCPU());
    }

    private IEnumerator ProcessCPU()
    {
        while (true)
        {
            var pc = mRegisters.PC;

            // Look ahead at the next N bytes worth of code/data
            const ushort kAddressDisplaySize = 8;
            var addressStart = pc;
            var addressEnd = addressStart + kAddressDisplaySize;

            var instructionNames = new string[kAddressDisplaySize];
            Interpreter.Assemble(mMemoryMap, addressStart, instructionNames);

            var debugText = string.Format("PC   Memory address   Value   Instruction\n");
            for (ushort address = addressStart; address < addressEnd; ++address)
            {
                debugText += string.Format(
                                    "{0}        {1:X4}          {2:X2}      {3}\n",
                                    address == pc ? "=>" : "  ",
                                    address,
                                    mMemoryMap.ReadValue(address),
                                    instructionNames[address - addressStart]);
            }
            Debug.Log(debugText);

            yield return null;
        }
    }
}

The code above can now disassemble the first 8 bytes of memory, and once the appropriate instructions are setup and instantiated, this is the output we should generate:

This is all quite exciting, we can now start to emulate the logical workings of the Spectrum ROM booting up. But it’s not going to get properly interesting until we can actually see what the Spectrum is trying to draw on the screen. We’ll deal with that madness in the next instalment!