MartyPC: PC emulation done right

I would like to give a heads-up about the new PC emulator on the block. It is called “MartyPC”, which indeed is a reference to Marty McFly from the Back To The Future movies. And indeed, that is a tribute to our 8088 MPH PC demo, which had various references to these movies.

And yes, that means that this PC emulator intends to be cycle-exact, in order to run 8088 MPH. So here is a video demonstrating it running Area 5150:

Because, long story short: The emulator started development before Area 5150 was released, but as it turns out, Area 5150 was easier to render correctly, as it doesn’t require emulation of the NTSC composite video signal. But 8088 MPH does run in the emulator as well, as developer GloriousCow assures us. He just wants to put a bit more work into the video signal to get more accurate colours, before releasing a video capture of that demo.

Speaking of video, note that MartyPC correctly renders the ENTIRE display area, including overscan, which means we get proper borders and overscanned effects, as used in Area 5150.

So wait, didn’t we already have an emulator that runs 8088 MPH? No, I said we had an emulator that *doesn’t break* when running 8088 MPH. It was close, but it was not cycle-exact, so it didn’t look exactly the same as on real hardware. It would speed up or slow down in various parts.

Not this emulator. MartyPC really *is* cycle-exact. And it is open source, available on Github. And as you can see, it is written in Rust. A somewhat unusual choice for emulators, as they are mostly written in C or C++. But Rust is an interesting and very new and modern language, and it is very suitable for this sort of low-level work.

What’s also cool is that GloriousCow is documenting his work on his blog. He has only just started blogging, so at the time of writing there are only two technical articles (one on DMA, and one on CGA waitstates). But they are very detailed, and very readable as well, with diagrams, oscilloscope signal views and whatnot, so this looks very promising indeed!

All in all, this is finally a PC emulator that TRULY tries to do exactly what the hardware does. Which has been a pet peeve of mine for years, given that emulators for most other platforms have been designed this way.

Advertisement
Posted in Oldskool/retro programming, Software development, Software news | Tagged , , , , , , , , , , , , , , , | Leave a comment

Cartridges for the IBM PC

Cartridges? For the IBM PC? There’s no such thing!

Well, there kinda is… The IBM PCjr has two cartridge slots. Now, one may argue about whether the PCjr is actually an IBM PC compatible or not, but let’s look at how the PCjr implements this first.

For that, we can check the IBM PCjr Technical Reference, page 2-107 onward. Apparently a cartridge merely contains a ROM, which needs to have a certain header in order to be detected. It starts with the bytes 55h, AAh, and then a length-field (in blocks of 512 bytes). Then 3 bytes where you can store a jump to your initialization routine entry-point. The byte after that, at offset 6, will indicate what type of cartridge it is.

Three types of cartridge are supported:

  1. “Initial Program Loadable” (IPL): The cartridge will register an INT 18h handler for itself at initialization. INT 18h will be called after the BIOS POST is done, and no bootable device was found. This is the mechanism by which BASIC is normally started on an IBM PC or PCjr. BASIC is then the ‘initial program’ that is loaded. When the type-byte at offset 6 is set to 0, the cartridge is expected to be an IPL-type.
  2. DOS Command Extensions: From offset 6 onwards, a list of commands is stored: first a length-byte indicating the length of the command. Then the ASCII-string of the command itself, followed by a (word) offset to the code that implements the command, according to the manual. However, that seems to be an error. Instead, DOS will jump directly to the position directly after the command-string. You have 3 bytes to put a jump to the actual code, similar to the main initialization entry-point (DOS creates a process for you, so you can just use ah=4Ch to exit back to DOS from your command). The list of commands is terminated by an entry with a length of 0.
  3. BASIC cartridges: Instead of the 3-byte jump at offset 3, you have the sequence: CBh, 55h, AAh. The type-byte at offset 6 will be set to 0. Then at offset 7, there will be a byte that is either FEh for a protected BASIC program, or FFh for unprotected.

After this initial header, the code (either x86 machine code or BASIC) will be stored, and at the end of the image, there will be two bytes that contain a 16-bit CRC code. We will get back to that later.

But first… did you see what they did there? With the 3 types? They tricked us! Because if you consider a DOS command image with no commands, then technically it is the same as an IPL-type. What you do in the initialization code is pure semantics. And the CBh byte for the BASIC cartridge is actually a RETF instruction. So that is technically an empty initialization function. Apparently the BIOS can still call into it, it will just return immediately. The 55h and AAh bytes are then additional bytes that BASIC can check for at a later point. Likewise it seems that the DOS commands are actually scanned by DOS at a later part in the boot sequence, not during BIOS initialization. So what this appears to boil down to is that the BIOS merely checks for valid ROMs by scanning for the 55h AAh marker, then checking the length and verifying the CRC value, and if that checks out, it will do a far-call into offset 3 to call the initialization routine of the ROM.

But we can trick them as well, can’t we? Yes, it would seem that although the jump at offset 3 should technically be jumping into an initialization function, and then RETF back to the BIOS initialization, in practice various cartridges actually jump directly into the program code, and just never return to the BIOS at all. This fundamentally changes how the cartridge works: If you just use the initialization routine to install an INT 18h handler, then the cartridge code will only be executed when no bootable device is found (so only when you have not inserted a bootable floppy, and you have no other boot device, such as a HDD installed). If you start your application code directly from the init function, then it will execute regardless of whether other boot devices are available.

If you compare this cartridge system with the BIOS extension system of the regular IBM PC and compatibles, you’ll find that it works in basically the same way. You can find it in the IBM PC Technical Reference, page 5-13 onward. That is, the PC BIOS does not appear to support DOS command extensions or BASIC extensions via ROM, but an IPL-type ROM for PCjr is very similar to a PC BIOS extension. The initial 55h, AAh bytes and the length-value are stored in the same way. And in both cases the BIOS will also do a far-call to offset 3. The PCjr simply extended this mechanism by jumping over the optional DOS commands that can be stored, starting at offset 6.

Also, where the PCjr uses a CRC16 to verify the ROM, the PC BIOS uses a simple checksum (just adding all bytes together, modulo 100h). But the idea is the same: after processing all bytes in the ROM, the CRC16 or checksum should be 0, so you should add a value at the end of your ROM to make sure it gets to 0. That will make it somewhat difficult to make a single ROM that works on both types of machines (it would need to pass both the CRC16 and the checksum-test), but you can trivially make both a PC and a PCjr version out of the same codebase, by only adjusting the CRC/checksum-value for the specific target before writing it to a ROM.

The address range of the ROMs is also different: the PC BIOS extensions use the segment range C000h-F400h, where the PCjr cartridge uses segment range D000h-F000h (allowing to add up to 128k of ROM space via two cartridges). In both cases, the ROM area is scanned at an interval of 2k blocks. So a new ROM image (indicated by the 55h AAh marker) can occur at every 2k boundary. The PCjr simply treats ROMs at D000h and above as cartridge ROMs (and as such expects CRC16), where the rest are assumed to be standard BIOS extensions (with a checksum).

So in theory you can create a ROM containing an application for a regular IBM PC as well, and stick it onto an ISA card, as a makeshift ‘cartridge’. In practice it’s just less convenient as you cannot easily insert and remove it. That might explain why the support for DOS command or BASIC expansion ROMs was never added to regular PCs, and as such, remains a PCjr-exclusive.

However, the cartridge port on the PCjr is basically just a cut-down ISA slot. In theory you could create an ISA card with a PCjr-like cartridge slot in the bracket at the back, so that you can insert cartridges with ROMs that way. And once you can easily insert and remove ROMs like that, it makes sense to create ROMs that contain games or other software.

Why cartridges/ROMs?

Is there a point to using cartridges/ROMs, you may ask? Can’t you just load your software from a floppy or harddisk? Well yes, there is. Roughly there are two advantages to storing software in ROM:

  1. The ROM is mapped into a separate part of the address space, so your code and data can be accessed directly from the ROM, leaving your RAM free for other things.
  2. Unlike the dynamic RAM in your machine, the ROM does not need to be refreshed. As such, when reading from ROM, you do not incur any waitstates. This makes ROM slightly faster than RAM.

For a regular PC this is not such a big issue, as they generally have lots of RAM anyway, and a regular PC refreshes memory only once every 18 IO cycles (roughly every 15 microseconds).

For a PCjr however, things are different. A standard PCjr has only 64k or 128k of memory, and this is shared with the video circuit. So it reserves 16k to 32k of video memory from this pool. The 64k models were sold without a floppy drive, as DOS would not be much in the remaining 48k of memory. Even 128k is still quite cramped for DOS. So if you could store your application in ROM, you can make more of the system RAM.

And PCjr shares its main memory between the CPU and the video chip. This means you don’t get a memory refresh cycle once every 18 IO cycles. Instead, the video circuit will access memory once every 4 IO cycles, generating a wait state that blocks the CPU from accessing memory. Effectively this makes the CPU run code much slower from system RAM than from ROM or expansion RAM. Putting code and data in a ROM cartridge will fix this performance disadvantage of the PCjr, making it run as fast as a regular PC (or actually slightly faster).

Lotus 1-2-3 was released on two cartridges for the PCjr at the time, as the application itself was already 128k in size. It was still a DOS application, the cartridge made use of the DOS command extension mechanism to register ‘123’ as a command, so it would start from the cartridge when you entered ‘123’ at the DOS prompt. That was a clever way to make the most of the PCjr. It ran from DOS, so you would have access to floppy storage without any issues. It saved precious system memory by storing the application in ROM. And the ROM also made the code run faster than if it were a regular DOS application running from RAM.

So let’s see what it takes to make our own cartridge.

The CRC check

Let’s look closer at how the CRC check works. We are going to need it in order to pass the validation check in the BIOS, so it will actually run our code from the ROM.

On page 2-109, the Technical Reference points to the ‘crc_check’ routine in the BIOS listing, to see how this works. We can find this CRC routine on page A-106:

So the CRC check basically scans the entire ROM image (including all headers), and the check passes if the resulting CRC value is 0.

On page 2-108, the manual says that you should put ‘CRC Bytes’ at the Last 2 Addresses’ of your ROM image. What they mean to say is that the CRC should yield 0. But the CRC is just a polynomial calculated over the ROM contents, so it can be anything, depending on the contents. In order to get it to 0, you need to add extra ‘dummy bytes’ to get the polynomial to reach 0 at the end. And since this is a 16-bit CRC, two bytes are enough to compensate.

But if you study the code, you see that it simply calculates the CRC over the entire area, and doesn’t specifically treat the last two bytes as something special. This means that theoretically you can place them anywhere in your image. In fact, in the rare case that the CRC already reaches 0 without adding extra bytes, you can just leave them out altogether.

However, in practice, it is easiest to place them at the end. Namely, how do you generate these two bytes? Well, one way is to just bruteforce all options (there are only 65536 possibilities), until you find one. But if you have to calculate the CRC over the entire ROM every time, this will be relatively inefficient. CRC is an incremental algorithm however. That is, you start out with an initial value (which by default is FFFFh as you can see in the code), and then add on to that.

So if you want to bruteforce the last two bytes of a ROM image of N bytes, you simply calculate the CRC up to N-2 bytes first. Then you use this CRC as the initial value, and calculate a CRC for the last two bytes. So you only have to calculate up to 65536 CRCs of 2 bytes in order to find the right combination. Even an 8088 at 4.77 MHz can do that almost instantly.

Writing the code

It’s somewhat tricky to write actual ROM code, because the usual programming environments are aimed at DOS, and as such want to generate either a .COM or a .EXE file. Generating a valid ROM image is not directly possible with most programming environments. A .COM file is pretty close to a raw binary image though: it is a single segment of up to 64k in size, containing both code and data. Obviously you have to avoid any dependencies on DOS in the .COM file, so you probably need to disable any runtime libraries. In which case the main difference with a raw binary is that a .COM file starts at offset 100h.

This is another DOS-dependency, as DOS creates a Program Segment Prefix (PSP) block for every process that it starts. This block is 256 bytes long, and the .COM file is loaded directly after it, in the same segment. DOS will then call the code at offset 100h as your entry point.

In a cartridge we do not have a PSP, but we do have a header, with a jump into the actual code. This header is much smaller than 256 bytes. So we could create a cartridge header with a jump to 100h, then pad up to offset 100h, and then append the .COM file there.

However, I chose to use MASM, and in that case you specifically have to place a ‘org 100h’ directive to make the code start at offset 100h. If you omit that directive, it will start the code (or data) at offset 0. This means I can actually place the header inside my assembly code, and place my code and data directly after the cartridge header. I created a template like so:

.Model Tiny

.code
; Cartridge header
marker dw 0AA55h
len db 0
jmp InitROM
; List of DOS commands
org 6	
db 0

; Start of ROM data

InitROM PROC
...
retf
InitROM ENDP

...

END

The generated .COM file will then start with the header, and the code will correctly assume that the image is loaded at offset 0 in the segment, not offset 100h. There are just three problems left:

  1. The length-field is left at 0, because it should be specified in blocks of 512 bytes. While it is possible to do simple addition and subtraction with labels in MASM to get the size of a block of code or data, it is not possible to divide the size by 512. So we cannot let the assembler generate the correct length in the image automatically.
  2. The image size needs to be aligned to blocks of 512 bytes. The assembler can only do alignment up to 256 bytes (the size of a ‘page’).
  3. The image needs to generate a CRC value of 0, which as mentioned above, will require adding additional bytes to adjust the CRC.

I have created a simple tool for this, based on the actual BIOS code. It checks if the length-field matches with the actual size of the binary. If there is a match, it assumes the image is already prepared for a cartridge, and it will calculate the CRC for the image, and report whether it is 0 or not.

If the size is a mismatch, it will assume the image needs to be prepared. It will first add two bytes to the size, to make room for the CRC adjustment. Then it will align the size up to the next 512-byte boundary, and write the size into the header. Then it will calculate the CRC for the entire image, save for the last two bytes. The final step is to bruteforce the last two bytes to find a value that gives a CRC result of 0. These are then written to the image, and the result should be a valid cartridge image.

I will probably put this tool in the DOS SDK when it has matured a bit. If you are curious, other people have also created cartridges, and released tools, for example, this entry on Hackaday by SHAOS.

For completeness, this more elaborate template demonstrates an initialization function, which registers an int 18h handler, and implements two DOS commands:

.8086
.Model Tiny

; Macro to make sure the jump to the command always takes 3 bytes
CARTPROC MACRO target
	LOCAL start

	start EQU $
	
	jmp target
	
	org (start+3)
ENDM

.code
; Cartridge Header
marker dw 0AA55h
len db 0
jmp InitROM
org 6

; DOS Commands
db 5		; Length of command string
db "TEST1"	; Command must be uppercase
CARTPROC test1	; This must be a 3-byte field, so a relative jump with 16-bit offset, or else pad with a nop before the next command

db 5		; Length of command string
db "TEST2"	; Command must be uppercase
CARTPROC test2	; This must be a 3-byte field, so a relative jump with 16-bit offset, or else pad with a nop before the next command

; End DOS Commands
db 0

; Zero-terminated string in DS:SI
Print PROC
	; Get current video mode, so current page is set in BH
	mov ah, 0Fh
	int 10h
	
	lodsb
	test al, al
	je done

outputString:
		; Output char to TTY
		mov ah, 0Eh
		int 10h
		
		lodsb
		test al, al
		jne outputString
done:
	ret
Print ENDP

; Start of ROM data
InitROM PROC
	mov si, offset initString
	call Print

	; Set INT18h handler
	pushf
	cli
	
	push ds
	xor ax, ax
	mov ds, ax
	
	mov ds:[18h*4 + 0], offset EntryPoint
	mov ds:[18h*4 + 2], cs
				
	pop ds
	popf		
	
	retf
InitROM ENDP
		
EntryPoint PROC
	sti

	push cs
	pop ds

	mov si, offset entryString
	call Print
	
	iret
EntryPoint ENDP

initString db "Cartridge Initialization",0
entryString db "Cartridge EntryPoint",0

test1str db "test1",0
test2str db "test2",0

test1 PROC
	push ds
	push cs
	pop ds

	mov si, offset test1str
	call Print
	
	pop ds
	
	mov ax, 4C00h
	int 21h
test1 ENDP

test2 PROC
	push ds
	push cs
	pop ds

	mov si, offset test2str
	call Print
	
	pop ds
	
	mov ax, 4C00h
	int 21h
test2 ENDP

.data

END

That gives you all the possible functionality, aside from BASIC stuff. You could combine everything in a single cartridge as this template demonstrates, but the idea is to just show the different techniques, so you can pick and choose. If you just want a DOS cartridge, you don’t need the initialization and int 18h handlers. Just a retf at initialization is good enough.

And the int 18h setup is only for when you want the cartridge to start only when no boot device is found. So you can control whether or not to run the cartridge by inserting a bootable floppy. If you want the cartridge to always run when inserted, just put your code directly in the inititalization.

Running the cartridge code

Now that we know how to create a cartridge image, how do we actually run and test it? The easiest way to do a quick test is to use an emulator. A file format for PCjr cartridges exists, originally developed for the Tand-Em emulator (which, as the name implies, was originally a Tandy 1000 emulator, but the PCjr is of course a close relative). A utility by the name of JRipCart was developed to extract PCjr cartridges to files with the extension .JRC, for use in emulators. The JRC format is very simple. It’s just a header of 512 bytes in size, with some metadata about the ROM, which is prepended to a raw cartridge image. You can easily make the header yourself, and then use the copy-command to append the two files into a new .JRC file:

copy /b HEADER.JRC+CARTRIDGE.BIN CARTRIDGE.JRC

A popular PCjr emulator with support for .JRC is DOSBox. You can load a .JRC file via the ‘BOOT’ command when you are using the PCjr machine configuration, similar to how you’d boot a floppy image. Note that DOSBox support is very limited, and the current release at time of writing, .074-3, does not actually verify the length field or the CRC, so it does not tell you whether or not your cartridge ROM is fully valid. It merely loads the ROM and executes the code from offset 3.

But of course we would also want to run it on actual hardware. There are various ways to do that. You can go the ‘classic’ way of creating an actual PCB with ROMs on it. You can find the schematics online from various projects, including one from SHAOS, or from Raphnet. The SHAOS-design is true to the original PCjr cartridges, and uses 32k EPROMs. So you need two 32k EPROMs for a 64k cartridge. Raphnet uses a design based around a more modern EPROM type which is available in 64k, so a single EPROM is enough (although it requires a 74LS08 IC for some glue-logic).

Speaking of Raphnet, he has also developed an interesting PCjr cartridge, which allows you to use an SD-card as a harddisk. That is especially interesting since the cartridge port is a read-only interface. So the design uses a clever hack to allow writing to the device as well.

However, this design currently does not allow you to store cartridge images on the SD-card, and load them as ROMs, as far as I understand. There is another device available, which can do exactly that, the BackBit Pro cartridge:

I bought one of these, for two reasons:

  1. I don’t currently have the tools to (re)program EPROMs.
  2. Using an SD-card is more convenient than having to reprogram EPROMs and putting them on a PCB everytime during development.

I suppose once you’ve developed and tested your cartridge image, you can always have a ‘real’ EPROM cartridge made from it later.

The BackBit Pro cartridge is a very interesting device, in that it is a ‘universal’ cartridge, that you plug into a suitable adapter for your target system. There is quite a selection of adapters available, and last year, a PCjr adapter has been added:

It works in a way similar to other devices that simulate cartridges, floppy drives and such via SD-cards, USB drives and whatnot: When you boot up the system (or press the reset button), a menu pops up, which allows you to browse the SD-card and select a ROM file. The system then reboots and runs the cartridge.

The BackBit Pro has a few other tricks up its sleeve. Aside from emulating cartridges, it can also be used to load alternative BIOSes (these were a thing on the PCjr, for example, to make it appear like a regular PC for better compatibility, or to skip the REALLY slow memory test when you have a large memory expansion). And it can load .COM files or floppy images directly (great for booter games).

The current version only seems to support raw cartridge images, which have to have the .BIN extension. It does not appear to support .JRC files yet. But that’s not a big deal, as it’s trivial to convert a .JRC to a raw image: just cut off the first 512 bytes.

I also had some issues with my jr-IDE installed: the BackBit Pro selection screen would not start. You would see it change the border colour to its characteristic pink colour, but then the system froze. Other than that I didn’t encounter any problems, so I can certainly recommend the BackBit Pro for cartridge development, or just as a nice tool for running software on the PCjr.

Update (5-6-2023): The BackBit Pro has received a firmware update (2.1.2), which solves the issue with the jr-IDE. Apparently the issue was that the jr-IDE switched to 80-column mode, which also enables a different page of video memory. So the BackBit Pro was actually working, it was just updating a part of VRAM that was not visible at the time. It now switches back to 40-column mode and a known page of display, which fixes the issue. I think in general it’s a good idea not to make assumptions about what mode and display page you’re in, when writing BIOS or cartridge ROMs. I have found some issues with using the BIOS int 10h TTY output from a cartridge as well.

The firmware also adds support for .JRC files, so you don’t need to convert them before copying them to your SD-card. Excellent work!

Anyway, hope this has been a helpful primer into the world of PC/PCjr cartridges and BIOS extension ROMs.

Posted in Hardware news, Oldskool/retro programming | Tagged , , , , , , , , , , , , , , , , , , | Leave a comment

Another adventure in downgrading, part 4: Fixed function

A few months ago, I discussed downgrading a modern codebase to .NET 4 and Windows XP. I managed to get the code working to the point that all functionality worked, aside from web views, given that browsers no longer support XP or Vista. The .NET 4/XP version of the application makes use of a DirectX 9 renderer and DirectShow or VLC for video playback. DirectX 9 you say? Well, I suppose I have to be more specific. So far we have only looked at the software side of things: which components, frameworks, libraries, APIs etc. are supported on Windows XP? But we worked under the assumption that the hardware had the same capabilities as the hardware that ran newer versions of Windows. Under certain circumstances, your software will also be limited by the capabilities of your hardware. This is especially true for DirectX, as it is a low-level hardware abstraction layer.

DirectX 9, the Swiss Army Knife of graphics APIs

As said, this codebase started life around 2008, at which time shader hardware was standard, even with low-end integrated GPUs. The Aero desktop required a minimum of a DirectX 9 GPU capable of Shader Model 2.0. The codebase made use of shaders through the Direct3D Effect Framework. DirectX 9 is an interesting API as it covers a lot of ground in terms of supported hardware. While at the high-end, it supports Shader Model 3.0, with floating point pixelshading, branching and whatnot, it also supports the first generation SM1.x hardware, and even the pre-shader hardware that was designed for DirectX 7 and below, where we had a fixed function pipeline. So DirectX 9 allows you to go all the way from relatively early 3D accelerators, such as an nVidia TNT or GeForce, or the original ATi Radeon, all the way up to floating-point shader cards. Crysis is a good example of what you can do with DirectX 9 when pushed to the extreme. The original Crysis had both a DirectX 9 and a DirectX 10 backend. While the DirectX 10 rendering quality was absolutely groundbreaking at the time, its DirectX 9 mode is not even that much of a step backwards visually, and it will run on Windows XP systems as well. Earlier games, like Far Cry and Half Life 2, would use DirectX 9 to support a wide range of hardware from fixed function all the way up to SM3.0.

But can our application also support this wide range of hardware? Now, as you may recall from some of my earlier exploits with old GPUs, recent DirectX SDKs include a compiler that will output only SM2.0 code or higher, even if the source code is written for a lower shader version. This also applies to Effects. As far as I can tell, our software has always used this modern compiler, or at the least, all shaders assumed SM2.0 or higher. You need to use an older compiler if you want to use Effects on actual SM1.x or fixed function hardware, otherwise the compiler will silently promote effects to SM2.0, and will only work on SM2.0+ hardware.

We build and distribute our application with pre-compiled shaders. We use the fxc.exe shader compiler for that. I had already created some scripts that compile a set of DX9 and a set of DX11 shaders separately, as the two APIs cannot share the same shaders. So I introduced a third set here, which I called ‘dx9_legacy’. I renamed the old fxc.exe to fxc_legacy.exe and added it to the build with a script to compile a new set of shaders from a dx9_legacy source folder and output to a dx9_legacy folder.

From there, I had to modify the application to support these alternative Effect files. That was relatively simple. Like before, I had to add the D3DXSHADER_USE_LEGACY_D3DX9_31_DLL flag when loading these legacy Effects. Or in this case, it’s actually the SharpDX equivalent: ShaderFlags.UseLegacyD3DX9_31Dll.

And I had to select the proper set of Effects. That is quite simple, really: if the hardware supports SM2.0 or higher, then you don’t need the legacy shaders, else you do. It gets somewhat more complicated if you want to support every single version of hardware (fixed function, ps1.1, ps1.3 and ps1.4). Then you may want to have a separate set for each variation. But at least in theory, I can run any kind of code on any kind of hardware supported by DirectX 9 now, as the legacy compiler can compile Effects for all possible hardware (it can also do SM2.0+, although the newer compiler will likely generate more efficient code).

More specifically, I only had to check if Pixel Shader 2 or higher was supported. Namely, first of all, the new shader compiler still supports Vertex Shader 1.1. Only pixel shaders are promoted to ps2.0. And secondly, there is the option for software vertex processing, where DirectX 9 can emulate up to vs3.0 for you. In my case, the vertex shading is relatively simple, and meshes have low polycount, so software processing is not an issue. Which is good, because that means I do not have to build a fixed function vertex processing pipeline next to the current shader-based one. All I have to do is rewrite the pixel shader code to fixed function routines, and the code should run correctly.

Or actually, there was a slight snafu. Apparently someone once built a check into the code, to see if the device supports SM2.0 at a minimum. It generates an exception if it does not, which terminates the application. So I decided to modify the check and merely log a warning if this happens. It’s purely theoretical at this point anyway. Hardware that does not support SM2.0+ has been EOL for years now, so it is unlikely that anyone will even try to run the code on such hardware, let alone that they expect it to work. But with our legacy compiler we now actually CAN make it work on that hardware.

A willing test subject

I have just the machine to test this code on. A Packard Bell iGo laptop from 2003 (which appears to be a rebadged NEC Versa M300):

It is powered by a single-core Celeron Northwood (Pentium 4 derivative) at 1.6 GHz. The video chip is an ATi Radeon 340M IGP. The display panel has a resolution of 1024×768. It originally came with 256MB of memory and a 20 GB HDD. These have been upgraded to 512MB (the maximum supported by the chipset) and a 60 GB HDD. It came with Windows XP Home preinstalled, and that installation is still on there.

The Radeon 340M IGP is an interesting contraption. It reports that it supports Vertex Shader 1.1 in hardware, but it is likely that this is emulated on the CPU in the driver. The pixel pipeline is pure DirectX 7-level: it is taken from the original Radeon, codename R100. It supports three textures per pass, and supports a large variety of texture operations. This is exactly what we like to test: Effects with a simple vertex shader, and fixed function pixel processing.

So I started by converting a single Effect to vs1.1 and fixed function. I chose the Effect that is most commonly used, for rendering text and images, among others. This will be my proof-of-concept. I first developed it on a modern Windows 11 machine, where it appeared to render more or less correctly. That is, the alphachannel wasn’t working as it is supposed to, but text and images basically appeared at the correct place on screen, and with the correct colours, aside from where they should have been alphablended.

Well, good enough for a proof-of-concept, so I decided to move over to the old laptop. Initially, it opened the application window, which is good. But then it didn’t display anything at all, which is bad. So I looked in the log files, and found that there were some null pointer exceptions regarding font access.

Interesting, as the application had been made robust against missing fonts in general. But as I looked closer, this was for the initialization of some debug overlays, where there was some special-case code. We use the Consolas font for certain debug overlays, as it is a common monospace font. However, apparently it is not THAT common. I hadn’t run into this problem on my other Windows XP machines. But as it appears, the Consolas font was not shipped with Windows XP. It could be installed by various other software though, such as recent Office applications. That might explain why the font was available on my other Windows XP machines, but not on this one. So as the application initialized the overlays on startup, it tripped over the missing font, and could not recover.

I added the font to the system, and tried again, and indeed: the application now worked, and rendered exactly the same as on the modern system. So the proof-of-concept works. For completeness I also added some checks to the code, so it will not crash on missing fonts in the future.

Success

This proof-of-concept shows that everything is in place, at least from a technical point-of-view, for the support of non-shader hardware. We can compile Effect files for non-SM2.0 hardware, and load them from our application. We can create a DirectX 9 device on the old hardware, and detect when to use the fallback for the legacy compiler and alternative legacy Effect files.

The only thing that remains is to actually write these Effect files. I will probably not convert all of them, and certain ones will not convert to fixed function anyway, as they are too advanced. But I will at least see if I can fix the alphablending and add support for video playback, and perhaps some other low-hanging fruit, so that basic stuff will display correctly.

It’s interesting how far you can stretch a single codebase, in terms of development tools, APIs, hardware and OS support. On this old laptop, the code can work fine, in theory. You now run into practical problems… For example, yes it supports video playback, with a wide range of formats. But it has very limited capabilities for hardware acceleration, especially for more modern codecs. Also, we are now back to a screen with 4:3 aspect ratio, where our content has been aimed at 16:9 for many years, and more specifically 1080p, which is far higher resolution than this system can handle. Also, we normally have 2GB of memory as the absolute minimum. This system only has 512MB, and that is shared with the IGP as well. You can configure how much of it to reserve for the IGP. By default it is set to 32MB, so you only have 480MB left for Windows and your application. That puts some limits on the fonts, images and videos you want to use, as you may run out of memory simply because your source material is too big.

But, at least in theory, the code can not only run on Windows XP, but it can actually be made to run the hardware of that era. With the right combination of content and Effect files, you can use this laptop from 2003. So where I normally use a 1080p30 test video with H.264 encoding, in this case I had to transcode it down to 720×480 to get it playing properly. H.264 does not seem to bother the system that much, once you install something like the LAV Filters to get support in DirectShow (you need an older version that still works on XP). But decoding frames of 1920×1080 seems to push the system beyond its limits. It does not appear to have enough memory bandwidth and capacity for such resolutions. Back in 2003, HD wasn’t really a thing yet. You were happy to play your SD DVDs in fullscreen.

As I converted a few Effects down to fixed function, I concluded that it is highly unlikely that this code had ever been used on non-shader hardware before. Certain implementation details were not compatible with fixed function. For example, certain constants, such as solid colour or opacity (alphachannel) values were multiplied in the pixel shader, while the vertex shader merely copied the vertex colour. With fixed function, there are a few constant registers that you could use, but that would require setting these registers specifically in your Effect, instead of setting shader constants. But since these are constants, it makes much more sense to calculate them in the vertex shader, and just output a single diffuse colour, which can be used directly in the fixed function pipeline. It is simpler and more efficient.

In general there’s a simple rule to follow… In order from least to most instances per scene, we have:

  1. The world
  2. Objects
  3. Meshes
  4. Vertices
  5. Pixels

So, you want to process everything at the highest possible stage where it is invariant. For example, the lights and camera are generally static for everything in the world for an entire frame (they are ‘global’). So you only need to set them up once. You don’t recalculate them for every object, mesh or vertex, let alone every pixel. So in general you only want to calculate things in shaders that you cannot precalc on the CPU and pass to the shaders as constants efficiently. And you don’t want to calculate things in a pixel shader that you can calculate in a vertex shader. After all, there are normally far fewer vertices in your scene than there are pixels, so the vertex shader will be executed a lot less often than the pixel shader.

Another interesting detail is that the fonts were stored in an 8-bit format. This was effectively an alphablend value. The text colour was set to an Effect constant, so that a single font could be rendered in any colour. However, the format chosen for the texture was L8 (8-bit luminance). In the pixel shader, this value was read, and then copied to the A component, while the RGB components were set to the constant colour. I couldn’t find a way to make this work with fixed function. The fixed function pipeline treats colour and alpha operations as two separate/parallel pipelines. For each stage you can calculate RGB separately from A. However, most operations will not allow you to move data from RGB to A or vice versa. And when you read an L8 texture, the value is copied to RGB, where A will always be 1, as the texture does not contain alpha information.

So instead, the font should be using an A8 format (8-bit alpha) instead. Then A will contain the value, and RGB will read as 1, because the texture does not contain colour information. That is how the shader should also have been designed, semantically. It should have read the A-component of the texture into the A-value of the output pixel, rather than reading the RGB components into the A-value.

So this once again shows that older systems/environments have limitations that can give valuable insights in the weaknesses of your codebase, and can make your codebase more efficient and more robust in ways that you might not normally explore.

I have decided to once again port back the small fixes and modifications to the current codebase. This way I can develop the legacy Effects on the current build of our software, and do not need to rely specifically on the .NET 4 version. I have decided not to actually put the legacy Effect source code and compiler into the main codebase though. Otherwise the legacy shader set would end up in production code. However, the code knows of this set, so if you manually copy it to the correct place on disk, it will automatically make use of it.

Posted in Direct3D, Oldskool/retro programming, Software development | Tagged , , , , , , , , , , , , , , , , , , , , , , | Leave a comment

The DOS SDK

A thing I have been working on, on and off, for many years now, is a set of headers and helper routines for programming DOS machines directly on the hardware in assembly and C.

As you may recall, my earliest retro-programming blogs focused on the Amiga. And for the Amiga you have what is called the “Native Development Kit”, or NDK. It’s a collection of C header files and assembly include files, which contain all sorts of constants, type definitions and such to interface directly with the hardware. This allows you to write nicely readable and maintainable code (although there’s a bit of irony in the fact that many Amiga coders came from C64, and were used to very raw and basic assembly, and just hardcoded addresses and values. And they took that raw style of assembly programming to the Amiga).

When I started my DOS retroprogramming, there was no equivalent available. So I slowly started building my own include and header files to codify any hardware-specific stuff from the chip manuals or Ralfs Interrupt List and BOCHS’ ports.lst. The idea has always been to eventually release it. But I never got round to that, until now. I have put it on Github here:

https://github.com/Scalibq/DOS_SDK

And here is the README for a short introduction:

An SDK for developing DOS software for x86 machines, including IBM PC compatibles and NEC PC-98

This SDK (Software Development Kit) is modeled after the Amiga NDK (Native Development Kit). The Amiga NDK contains a set of header files and libraries for both assembly and C development, which provides all the required constants, flags, data structures and whatnot to interface directly with the hardware, and having readable code making use of human-readable symbols and type definitions. An equivalent for the IBM PC platform, or PC DOS/MS-DOS/compatible environments has never been available to my knowledge. This SDK attempts to fill that void. Think of it as Ralfs Interrupt List and Bochs ports.lst turned into .inc/.asm and .h/.c files ready for use in a programming environment.

There are some basic rules in how this SDK is structured. Since a PC is composed of a number of off-the-shelf chips, the SDK is structured in a way to reflect this. This means that the definitions related to specific chips such as the 6845, the 8253, 8259 etc. are separated from how they are implemented in the PC architecture (the IO addresses, memory addresses, IRQ and other resources they use). A header file for a specific chip will only contain the generic information for the chip. A separate system-specific header file (in this case IBMPC.inc/IBMPC.h or PC98.inc/PC98.h) will then contain the information specific to the implementation of that system. This allows you to use the header file for the chip for any system that implements it. This is especially useful for writing code for both IBM PC and NEC PC-98, which mostly use the same hardware, but not at the same locations. In future, it may also be expanded to other systems, such as the Tandy 2000.

For system-specific or hardware/standard-specific definitions, a prefix is used for the symbols. For example, the IBM PC-specific symbols are prefixed with PC_, and NEC PC-98-specifc symbols are prefixed with PC98_. For graphics standards, we can see MDA_, HERC_, CGA_, EGA_ and VGA_ prefixes.

This is a work-in-progress. Feel free to contribute changes, additions or suggestions to make the SDK as complete as possible.

Posted in Oldskool/retro programming, Software development | Tagged , , , , , , , , , , , , , , , , , , , | 1 Comment

A great upgrade for the PCjr: the jr-IDE

Adding a hard drive to a PC was a reasonably standard upgrade even in the 80s. And in today’s world of retro-computing, we have the XT-IDE card, which adds proper IDE support to old PCs. Which also allows you to use more modern storage, such as Compact Flash cards and Disk-on-Module (DOM) devices.

Before getting into the actual topic of today, the jr-IDE, I want to clear up some confusion regarding IDE first. There are three basic things you need to know about IDE:

  1. IDE stands for Integrated Drive Electronics. This means that, unlike early hard disk systems, the actual controller logic is integrated on the drive itself. The IDE ‘controller’ on the PC side is little more than an interface (very similar to Roland’s ISA card and MPU-401 MIDI solution).
  2. IDE was first introduced in 1986, for AT-class machines, and is more or less a direct connection of the ISA bus to the integrated controller on the drive. As such, alternative names are AT-BUS or ATA (AT Attachment). With the introduction of Serial ATA, classic IDE became known as Parallel ATA.
  3. IDE is a 16-bit protocol, which made it incompatible with PC/XT-class machines, as they only had an 8-bit ISA bus.

In the early days of IDE, PC/XT-class machines were still relevant, and as such, there was a short-lived XT-compatible variant of IDE, which goes by the name XTA or (confusingly) XT-IDE. It was flawed because it used the same approach as regular IDE: a direct bridge between the ISA bus and the drive controller. As the ISA bus was only 8-bit in this case, the integrated controller on the drive had to be specifically compatible with this 8-bit IDE variant. Only a handful of such drives were ever made.

The new XT-IDE

Back in the day, there were a handful of 8-bit IDE controllers available that DID work with 16-bit drives (such as the ADP50). After all, 2×8 bit is also 16-bit. So if you implement a small buffer on the controller, which buffers the 8-bit ISA bus reads and writes and converts them to a single 16-bit IDE bus read or write, you can make any 16-bit IDE drive work on an 8-bit XT bus. From there, all you need is a custom BIOS that implements the required int 13h disk routines that properly interface with this simple buffered interface.

These controllers were rare however, so they are difficult to get hold of for retrocomputing. And this is a problem that the XT-IDE aimed to solve: some retrocomputing enthusiasts developed their own modern IDE controller specifically for 8-bit PC clones. It is a cheap and simple design, which is open source, and you can buy them in kit-form or pre-assembled from a variety of places on the web. And the XT-IDE universal BIOS is also freely available. It even supports DMA transfers.

Yeah yeah, get to the jr-IDE already

This rather elaborate introduction was required to have a good idea of what an XT-IDE is, and why you want one in your vintage 8 bit DOS-machine. And well, the PCjr is an 8-bit DOS-machine. Since it has no ISA slots, you cannot use the XT-IDE as-is. However, it has a ‘sidecar’ expansion bus on the side of the machine, which is similar to an ISA slot. There are adapters available that allow you to use an ISA card in the sidecar slot:

There are some limitations (not all IRQs are available on PCjr, there is no DMA, and BIOS ROMs may or may not work depending on how compatible they are with the PCjr environment), but you can at least get certain ISA cards working on a PCjr with this contraption, including some hard disk controllers.

And that’s interesting. Namely, since the PCjr failed in the marketplace, hard disk interfaces are extremely rare for the PCjr, and to my knowledge, no IDE interface existed, only SCSI or MFM. Which will be different to interface with more modern drives. So, some PCjr fanatics tried to get the XT-IDE working on the PCjr. Did it work? Not initially, but with some massaging of the BIOS, they could get it running.

The obvious next step would be to not use an ISA card with modified BIOS in an ISA-adapter for the PCjr, but to design a new PCB that would plug into the PCjr’s sidecar slot directly, which would no longer make it an XT-IDE, but rather a jr-IDE. Somewhere around this point, Alan Hightower of retrotronics.org joined the project. He would design the final PCB that became the jr-IDE as we know it today.

This project quickly got out of hand though (in a good way), as people were thinking: “If we’re going to make a new sidecar for PCjr anyway, why not add all sorts of other useful features as well?” This resulted in the following:

  1. The IDE interface (including phantom power on pin 20, to power Disk-On-Module devices directly)
  2. 512k of flash ROM BIOS that is fully PCjr-compatible, which can contain the required firmware for the IDE controller, as well as other things you would like to add
  3. A fully PCjr-compatible BIOS image that can boot directly from hard drives
  4. 1 MB of static RAM
  5. Battery-powered real-time clock
  6. Power On Self-Test (POST) display

See the official retronics jr-IDE page for more background information on the design and features.

I think items 4, 5 and 6 may require some explanation…

128k ought to be enough for anyone

In the category of “Things Bill Gates never said”, we have the memory limit of the PCjr. This is at 128k, rather than the common 640k of the regular PC. Or well, technically it is at 112k, give or take. I already made a short mention of it in an earlier article:

This limitation is a result of IBM sharing the system memory between the CPU and the graphics adapter. IBM designed it so that the memory is divided into ‘page’ of 16k each (by default, it will use 16k of system memory and mirror it to segment B800h, to be compatible with the CGA standard, which has 16k of video memory there, which means there’s only 112k left to DOS from the 128k, not a whole lot). With a maximum of 128k internal memory, you can select from a total of 8 pages. Although it is possible to add extra memory to the machine via a sidecar, it is not possible to address the extra memory as video pages. So the video memory always has to be in the first 128k of memory (this is a limitation that was overcome in Tandy 1000 models, which put memory expansions before onboard memory, effectively moving the onboard memory, including video RAM up, so the video RAM was always in the last 128k of the total system memory, instead of the first 128k on the PCjr. Some models even allowed you to add extra memory beyond 640k, which was only used for video).

So, what does it matter where your video memory is exactly? Well, in theory it doesn’t, if you are running PCjr-specific software, for example from a cartridge, of self-booting software, which are aware of the video memory being somewhere in the first 128k. But if you use DOS, there is a problem: DOS does not understand the PCjr memory layout.

DOS was designed with the assumption that system memory is a single contiguous area of memory, starting from address 0, and going up to a limit that is reported by the system BIOS. This assumption does not hold for the PCjr. So what can you do? Well, the workaround is as beautiful as it is kludgy: you write a dummy device driver, which allocates all the memory up to 128k. This driver is loaded at boot time, so DOS will consider all memory below 128k in use, and once DOS is booted, it will load any application into the extended memory beyond 128k. This means that it won’t interfere with the video memory.

The downside is of course that you are wasting memory below 128k, which is neither used by DOS nor by video. Since the device driver ‘owns’ the memory, it can repurpose it for other uses. So a common use is to make a small RAM disk out of it, so you can store small files in-memory.

This article also explains it nicely with some diagrams. The most popular driver that allows you to make use of PCjr memory expansions under DOS is JRCONFIG.SYS, originally written by Larry Newcomb.

Fun fact: because static RAM is used, no memory refresh is required. As you may know, the original IBM PC uses a combination of the PIT and the DMA controller to periodically (once every 72 IO cycles) read a single byte, which effectively refreshes the system memory. The PCjr has no DMA controller. It does not need to refresh its memory explicitly, as it is shared with the video controller, which periodically reads from memory anyway (once every 4 IO cycles), to display the image.

So where a stock 64k or 128k PCjr is slower than a PC because the video controller inserts more wait states on the bus than the DMA refresh does… with a static RAM expansion it is actually the opposite! Since the static RAM is not used as video memory, and does not require a refresh either, there are NO wait states. This means that the PCjr actually runs slightly FASTER than a PC now. Also, where 112k (or even 96k if you want to use the fancy PCjr modes that require 32k of VRAM) is very cramped for any kind of DOS application, you can now have over 600k available for DOS, similar to a regular 640k PC configuration.

Because, wait a second, there’s 1 MB of static RAM on the card? Yes, well, similar cards also exist for the IBM PC. And you cannot necessarily use everything. However, on the PC you CAN use upper memory blocks (UMBs) above the 640k range. It’s just that the memory directly above 640k is reserved for EGA or VGA adapters (segment A000h). So DOS itself will not normally go beyond 640k, and UMBs are a hack to load small device drivers in parts of the range above 640k that is not already mapped to a hardware device.

On a PCjr however, you have a fixed onboard video controller. And it is basically an extended CGA controller, which maps its memory at B800h. So there is no reason why you can’t just push memory up to that limit, which is 736k. This would make up for the ‘lost’ memory below 128k where the video buffers are mapped. JRCONFIG.SYS will do this for you, giving you well over 600k of free conventional memory in DOS. So, with the jr-IDE you not only get FASTER memory than on a regular IBM PC, you get MORE memory as well.

What time is it?

As you may know, the IBM PC/AT introduced a standardized battery-powered clock as part of the MC146818 CMOS chip. But what happened before that? In the early days, computers had no absolute time source at all. DOS used timestamps for its files, so a proper real-time date and time were required. It did this by prompting the user to enter the correct date and time at every boot. Then DOS would use the PIT at 18.2 Hz to periodically update this internal time (this also meant that if you reprogrammed the PIT for your own software, and didn’t call the original handler periodically, the time would not be updated. So running certain software would mess up the time keeping of DOS).

Technically this never changed, once the AT was introduced. The only difference is: when DOS detected an AT, it knew there would be an MC146818 or compatible chip with a real-time clock (RTC) available. And it could just read the correct date and time at startup, rather than having the user enter them manually. The actual time keeping during the DOS session was still done via the PIT at 18.2 Hz.

But for PC/XT-class machines, there were also third-party addons for a battery-powered RTC. And many PC/XT-class clones would integrate a battery-powered RTC on the motherboard. They basically worked the same, except you had to load a non-standard driver from CONFIG.SYS or run a command line utility from AUTOEXEC.BAT to get the date and time from the RTC into DOS at every boot.

And this is what the jr-IDE does as well. It adds a Dallas DS12887 chip, which is a clone of the MC146818. So it is actually register-compatible with an AT. The only difference is the timer interrupt, as the PCjr has only one PIC, where the AT connects it to the second PIC. The sidecar interface only supplies three interrupt lines (IRQ1, IRQ2 and IRQ7). So you can configure the jr-IDE to select one of these, or disable it if you want to keep these scarce resources available for other devices. And it supplies a simple utility to load the date/time at bootup. There we are, you never have to manually enter the date and time again, at every boot.

POST cards from the edge

Power-On Self Test. You are probably familiar with that. Every PC-like machine will first do a self-test when it is powered on, before it will try booting from a disk. The most visible part of that is usually the memory self-test, which shows a counter of how much memory has been detected/tested. This POST is part of the BIOS.

But what you might not know, unless you have been hacking around with BIOS code before, is that as early on as the IBM 5160 PC/XTs, many BIOSes had a simple debugging mechanism built in. The BIOS would write a status byte to a designated port at different parts of its self-test, as a primitive progress indicator.

If you have problems with booting such a machine, you can install a so-called POST-card into the machine, which is a simple ISA card with a two-digit display, which allows you to see the codes as the BIOS is performing its tests.

The jr-IDE also integrates such a POST display on its board, and makes it available through port 10h, which is what the PCjr BIOS uses. This can be a very useful debugging tool when you want to play around with your own low-level/BIOS code.

But does it perform?

There is more than one way to skin a cat. And the same goes for implementing a hard disk interface. On an x86 CPU, you have two ways of interfacing with external devices. You can either map the registers (or onboard storage) of an external device into the memory address space of the CPU, as most systems do. Or, a somewhat quirky characteristic of the x86: you can map the registers into the IO address space of the CPU (‘port-mapped IO‘).

The IO address space of the x86 seems to be a leftover from its earlier 808x predecessors: when you have limited address space, having separate IO space can prevent you from having to sacrifice precious memory address space for device IO. This made sense for the 8080 and 8085, as they only had 16-bit address space. For the 8086 however, it made considerably less sense, as it had a 20-bit address space. It’s also somewhat archaic and cumbersome to use, as you only have the special in and out instructions, which have the DX register hardwired as the IO address, and the AL or AX register hardwired as the operand. This makes them far less flexible than using memory addressing, which has various addressing modes, and memory operands can be used by many instructions directly. The in/out instructions are also slightly slower than a regular memory access. Nevertheless, the CPU supported port-mapped IO, and most of the PC was designed around port-mapped devices, rather than memory-mapped ones.

The original XT-IDE was also designed with port-mapped registers. The IDE standard uses 8 registers, which were mapped on an IO-range 1:1. The problem here is that the IDE interface uses a 16-bit register to transfer data, and the IO registers are 8-bit. So in order to implement the ‘missing’ 8 bits, a latch was added on the card, and placed at the end of the IO registers. This meant that the two 8-bit parts of the data register were not in consecutive IO-ports, which means it is not possible to do a more efficient 16-bit in or out operation (which is more efficient even on an 8-bit bus, as there is less overhead for fetching and decoding instructions).

User Chuck(G) then suggested a small modification to the PCB, which swaps some address lines around, so you do get the two data registers side-by-side, and in the correct order. Combine this with an updated BIOS that supports the new addressing, and you got a 2x to 3x speedup.

From this point onward, the paths of the XT-IDE and jr-IDE split. On a regular ISA-based machine, there is a DMA controller available, and DMA transfer is the fastest option, at least, on low-end devices (on faster machines, the DMA controller will be a bottleneck, as it always runs at ~1 MHz. This is enough to saturate the memory of the original PC, at about 1 MB/s in theory, but no more than that). This is because DMA transfers do not require any CPU intervention, so all cycles are spent transferring data, rather than fetching and decoding instructions. James Pearce of lo-tech presented the XT-CFv3, a modified PCB design and matching BIOS, which had DMA capability. It can reach speeds of over 500 KB/s, which is pushing the limits of the machine.

Since the jr-IDE has no DMA controller, and there is no way to even add one via the sidecar interface, as the required lines are missing, a different path was followed: memory-mapping. Instead of making the data register appear only once in the IO-space, a region of 512 bytes is mapped. This is the size of a single sector. This means that you can read or write consecutive bytes on the device by just using REP MOVSW, making things very elegant (as memory-mapped IO does). This is almost as good as DMA, as the instruction needs to be fetched and decoded only once. The looping is done inside the CPU’s microcode. The result is that even on the relatively slow PCjr, you can get over 330 KB/s in transfers. That is faster than any of the port-IO-based XT-IDE implementations. So very impressive indeed. It is a modification that could be backported to the XT-IDE design, but I have not heard of anyone doing this.

Conclusion

So I’m very happy with the jr-IDE I got. The extra memory and the HDD support make it much easier to develop on the PCjr. And I mean that literally: I will use it for development only. I still want to develop software that runs on a stock 128k (or perhaps sometimes even 64k) PCjr. But during development, the extra memory and storage make the process a lot nicer. All in all, it’s a very impressive and useful product, and great work from all the people involved.

By the way, Alan Hightower no longer builds the jr-IDE itself. The design is available from his website, so you can either build one yourself, or get one from a third party, such as TexElec.

Some other resources that may be helpful are Mike Brutman’s PCjr pages. He has a special page on the jr-IDE. On his downloads page, you can find tools such as the aforementioned JRCONFIG.SYS to make use of memory expansions under DOS.

And on the XT-IDE itself, you can find more info on the minuszerodegrees.net XT-IDE page, which contains a large collection of information on the development of the XT-IDE cards, its various revisions, the BIOSes, quirks, bugs and more.

Posted in Hardware news, Oldskool/retro programming, Software development | Tagged , , , , , , , , , , , , , , | 2 Comments

When is a PC not a PC? The PC-98

I’ve covered PC compatibility in the past, and tried to explain how just having an x86 CPU and running DOS does not necessarily make your machine compatible with an IBM PC. At the time, this was mainly about the IBM PCjr (and its clone, the Tandy 1000), which is still relatively close to a regular PC. As such you could create software that would run on both IBM PCs and compatibles, and on the PCjr/Tandy, with custom code paths for specific functionality, such as sound and graphics.

But pretty much all other DOS/x86-based machines failed, as their hardware was too different, and their marketshare was too small for developers to bother adding support. In fact, the main reason that the Tandy 1000 existed at all, is because the earlier Tandy 2000 was falling into the trap of not being compatible enough. The Tandy 1000 may actually not be a very good example, as Tandy tried to make it nearly 100% compatible, fixing the main reason why the IBM PCjr also failed. So later Tandy 1000 models were more or less a ‘best of both worlds’: nearly 100% compatible with IBM PC, but also offering the enhanced graphics and sound capabilities of the PCjr.

Meanwhile, in Japan…

In Japan however, things took a different turn. The Japanese do not just use the Latin alphabet that is used on all Western machines, including the IBM PC. The Japanese language uses more complex glyphs. They have multiple systems, such as kanji, katakana and hiragana. To display these in a comfortably readable form, you need a high-resolution display. Also, where Latin letters encode sounds, a kanji glyph encodes a word or part of a word. This means that your glyph alphabet contains over 50000 characters, a lot more than the maximum of 256 characters in your usual 8-bit Western character set.

So the Japanese market had very specific requirements, that PCs could not fulfill in the early DOS days. You couldn’t just replace the character ROM on your PC and make it display Japanese text (IBM did later develop the 5550 and the JX, a derivative of the PCjr, specifically for the Japanese market, and later, they developed the DOS/V variant, which added support for Japanese text to their PS/2 line, using standard VGA hardware, which by now had caught up in terms of resolution).

Instead, Japanese companies jumped into the niche of developing business machines for the home market. Most notably NEC. In 1981 they introduced the PC-8800 series, an 8-bit home computer based on a Z80 CPU and BASIC. In 1982, the PC-9800 series followed, a more high-end 16-bit business-oriented personal computer based on an 8086 CPU and MS-DOS. These families of machines became known as PC-88 and PC-98 respectively (Note that the ‘PC’ name here is not a reference to IBM, as NEC had already released the PC-8000 series in 1979).

In this article, I will be looking at the PC-98 specifically. So let’s start by doing that literally: here are some pictures to give an impression of what these machines looked like. They look very much like IBM PC clones, don’t they?

For more machines, specs and background info, I suggest this NEC Retro site.

How compatible is it?

It has an 8086 CPU, an NEC 765 floppy controller, an 8237 DMA controller, an 8253 programmable interval timer, and two 8259A programmable interrupt controllers. Sounds just like a PC, doesn’t it (okay, two PICs sounds more like an AT actually, so NEC was ahead of its time here)?

Well it would be, if it used the same IO addresses for these devices. But it doesn’t. What makes it especially weird is that since it has always been a system with a 16-bit bus (using an 8086 as opposed to the 8088 in early PCs), NEC chose to map any IO registers of 8-bit devices either on even addresses only, or on odd addresses only (so the 16-bit bus is seen as two 8-bit buses). For example, where the first 8259A on a PC is mapped to ports 0x20 and 0x21, the PC-98 places it at 0x00 and 0x02, leaving 0x01 as a ‘gap’ in between. The 8237 DMA controller is actually mapped on address 0x01, 0x03 and so on.

Another major difference is that the base frequency of the PIT is not 1.19 MHz like on the PC, but depending on the model, it can be either 1.99 MHz or 2.46 MHz.

And like the PCjr and the Tandy 1000EX/HX models, it has an expansion bus, but it is not the ISA bus. The PC-98 uses the C-bus. So you cannot use standard expansion cards for IBM PCs in this machine.

Clearly the video system isn’t compatible with any PC standard either. It does not even use int 10h as the video BIOS. Speaking of BIOS, the PC-98 BIOS is not compatible with the IBM PC BIOS either. But as said, the video system was far superior to the IBM PC at the time. The first version in 1982 already supported 640×400 with 8 colours, based on NEC’s own uPD7220 video controller. In 1985 they extended this with a palette of 4096 colours to choose from, and an optional 16 colour mode if an extra RAM board was installed. In 1986 the extra RAM became standard on new models, and they also added a hardware blitter for block transfers, raster operations and bit shifting.

What’s also interesting is that they chose to actually use TWO uPD7220 chips in a single machine. One of them is used for text mode, the other for bitmapped graphics mode. They each have their own video memory, and are used in parallel. So you can actually overlay text and graphics on a single screen.

But, on the other hand…

There are two things that we can use to our advantage:

  1. It runs (an NEC PC-98 OEM version of) MS-DOS
  2. A lot of the hardware is the same as on the PC

So this means that for basic functionality such as file and text I/O, memory management and such, we don’t need the BIOS. We can use MS-DOS for that, which abstracts the machine-specific BIOS stuff away. Also, if we write code that uses the 8237, 8253, 8259A or other similar hardware, in most cases we only need to change the I/O-addresses they use, and adjust for the different PIT frequency (and other minor details, such as the different cascaded configuration of the two PICs and different IRQs for devices), and we can make it work on the PC-98.

So just like with Tandy and PCjr, we can write DOS programs and make them work on PC-98. We can even write a single program that can run on both types of systems, even though it is a bit more complicated than on Tandy/PCjr (on those you mainly had to avoid using DMA, and you should be aware that the keyboard is different, so you should only access it via BIOS, or have separate routines for the different machines).

Challenge accepted

I decided to give this a try. I have made my own little ‘SDK’ of headers and library functions for ASM and C over the years, which includes quite a few constants for addressing I/O ports or memory areas of all sorts of PC and Tandy/PCjr hardware (I modeled it after the Amiga NDK). I figured I would try a PC-98 emulator and port my VGM player over to the PC-98, and update the SDK with PC-98 support in the process.

A convenient emulator is DOSBox-X. It is a fork of DOSBox, which adds a PC-98 machine, among other features, and like DOSBox, the BIOS and DOS emulation is built-in, and you can just mount host directories as drives, so you don’t have to juggle all sorts of ROMs and disk images to get the system running. If you want a more serious emulator though, Neko Project 21/W is one of the more compatible/accurate ones.

And indeed, you can just use OpenWatcom C to write a DOS application, and it will work, as the basic runtime only requires DOS interrupts, no BIOS or direct hardware access. All the BIOS and direct hardware access is done via my ‘SDK’ anyway, so as long as I write the correct code for PC-98 BIOS and addresses, I can use any hardware from C (or assembly of course).

What’s more, it turns out to be relatively simple to detect whether you are running on an IBM PC-compatible or a PC-98 compatible machine. A trick that is used is to call int 10h with AH=0Fh. On an IBM PC-compatible, this will return information about the current video mode, with AH containing the number of columns. On the PC-98, this will not be implemented, so after the call, the value of AH will be unaffected. Since there is no video mode with 15 columns, you can assume that if AH is 0Fh after the call, that you are running on a PC-98 machine.

Anyway, before long I had a basic version of my VGM player working on both the IBM PC and the PC-98. It’s a pretty fun and quirky platform so far. So I might be looking into the graphics chip in the near future.

If you also like to play around with DOS and x86, and want to give the PC-98 a try, here are some good resources (you might need to translate, as most documentation can only be found in Japanese):

http://radioc.web.fc2.com/column/pc98bas/index_en.htm

https://www.webtech.co.jp/company/doc/undocumented_mem/

Posted in Oldskool/retro programming, Software development | Tagged , , , , , , , , , , | 6 Comments

Why I left the 8088 MPH team

In short, it wasn’t my choice. Then who decided? Jim Leonard, aka Trixter. So is Jim able to singlehandedly decide who is and isn’t part of the ‘team’? Yes, apparently. So, it isn’t really a ‘team’ then, is it? No, it is not.

What happened is a long and complicated story, some of which you may have been able to read between the lines already in these two earlier blogposts. I think it is only fair that I make the full conversation between Jim and myself available, so it’s not just a “he said, she said”. So you can read back what happened exactly. And I will give my view on this:

In short, given the aftermath of BLM protests-turned-riots after George Floyd’s death, and the fact that there were even BLM/Floyd-inspired protests in my country, I was curious about Jim’s view on this, given that these riots had taken place in his country, and some even close to where he lives.

As far as I am concerned, this was a conversation between two people who have known each other for many years, and could be considered friends. And it was a conversation where I showed interest in current affairs and cultural phenomena.

Also, given that we are more or less the same age, I would have assumed that Jim would have a similar liberal, humanist outlook on these things, given that this was the dominant view in Western culture for as long as we can remember, and this woke ‘successor ideology’ has only come into vogue in recent years. Not to mention that just common sense, logic and rationality would lead you in that direction anyway.

But apparently I was wrong. For some reason, Jim did not apply any kind of rational thinking or common sense, but seemed to have been fully emerged into the dogma’s of the woke cult. Which resulted in him just giving knee-jerk reactions. To the point where he escalated by asking me if I were a “White Supremacist”. Because apparently that’s what you call people who don’t want to racialize everything, and who don’t want to judge people based on the colour of their skin, but on the content of their character?

Shortly after that, I let the conversation rest for about a month, and I wrote the first article on wokeness. I don’t know if Jim read that, but when I picked up the conversation again, a month later, he only insisted even more. Instead of just asking, he started literally calling me a “White Nationalist”, and basically became completely unhinged. For no apparent reason as far as I am concerned, as I have said nothing even remotely racist, unlike him.

Not wanting to get too deep into this whole woke/CRT/intersectionality thing again, but there are a few things I’d like to mention.

First, there is this article that talks about Learned Helplessness, which also explains why this woke view on ‘White Privilege’, ‘White Supremacy’ and such, is actually a form of racism in itself.

Secondly, the article also explains how this creates a mindset where black people think they have less agency than they actually do. The included diagrams make it very obvious that the *perception* of racism has changed a lot, while the actual racism has not. Because of the framing of CRT/BLM, people blame more things on racism/discrimination, rather than on their own actions.

Or, in the words of the ever insightful and eloquent Helen Pluckrose:

Speaking of Helen Pluckrose, for more background regarding woke/postmodernist ideology, as well as its antithesis, premodernist ideology, I can suggest this excellent article from 2017, which is still relevant today:

This also covers another pet peeve of mine: how the enemies of Modernity make everything political, and put every topic and opinion either under the left-wing or the right-wing label. If it’s not the one, it must be the other.

Which boils down to this: CRT/BLM is supported by left-wing. So if I am critical of CRT/BLM, then I must be right-wing, and the fact that I even want to discuss this topic means that the discussion is ‘political’.

None of which is true. But that is what I ran into. First, Jim clearly gave me an ‘ultimatum’: as a proper wokie, he made ‘political’ topics ‘off-limits’, where ‘political’ can be any subject we don’t agree on. Free speech, free opinions… we can’t have any of that! And apparently, it was fine that he called me an asshole, White Supremacist and whatnot… but ohnoes, I responded to his many insults and deliberate misrepresenting of my statements with an f-bomb. Totally inappropriate! I mean, why would Jim hold himself to the same rules as me? He can say and do anything, but I must bow to His Greatness, and only speak when spoken to.

These were terms that I was not willing to accept. So I mentioned on the team mailing list that Jim called me a White Supremacist, and that I was not willing to work under those circumstances. To which Jim responded by removing me from the mailing list, Github, Trello and whatever other team resources I had access to. He even blocked me on Twitter. He tried to cover up this cowardly ego-act by claiming he was afraid that I would delete the team resources if I had access. Yea right… Firstly, I didn’t have the rights on most resources to do that, and secondly, I would never do anything remotely that childish. I guess it says a lot about him that he thinks like that.

Okay, so now the ball was in the court of the rest of the team. Did I say team? Whoops, nope. There is no team. Only one person reached out to me, and we could discuss what happened in more detail. I haven’t heard from the others at all. Apparently they were fine with Jim calling me a White Supremacist, and removing me from the team single-handedly (does that mean they too think this? Based simply on Jim saying that?). They didn’t care that I wasn’t part of the team any longer, or that Jim had made it impossible for me to return to the team, as it stands. The disconnect between him and me would have to be resolved, and it is pretty obvious, at least to me, who is in the wrong here.

I mean, I can’t even begin to imagine how you can think that someone whom you’ve known for many years, who’s never made political references, let alone racist references, is somehow a White Supremacist, and is somehow trying to be a political activist, or whatever it is he thinks. What’s worse, I can’t even begin to understand how you can think that it’s just fine to call someone a White Supremacist, and then think you can still make a demo together. You know that when you do that, you cross a line, one that is not easy to get back from. You chose to burn that bridge.

And I don’t understand either how the rest of the people from 8088 MPH could just be silent when someone boots one of the members out of the team, under some vague accusation of “White Supremacist”. I mean, unless you’ve been living under a rock, it should be pretty obvious that accusations of “White Supremacy” are being thrown around gratuitously by radicalized activists, so when you hear that accusation being thrown around, the first thing you do is smell a rat, not think the accusation is an accurate representation of someone’s stance.

But well, with the exception of one person, apparently nobody thought that. After the release of Area 5150, I asked what exactly the other team members thought. But only two people responded, and both of them gave me the same bullshit ‘political’ nonsense. One of them even went into an unhinged rant, making all sorts of assumptions about me that were obviously wrong. When I pointed out that in all my years of demoscene and writing music, I have NEVER included any kind of political statement whatsoever, and even on this blog, I have only written two blogs regarding ‘wokeness’, which obviously were related to what happened with Jim (but were written AFTER our initial clash, so could not have *caused* the disagreement with Jim), there was no response.

I guess I am just deeply disappointed. Both in people in general, and in these people I considered to be friends in particular. You’d think there’d be some kind of bond, some mutual trust and respect after having made 8088 MPH together, visiting Revision, and then staying in touch for years after that, working on a successor. But nope, apparently none of that means anything to them.

But to me it means I won’t join another team lightly. It takes months, if not years, of intense work and collaboration to make a demo of the caliber of 8088 MPH or Area 5150. I thought it should be obvious that this requires a good team, with mutual trust and respect. But apparently others think you can just call people a White Supremacist and think they’ll just continue making a demo with you. Well, not me anyway.

In closing, I would like to mention the book The Parasitic Mind, by Gad Saad. I read it recently, and I recognized various aspects. Like Gad Saad, I value freedom and truth a lot. And I like to approach things with a healthy combination of rationality and humour. Jim’s behaviour in the conversation can be described as ‘enemy of reason’. Some of Gad Saad’s descriptions are spot-on for this case.

This is also about the two modes of thought, as formulated by Daniel Kahneman. “System 1” is where you act primarily, based on emotions and preconceptions, where “System 2” is more elaborate, logical reasoning.

Gad Saad argues in his book that you can use nomological networks of cumulative evidence to show that something is likely to be true (or false). And if you read back the conversation, you can see that I try to bring in various sources of information, and try to approach topics from various sides. Sadly, Jim does not bother to even look at them. He outright rejects sources, simply based on the messenger (or more accurately: the radicalized activist propaganda against the messenger that he has been exposed to). So there’s a problem with that approach: you can bring a horse to water, but you can’t make them drink. Well, at least I tried…

It just surprised me that even demosceners, who should be more System 2-oriented, can be so socially awkward they they are only going for knee-jerk System 1-thinking when the topic is more social/cultural than technical, and don’t bother to dive any deeper in the subject matter, even when it’s being spoonfed to you, and aren’t able to keep an open mind and have a more logical, rational approach (not even the rather obvious possibility that organizations such as BLM and Antifa might not be exactly what their name implies). And instead, actually choose to behave like a total asshole toward you, and not even have the self-reflection to see what you’re doing. To me, Jim shows all the signs of having been radicalized.

For everyone who is as daft and insensitive as Jim: https://thepushback.us/2022/12/09/the-woke-mind-virus/

Anti-woke isn’t anti-black, anti-immigrant, anti-LGBT, or anti-equality. It’s neither MAGA nor fascist, and it’s absolutely not discriminatory or extremist. Anti-woke rejects the artificially constructed forced morality binary. Elevating arbitrarily chosen “grievances” and protecting select groups above others can only end in disaster. Not all grievance rises to a level that requires or demands redress. Don’t allow yourself to be manipulated.

This sounds like Jim:

Ironically, in practice, wokeness manifests in bullying, threats, identitarian essentialism, and devaluation of others. Many times it’s just narcissism masquerading as empathy.

Update: It has come to my attention that Jim has posted a response: https://trixter.oldskool.org/2022/12/30/the-semantics-of-discourse/

Sadly, it does not address any of the core issues, and he also lies about the fact that he singlehandedly removed me from the mailinglist and other resources. Also, it doesn’t make sense that he claims “we could continue making demos together as long as we never discuss politics again” (which as I already said, is an unacceptable ultimatum), while at the same time he has blocked me from Twitter and other resources. The one who would want to continue is the one who doesn’t block the other party, and that would be me. As I said, just not under circumstances that are unacceptable, so those had to be worked out first. With the whole team obviously (as it was clear at this point that Jim had completely shut down the conversation with me, and there was no way for me to get any kind of point across, and trigger any kind of introspection. He had dug himself in too deep for that). There was no reason to block me if I had just chosen to no longer contribute. But blocking me does guarantee that I CANNOT contribute, and cannot even get in contact. And that is what Jim did. For those who still need subtitles: look up “passive-aggressive behaviour“.

Bonus: there is a rather long comment that provides quite a bit of scientific/historic information, and you can find Jim’s response once again denying science and history, as he’s still fully sucked into the cult. Jim’s response also seems to indicate that he sees no difference between my stance and the commenter’s, while there clearly is a fundamental difference (as anyone who has read my earlier articles on wokeness should be able to pick out). Spoiler:

To me, the only race that exists is the ‘human race’.

Update: Jim continues lying. I was removed from the mailing list almost immediately after my last message. That is BEFORE anyone could even respond. Which means Jim had not had feedback from anyone. Jim was also the sole administrator of the mailing list, so he was the only person who could remove me from it. Ergo, he singlehandedly removed me from it, without consulting the rest of the team (if you read between the lines, he basically admits this, since he doesn’t mention any feedback from the team, between points 4 and 5. He just pulls up a smokescreen to hide that, by claiming I’m wrong, which I’m not. He’s just being manipulative, as we’ve already seen in the private exchange as well).

Aside from the obvious logic that I had no reason to leave the team other than Jim calling me a white supremacist repeatedly, combined with all the other underhanded bully tactics you can find in our mail exchange. It should be obvious that I would have wanted an apology and some kind of reconciliation, but other than that, there was no reason not to continue on the demo. But Jim clearly didn’t want me on the team anymore, so that never happened (Jim isn’t man enough to just outright say “I didn’t want you on the team because I think you’re a racist and white supremacist, and that’s why I took action”. He wants to disguise his actions and motives, but he’s the one who decided to burn that bridge. Now he can’t own up to it).

Even now, if you read his message, he doubles down on me actually being a racist/white supremacist. He just regrets saying it to me. That’s not an apology, is it? The problem is that you think I’m a racist. Not whether or not you say it out loud. Who wants to be friends with a racist? And who wants to be friends with someone who thinks they’re a racist? So what we would need, is some reconciliation where you understand that my viewpoints are not racist (nor political for that matter). Jim is just a terrible person. And he tries to cover up for how terrible he is. He even removed the comment from user ‘Catweazle666’, to hide his denial of science and history. Just keep digging that hole deeper, Jim!

He removed it under the guise of ‘political’. Which it wasn’t. It was a combination of some historical facts, and Catweazle666’s personal view on these. There were no politics involved. History isn’t political (well, it is when you’re woke, because you want to rewrite history to suit your political agenda). So there we are, Jim has given us the perfect proof that he indeed will censor anything he doesn’t agree with, under the guise of it being ‘political’, exactly as I said.

The part where he claims he was “trying to help me” because I would have been “deeply troubled” is hilarious (not to mention completely arrogant and misplaced). Anyone who reads the conversation, can see I was not asking for help, nor was he giving it (but perhaps he means that he thinks that everyone who doesn’t share his opinions, is deeply troubled and needs help). Also, it is clear that I wasn’t deeply troubled. I was more surprised that groups of people in various parts of the country were tearing down entire city blocks in riots, based on ideology that at least as far as I had looked into it, was basically a conspiracy theory (similar to the classic antisemitic conspiracy theory where a small group of powerful Jews would control the world, except the Jews were replaced with ‘White Supremacists’ in this version. A conspiracy theory that Jim has clearly bought into, given that the only ‘proof’ he goes on for me being a ‘White Supremacist’, is that I do not condemn ‘The System’ enough to his liking. As such, I must be ‘complicit’ in this ‘system’, which makes me a ‘White Supremacist’, despite the fact that I have never uttered anything remotely racist or white supremacist myself. That is how radicalized he is). Given Jim’s responses, and his inability to have a reasonable, factual conversation, but instead shut down and start insulting his otherwise calm and rational discussion partner, he is the one who is deeply troubled. Are his social skills that bad that he can’t even read a conversation properly? Or is he again lying and manipulating to make himself look good in the shitshow that he himself created, at my expense?

His claims about deleting resources are also quite sad. The fact that *someone* may have done this in the past still doesn’t mean that *I* would ever do that (aside from the fact that as I already said, I wouldn’t have the rights to do that in the first place). I’m a different person. Apparently Jim is a very poor judge of character. And apparently, Jim thinks very lowly of me. Again, where’s the mutual trust and respect, if I were to do a demo with him? The fundamental difference here is that Jim judges me on things that he *thinks* I would say or do (but never actually said nor did, nor even planned to say or do, as he is very wrong in his judgement), whereas I merely judge Jim on things he *actually* said and did.

His tribal groupthink is also obvious in how he thinks he is a spokesperson for all Americans, when he describes what he thinks. Whereas in the conversation it is clear I only asked him what *he* thought, and would in no way generalize that to ALL Americans. We’re all individuals, all capable of making our own decisions, forming our own thoughts etc.

Posted in Uncategorized | Tagged , , , , , , , , , , , , | 5 Comments

A new MPU-401 solution: The Music Quest clone

This is a draft that I have kept around since March 16, 2019, so it is about time I finally finish and publish it.

This story started somewhere in 2015, when a user named Keropi posted a thread on the Vogons forum, about cloning a Music Quest card. What is a Music Quest card, you may ask? It is a clone of the Roland MPU-401 MIDI interface. Keropi had collected a lot of information on it in a previous thread. But I suppose that begs the question: what exactly is a Roland MPU-401?

I would gather that most people know MPU-401 as a standard for MIDI on the PC, and then usually they are familiar with the UART-mode aka ‘dumb’ mode, since that is the simple interface, which most clones, including many Sound Blasters, support, and which was also used for the later Wave Blaster standard.

An actual MPU-401 is somewhat different though. For starters, strictly speaking, the MPU-401 is not a PC device. It is an external module, and it looks like this:

As you can see, it has a connector labeled “To Computer”. So what do you connect this to? Well, this explains why Roland designed it like this: the MPU-401 is the “MIDI Processing Unit”. This unit contains a Z80 processor and some support chips, and does the actual MIDI handling:

So technically this is like a computer on its own. Roland decided to make this part generic, so they only had to design a simple interface for the various computer systems around in the 80s, and use the same MPU-401 unit for all of them. For the PC, the interface was a simple ISA card, known as MIF-IPC:

So for the PC, a Roland MPU-401 interface was actually a setup of both the ISA card and the MPU-401 unit.

Intelligent Design

Okay, so we mentioned the ‘dumb’ mode. But what is ‘intelligent mode’? This is why the MPU-401 is such a complex design: it is actually a basic hardware MIDI sequencer (Roland calls it a ‘Conductor’ in its manual). That is: you can send MIDI data to it in advance, and queue it up with timestamps, and the device will send out the MIDI data at the correct time by itself. Then it will signal the host with an interrupt that it is ready to receive new data.

Why did they make MIDI so complex back in the day? These were early days. MIDI was introduced somewhere around 1983, and the Roland MPU-401 is to my knowledge the first available MIDI interface for home/personal computers, and was introduced in 1984, and supported computers such as the Commodore 64, Apple II and MSX, and of course the IBM PC.

So we are talking early 80s 8-bit machines, with limited memory and processing power. Sure, these machines could play music, but this was generally synchronized to the display refresh rate, so you had an update rate of 50 or 60 Hz. This severely limited you in terms of tempo. MIDI allowed for much higher resolutions, as it was aimed at professional musicians in a studio setting.

Now, while I have developed a MIDI player myself a few years ago, and explained how you could get the high resolution of MIDI even on an original IBM PC or PCjr, there are two obvious limitations here:

  1. My player requires preprocessing of the data, so you cannot play MIDI files as-is. It can only be output as a single data-stream, no separate MIDI tracks, as you would have in a sequencer.
  2. Playing a MIDI file is possible, but recording and editing of MIDI will not work in realtime with this approach, as you would need to convert between MIDI and preprocessed data.

So for various reasons it made a whole lot of sense to use an MPU-401, which has hardware that is custom-designed to record and play back MIDI data in realtime, with its own clock (we have seen that the IBM Music Feature Card also has its own clock that is designed to handle MIDI timings, although it is not as advanced as the MPU-401 is). You could easily guarantee that your MIDI data was played at the correct time, without having to worry about what the CPU was doing. And editing MIDI data was also simpler, as you wouldn’t need to convert to some internal system clock that was not even remotely similar to the timestamps that MIDI uses (which are based on a 1 MHz clock).

So in short, ‘intelligent mode’ is where you offload playback to the MPU-401, rather than just outputting the MIDI data in realtime, where you need to make sure that your CPU output routine is accurately timing each byte. The MPU-401 has a total of 8 ‘tracks’ internally, where you can queue up timestamped data for each of these tracks in a fire-and-forget fashion.

Who cares?

One of the problems with the MPU-401 was that it was very expensive. Aside from requiring the ISA board and the MPU-401 unit, you’d also need an actual synthesizer. In 1987, Roland introduced the MT-32, a somewhat affordable semi-professional MIDI module (a synthesizer without a keyboard attached, basically), which became somewhat of a de facto standard, as Sierra and various other game developers started to support MPU-401+MT-32 as an audio device.

But by the time MIDI started to gain traction, we were approaching the 90s, and PCs had become much more powerful. On the one hand, playing MIDI in realtime wasn’t such a big problem anymore. On the other hand, games generally had MIDI music that was already quantized to a relatively low resolution anyway (somewhere in the 140-270 Hz range for example) that ‘dumb’ UART mode output was good enough. And since the PC has always been the platform of the lowest common denominator, and most cheap sound cards only supported the UART mode of the MPU-401, most games won’t require ‘intelligent mode’ compatibility.

For that small selection of software that does require a full MPU-401 though, your options were limited. There is a software solution known as SoftMPU, which emulates the intelligent stuff on the CPU-side, so you can upgrade your UART MPU-401 to a full intelligent MPU-401 with this TSR. Downside is that it requires a 386 CPU or better. So for your 286 or older machine, you still need a hardware solution for true MPU-401 compatibility.

But we can anyway

As technology progressed, it became simpler and cheaper to integrate both the host interface and the MPU-401 on a single ISA card. Roland introduced its own MPU-IPC, where the external module was now a simple breakout box:

And later still, they introduced the MPU-401AT, which had mini DIN connectors on the card itself, and a Wave Blaster interface to plug a MIDI module directly on the card:

And of course there were other parties that made their clones of the concept. The original MPU-401 setup was rather complex and expensive to clone. But when one can integrate it all on a single card with off-the-shelf parts, it becomes feasible to try and make a cheaper alternative to the Roland offerings. This is where the Music Quest comes in. It was one of these clones, and it was a good one, as with the later v010 firmware, it is considered to be 100% compatible with Roland:

So, after this lengthy introduction, we can now finally get to the actual card this blog is about:

Because well, if a card is made mostly from off-the-shelf parts, like a Z80 and an EPROM, in this day and age, it is not that difficult for a single skilled individual to design and build their own PCB, dump the roms, and clone the clone.

And this is what was done here. What’s great about this, is that it is a good clone of a good clone: people can now get their hands on reliable MPU-401-compatible hardware at relatively low cost. I received mine a few years ago, and wanted to write about it, as I was quite happy with the product. Better late than never, I suppose.

I received a somewhat later revision, which also includes a Wave Blaster interface:

And another revision has been developed since, which uses a higher level of integration on the PCB:

So, if you want a good MPU-401 interface for your retro DOS machine, either for running MIDI software, or developing your own MPU-401 routines, I can recommend this card. For some more information, software, and a link to the order form, you can go to the Serdashop store page for this card.

Posted in Hardware news, Oldskool/retro programming | Tagged , , , , , , , , , , , , | 1 Comment

A new game for DOS/EGA/AdLib: Super Space Fuel Inc.

It seems there has been a boom in Match-3 games. They seem to be a popular target for casual gaming on mobile devices/in browsers. But what’s better than playing Match-3 on your mobile phone? Playing Match-3 under DOS!

I say ‘new game’, but it was released in August last year. So yes, it’s a ‘new’ game for DOS, but I’m rather late to the party. Anyway, the game is Super Space Fuel Inc., and it was made by some friends I know from my demogroup DESiRE.

Code is done by sBeam, graphics by VisionVortex, and music by No-XS.

The game requires EGA, and has AdLib music. In theory it can run on any machine with an 8088 CPU or better. In practice you probably want at least a Turbo XT or 286 for the best gaming experience.

The game was written in Turbo C. You can download the game for free, and if you make a small donation, you can also download the source code, if you are interested. Of course I hope you will donate, as any retro DOS stuff, especially with EGA and AdLib, deserves a reward.

Dosgamert made a video with gameplay to get a decent impression of the game:

You may recognize the music. And I must say, the graphics and animation look very slick. Hope you like it!

Posted in Oldskool/retro programming, Software news | Tagged , , , , , , , , | Leave a comment

Windows 7: the end is nigh

Windows 7, originally released on October 22, 2009. It’s had a good run of over 13 years so far. At its introduction, I wrote a somewhat cynical piece, given that people were so negative about Windows Vista, and so positive about Windows 7, while technically Windows 7 was much closer to Vista than it was to the XP that people wanted to stick to.

And during its lifetime, Windows 7 remained a favourite, particularly because Windows 8 was a mixed bag, as Microsoft tried to create a single OS for both desktops and tablets/smartphones, leading to a somewhat confused UI with large tiles to cater to touchscreens. And technology-wise, there was little reason to move from Windows 7 to Windows 8 or 8.1 either.

This changed when Windows 10 arrived in 2015, where the UI was more friendly to desktop users again, and Windows 10 also introduced new technology such as DirectX 12. Even so, Windows 7 remained popular, and Microsoft continued to support it.

Until recently, that is. The first cracks in the armour are starting to show. For starters, Microsoft officially ended mainstream support of Windows 7 on January 14, 2020. But despite this, various software and hardware vendors would still release products that supported Windows 7 to a certain extent, including Microsoft themselves.

But I’ve run into a few devices already that no longer have Windows 7 drivers. I got an Intel AX210 WiFi adapter, which does not have drivers for Windows 7 at all, requiring me to install an older WiFi adapter to get internet access when I boot my machine into Windows 7.

My GeForce video card also hasn’t received mainstream driver updates for a while now. It only gets ‘security updates’ from time to time.

And when you install Visual Studio 2022, it also gives a warning that not everything will work correctly. Most notably, .NET 4.8.1 and .NET 7.0 are not supported on Windows 7. On the other hand, it’s somewhat surprising that Visual Studio 2022 installs and works on Windows 7 at all, even if some features are not available. Vista was nowhere near as lucky, and was cut off after Visual Studio 2010 already, which somewhat ironically meant that even though Vista supported .NET 4.5 out of the box, and could also support .NET 4.6, there was no Visual Studio environment you could run on Vista itself to develop for these versions of .NET.

Another sign is that Microsoft has released the last update of Edge Chromium for Windows 7. There is now a notification that you should upgrade to Windows 10 or newer. Google will follow suit with Chrome.

Anyway, it seems we’re in the final stages of Windows 7 support. Windows 8 and 8.1 have already bitten the dust, as have early versions of Windows 10. We have reached the point where you need a fairly up-to-date version of Windows 10 to get decent driver and application support.

Posted in Software news | Tagged , , , , , , , , , , | 2 Comments