The ultimate dev‐environment
A personal tale about experience, preferences and accidental discoveries.
[TL;DR? Skip to the Dream System section] In development, tools and scaffolding are key to both quality and productivity. In a cross development setting like this, the tools reside on the development host (and on github), while the scaffolding - the environment created to test and use the code - is either an emulator or a physical system. In some cases even several physical systems, as required to verify compatibility across target platform variants.
Emulators like QEMU or 86Box are convenient and fast, and certainly a preference when speed and timing don't matter or disturb testing. However, when working on OS, IO and network level code, timing often matters - some times to such a degree that a single printk or even kputchar changes or at least affects the behaviour. That's why I like - and prefer - to work on real hardware whenever possible.
My history with ELKS, later TLVC, started on a Compaq Portable III, a 286/12.5MHz system with a 40MB (original) disk, a 1.44M floppy and 6MB of extended memory. A similar 386/20MHz system was added later, equipped with a CF card as disk drive, 1.44M and 1.2M floppies and 10M extended memory. The extended memory part is important because it made a world of difference speedwise when @ghaerr introduced XMS buffer support in ELKS. The floppy-based 286 (the HD was (and is) unavailable, occupied by other systems) suddenly went from sluggish to champion with thousands of buffers.
My development systems are located in the garage because they are big and noisy. They're being accessed via serial lines - serial console and in most cases one or two additional serial ports. The additional port(s) become the work terminal, leaving the console, often cluttered with debug messages, as log screen more than anything else. This way the physical system gets tested (and pushed) in a different way than if just using the console and keyboard, beating up otherwise little used parts of the systems and provoking different bugs.
After ELKS networking and ktcp became stable, development speed increased dramatically. New versions of apps, kernels, whatever were ftp'd over in a snap, and using physical systems became close to as fast as using emulators. In particular if the system speed was decent. A simple ftp-command would get a new kernel across in a couple of seconds - for example:
ftp -u ftp://xx:[email protected]/linux target/linux
Or a full floppy images in a few more seconds. Adding to the convenience, the net command - which allows switching between different network interfaces in a snap - turned network development into a whole new ballgame: Test-switch-compare-download-test again etc.
The Compaq Portables had their limitations though. However convenient, they have only two ISA slots in the expansion box, a rather severe limitation - not the least because one of them was needed for additional serial lines.
That's where the 'dream system' comes in. Just like the Compaqs, it started almost accidentally. Browsing eBay looking for something else, I noticed this SBC, single board PC with an ISA16 edge-connector. What a neat idea. More searching and the perfect match popped up. A 40MHz 386SX with VGA, 4x serial lines, keyboard connector, AT-style IDE disk interface, floppy interface, 4MB memory on board and SIMM socket for another 16MB.
This board, a power supply and a 8 slot backplane and we have and extremely powerful and flexible system. A month later it was working - and has been my primary development system since. Very responsive, very flexible.
No enclosure, just the backplane on a piece of styrofoam on a shelf in the garage. A CF card with IDE adapter plugged into the HD connector, 3.5 and 5.25in floppies and a collection of additional ISA cards, typically 3 or more network cards. Serial lines were connected to a terminal server (a rarity these days, but I got a 4-port unit at a very decent price on eBay), making them available via the network.
Further, a small PC keyboard and equally small (5") VGA display for occasional use next to the system in the garage. And not the least, a remote controlled power switch so I can powercycle the system from my office if it hangs hard.
Here's the thing: Once the tooling is in place, the Dream System equals using QEMU in terms of both speed and convenience. It takes 5 more seconds or so to boot, but other than that, they come out about the same. For driver and kernel development it's unbeatable, not only because of the speed issues, but with an emulator you're always limited to the devices supported, not the ones you may need.
The list goes on - but the point has been made.
The 2nd Dream System is an XT architecture - and a (relatively) fast one. Technically it's a true XT clone although faster (8MHz and with a V20 rather than the original 8088 processor. With its 8bit ISA edge connector it fits into a passive backplane just like the AT+ Dream System above. Some CGA or VGA interface, a floppy interface and a disk and we have a great XT environment for testing and development.
Like its sibling above, even this system has a history. Here it is: After repairing, playing with and writing drivers for the IBM 5155 XT portable and the Compaq Portable Plus, I decided it was time for simplification. These monsters are charming and deliver impressive performance with their limited resources but they're still monsters. IOW - time for Dream System II.
Elijah Miller has been designing and making XT class boards for some time and I've been following his work on and off. Armed with intent to create the XT Dream System, I headed back to his site hoping to find what I needed. I didn't - and everything he had was out of stock - for good. I contacted him about availability - and luck would have it that he was designing a new XT board, more complete than the previous series - exactly what I needed. The new design was fresh from the board manufacturer and Elijah soldered up a working board for me, while I ordered the other required components: The ISA backplane, a VGA card, a floppy/serial card, and a disk controller, the latter not quite XT style but more conveniently a AT/IDE controller with XT support via BIOS.
Eventually all the pieces were in place and the system ready for testing. After a while, I had to give up on the IDE card above. Even though the card came unused, I was not able to make it work on the XT system, or even to recognize my CF-card based disk drives. So I caved in and ordered a XT-IDE card instead, which in turn initiated a rewrite of the directhd (IDE) driver in TLVC to deliver full XT-IDE support (more about that in the TLVC Developer Notes pt. I Wiki.
Eventually, the system was complete - with a KVM switch to share display and keyboard between the two systems, a modern day ne1k compatible NIC, a CF disk and floppies - 360k and 720k 3.5in. But - as became apparent when I started to use the system - there was still a piece missing: A real time clock. Entering the date and time on every reboot was just too time consuming (and admittedly, forgotten most of the time). And not having correct date and time on files modified locally was unacceptable. So I found this on the net - and ordered it via Tindie:
Simply an AT-compatible real time clock for XT system. A minor change to the TLVC kernel routine to look for and recognize the clock even on XT-class systems, and we were all set. Actually there was one more thing: The clock utility was set up to fail if it couldn't read the time off of the clock, assuming that it would always have been initialized by some other system (i.e. DOS). Not so in this case, so clock was updated to do first time initialization (see this PR ).
The clock module also has AT-compatible CMOS storage. This has not been put into operation (yet).
A final note on hardware: I did manage to get my hands on a true XT-generation Qume 360k 5.25in floppy drive for the Dream System II. The exact same drive as IBM used on the 5155, and an unused one, like the controller mentioned above. However, the floppy controller was AT-generation, so the hardware interfaces do not match. Not a big deal but some hardware mods are required, as per this schematic (the old style to the right:
A little bit of cutting and soldering on the Qume is all it takes, note that the Drive Select 1 jumper is in place:
Also note that this mod is not sufficient to use the drive as unit 0! Check the schematic above and add the missing mod to include that functionality (you'll figure it out).
So the Qume becomes floppy drive 1 (B). Drive 0 (A) is a modern day 3.5in drive which matches the controller's interface and doesn't need any mods. It runs as a 720k drive which makes much more convenient for practical (TLVC) use. 360k is really cramped for a root filesystem. This - using a 720k drive with an XT system - in turn requires more software mods: There is no CMOS config to tell the OS that this is not a 360k drive and probing to find out may or may not work. This is what the xtflpy= parameter in bootopts is for, in this case the setting would be xtflpy=3,1. Check the bootopts details in the Boot, Startup and Runtime Configuration Wiki.
Finally - the Dream System II has become a platform which can accommodate any hardware from that period and allow drivers to be created and tested. Surprisingly fast - and flexible.