PrintStar's Blog
Ramblings of a Fortran Nut
January 31, 2009 by Jeff

And I’m Done.

Well, that about wraps up RetroChallenge Winter Warm-Up! I achieved the two goals I originally set forth, so I can say that everything was a success. The products of this challenge are:

More importantly, I did learn a considerable amount over the course of the month. First, I finally learned the physical layout of disks, including the meanings of “sectors” and “tracks.” Second, I gained a good deal of knowledge about programming the Rainbow through BIOS calls. Third, I finally cracked the nut that is uIP; I understand a great deal more about how a TCP/IP stack works.

Great work to all the other participants this year!

  •   •   •   •   •
January 31, 2009 by Jeff

uIP/FOSSIL Version 1.0

Version 1.0 of uIP, the portable, tiny TCP/IP stack, is now available for MS-DOS supporting Serial Line Interface Protocol (SLIP) connections via a FOSSIL driver. The source code is available below along with a sample http server application. The source code is released under a modified BSD license.

The web server application, and any other apps compiled in the future using the stack, should run on any FOSSIL-compatible systems, including the IBM PC, DEC Rainbow 100, PCjr/Tandy 1000, Tandy 2000, and many others.

This port implements a SLIP device driver taken from the Contiki Operating System and modified to use the FOSSIL interface. Most uIP-specific code should not differ from the actual uIP distribution. All testing was performed using a GNU/Linux system as a gateway, although SLIP on other OSes should work as well.

The sample web server contains highly modified code to allow hosting files on disk.

  •   •   •   •   •
January 31, 2009 by Jeff

uIP Web Server Online!

The uIP web server on a DEC Rainbow 100 is now online! Check it out here:

http://uip.rainbow-100.com/

Beware of slow loading speeds! The uIP SLIP interface seems to be capable of about 400 Bytes/Second.

The actual source code and the web server executable will be posted shortly as well.

  •   •   •   •   •
January 31, 2009 by Jeff

One Day Remains

I have just a single day left to complete the RetroChallenge 2009 Winter Warm-Up. Luckily, I fixed the last remaining uIP issue with some luck. I had changed the some of the HTTP filesystem code early on in an attempt to build a standalone uIP library. However, uIP exhibits a strong interdependence on modules and application code.

The short explanation of my problems is that I never fixed the minor code change in the uIP code that was meant to support HTTP filesystem changes. Reseting a single #define in the code seemed to wash out the remaining multiple connection issue. Some minor 404 errors were still present, but they have been since fixed.

The tasks for tomorrow include bringing the actual Rainbow server online with a tiny selection of web pages, documenting the uIP port a little better (it could use a README), and packaging up uIP for distribution. Exciting stuff!

I put together a tiny, four-picture album of the equipment used in this challenge. The album can be viewed here. The tiny black computer is the decTOP server, and the array of 4 computers is where the uIP port was performed and tested. The two GNU/Linux desktops are both hand-me-downs, and the laptop is used for compiling uIP running OpenWatcom via Wine.

Please excuse any of the mess, as the equipment is all located in my unfinished basement…

  •   •   •   •   •
January 30, 2009 by Jeff

Fun with SLIP and the Linux Kernel

I yanked out a beautiful Rainbow 100B last night to act as a web server. I stripped the hardware down to the minimum for that authentic “unnecessarily obsolete” experience. The Rainbow I chose was in pretty good condition, with a little bit of dust buildup on the motherboard. To make sure everything was okay, I pulled the motherboard assembly out (a trivial task requiring no tools on a Rainbow), removed all her daughterboards, and cleaned off the caked-on filth. This particular Rainbow had a fully populated memory expansion board, bringing system memory up to 896KB, a graphics option, and a hard disk controller. I decided not to replace the graphics option and hard disk controller while I reassembled the system as they were unnecessary for this application. After reassembly, I ended up with a Rainbow 100B with 896KB RAM and a single RX50 floppy drive unit.

I set up the box on the table that holds my decTOP box, replacing the giant HP Visualize workstation. For a monitor, I chose to use a VR241, which is the color monitor for Rainbows. The VR241 is giant and bulky, comparable to the monitors IBM shipped with the XT and AT computers. Unlike the Rainbow’s quaint VR201 monochrome monitors, however, the VR241 is independently powered and can be switched off (VR201 monochrome monitors are powered by a 12V power line in the monitor cable). The whole setup looks a little silly, but I’ll post a picture tonight or tomorrow.

The Rainbow is connected to the decTOP server using the USB-to-Serial adapter and a null modem adapter (and a selection of cables and gender changers). Once everything was plugged in, I turned on the Rainbow and fired up the SLIP connection. I’m booting the Rainbow off an MS-DOS 3.10b boot floppy created with my imaging program, the other half of my RetroChallenge. The disk in drive B hosts the http server and all its web pages.

The first attempt at creating a SLIP connection failed for reasons I’m not quite sure about. The usual way of creating the SLIP connection on GNU/Linux is to:

  1. Run slattach -p slip -s 19200 -L /dev/ttyUSB0 as a background process
  2. Use ifconfig to bring up the sl0 network

Usually, to kill the SLIP connection, the slattach process needs to be killed. This method works like a dream on SLAX where I’ve been testing this, but on Debian Etch, this process doesn’t quite work. The only way to bring down slattach is to issue a nasty SIGKILL signal, which simply destroys the process. Apparently using SIGKILL does not seem to allow slattach to shutdown elegantly because it never starts up again successfully afterwards.

After trying to get it to reopen the SLIP connection, I finally had to reboot the GNU/Linux box. I was truly dissapointed because the uptime was at 138 days. The problem, of course, is that SLIP lives in the kernel. The unsuccessful attempt to restart slattach, at least in Debian, meant that the whole kernel needed to be restarted. Sigh… Rebooting a 400MHz Geode-based system is not a quick process either, especially when it is attempting to run Apache 2.2, a MySQL daemon, and a Postgres daemon (don’t ask…).

After about four reboots and some cable switching, I finally found a nice configuration where the SLIP connection was up and running in a reliable manner. I made the necessary changes to my Apache configuraiton to pass all requests for a given server (I’ll tell everyone soon!) onto the Rainbow via the SLIP connection.

Testing revealed two bugs in the current web server, one easy to fix, one rather difficult. The first problem was that uIP wasn’t marking files ending in .htm as HTML files. Requesting the same file as .html works fine thanks to an undocumented Watcom function, _lfntosfn, which converts long filenames to short filenames. The change was a quick addition to an if statement to ensure that the three-letter extension was also recognized as valid HTML.

The second issue seems to be that uIP is not dealing with multiple connections correctly. I observed similar behavior when working with version 0.6 years back, but version 1.0 should not be experiencing these problems. For example, if a web page contains two images, most browsers will request both at the same time. What currently happens in my implementation of uIP is that the first request received is honored, while the second is treated to a fine sampling of garbage data. I’m still trying to figure out how the problem is occuring as I don’t see any points in the code where they should be stepping on each other. Under a worst-case scenario, I could limit uIP to only 1 connection at a time, which would also fix the problem.

Hopefully, a completed version will be available tomorrow!

  •   •   •   •   •
January 29, 2009 by Jeff

Finishing Up

The end of Janurary is almost here, and it brings the close of the RetroChallenge 2009 Winter Warm-Up. Its been a few days, but things have been relatively quite on the retrocomputing front. In my last post, I said I was going to work on porting the example telnet client application shipped with uIP version 1.0. Upon examining the uIP version 1.0 distribution, I was surprised to learn that it did not ship with a telnet client example. Writing one from scratch seemed to be far beyond the scope of this challenge. I did look at the specifications for telnet, namely RFC 854, but decided that a from-scratch version would probably be severely lacking if I attempted it in a matter of days. For now, my uIP port will simply focus on the web server. The important code, interfacing the FOSSIL driver and providing a SLIP device, is present and working for others (or myself) to expand on in the future.

My development and testing thus far has focused on using a pair of giant desktop machines to provide SLIP servers. The Rainbow would download its compiled (via Open Watcom) web server from my trusty Debian box, and, using a serial switch, I’d quickly test the SLIP connection with an overpowered SLAX LiveCD box running an Athlon64 2800+ with a gig of RAM (it’s currently waiting on a HD replacement for after the challenge). However, I really want to connect it to my decTOP server, which is the only computer in my house currently running 24/7. The decTOP (previously the AMD PIC) is a tiny Geode-powered computer with a 10GB hard drive and 512MB RAM (in mine at least…) that consumes only about 10W of power. It had been purchased as my main web server to replace an HP Visualize B2000 that had been hosting my startup’s website on Debian GNU/Linux, which consumed somewhere in the neighborhood of 500W. Alas, the Geode is an archaic design, and I decided to move all web hosting for Approximatrix to a Virtual Private Server for hosting. The Geode just couldn’t handle Zope/Plone, an unbelievably processor-intensive python-based content management system.

The decTOP does, however, host this site! And because it is always on, I wanted to route my Rainbow through the decTOP. However, the decTOP does not have a serial port, only USB 1.1 ports. My friend let me borrow a USB-to-Serial adapter made by IOGear to try SLIP connections. Suprisingly, Debian immediately recognized the adpater when it was plugged into the decTOP, and the Rainbow had no problem talking with it.

My plan is to set up a Rainbow to host actual web pages for the next few weeks. The uIP connection is painfully slow, clocking in at around 300-900 bytes/second. With the USB-to-serial connection, I’ll just need to add a quick proxy line to the Apache configuration on the decTOP to route one subdomain to the Rainbow’s SLIP connection.

My plans last night were foiled by a dead hub. The decTOP (whose name is luna, by the way) was not visible on the home LAN at all. In fact, this blog may have been inaccessible yesterday for a period. After some frantic troubleshooting, it turned out that the hub next to luna had failed somehow. While the problem wasn’t a showstopper, it did consume a good amount of time.

Tonight I plan to set up yet another Rainbow to act as the web server. For the retro appeal, it’ll be using only floppy disks for serving pages. It should be fun to here the RX50 drives growl every time someone connects to the pages!

If you’ve never heard an RX50 floppy drive, you’re really missing out. They provide a very satisfying set of noises whenever they’re running.

  •   •   •   •   •
January 26, 2009 by Jeff

A Functional Web Server

I believe in my last post I set forth some goals before closing out this RetroChallenge. Well, I think I’ll be ignoring them for now. There were some other serious problems with my uIP HTTP server code that needed some work. Specifically, I “fixed” the file system routines. Originally, in order to create a drop-in replacement for the uIP HTTP code, I created a rouitne that would load files from disk into memory completely before sending them along through the uIP stack. The original uIP code actually contains all its web pages compiled into the executable, which is a bit silly for a more-or-less resource-rich machine like the Rainbow.

The big downside of loading the entire file into RAM prior to sending it is the memory limits. I have been compiling the web server (and all the uIP code) using a medium memory model. This model in Open Watcom provides 32KB of addressable memory (I believe) to work with. This restriction means that the largest file the web server would be able to send would be 32KB less the various program memory in use. Now I could compile using a large or huge data model, but suddenly pointers become far pointers, and compilers (like Microsoft C) become flaky. Besides, the buffering of files into memory first is completely unnecessary.

I spent a few hours (while watching television with my wife…) changing the filesystem code to read in an IP packet’s worth of file information at a time when it was ready to send. The changes were somewhat complicated, and I was working blindly from a Fedora-powered laptop. I finally got my changes to compile, but had to wait until the next morning to try them out on the actual Rainbow.

Much to my surprise Sunday morning, the changes were completely successful. I transfered the server executable to the ‘bow and set up the SLIP connection. Surprisingly, requesting a large (172KB binary) file worked perfectly, and the server never crashed or ran out of RAM. I now feel quite confident about its stability.

I think to finish off the contest, I’m going to attempt to get a telnet client running. uIP ships with sample telnet client code, so I’ll have to see if I can wrangle it into a workable form. Hopefully I should have another surprise before the week is out as well…

  •   •   •   •   •
January 23, 2009 by Jeff

Friday Evening Success

This morning while glancing over the uIP version 1.0 example UNIX implementation, I noticed some tidbits related to an issue I was having. My SLIP implementation was base on the example tap driver implementation, which implemented ARP. I started comparing my code using uIP version 1.0 to the older uIP version 0.6 distribution I had created ages ago. The biggest difference that immediately jumped out was that there was no ARP code present. I’m familar with ARP on a very high level from attempting to give my Atari TT030 network access via a PPP link. My understanding is that it allows a point-to-point connection to access and become visible to the entire LAN.

Anyway, last night I spent a long while trying to understand why my uIP implementation (compiled in Open Watcom), was not responding to a ping. After some fuss, I was able to determine that the Rainbow was indeed receiving serial information. Whats more, the amount of data was exactly correct (84 bytes for a ping), but the uIP stack summarily dismissed the data as useless.

The last debugging line I added printed the data packet’s type. In uIP incoming/outgoing data is held in a buffer structure. One element of the structure is type, which can specify UIP_ETHTYPE_IP or UIP_ETHTYPE_ARP. The type is magically set by the black box that is uip.c (the real meat of the uIP stack). I added yet another debug line to print out the buffer type, and it turned out to be zero, which isn’t equivalent to either of the packet types above. It occured to me that I really don’t care about ARP at this time (I really just want to see a point-to-point connection work), so why would I need to check the packet type? Furthermore, the fact that the SLIP driver was receiving exactly the right amount of data on a ping suggested that the packet being transfered was a-ok.

This evening when I had a chance, I stripped out the code related to ARP. Specifically, I removed the packet type checking (which was not present in the old uIP version 0.6 implementation) and just assumed any incoming data in the buffer was ready for processing by the uIP stack. This assumption should have been obvious as the SLIP driver doesn’t show any sign of dealing with ARP or different packet types.

Once the code was updated, the resulting web server executable still didn’t work. Oops! After fixing a little IP address problem I tried again. Miraculously, the Rainbow responded to a ping! Furthermore, requesting a file via http worked as well! Hooray!

Much still remains to be done, however. First, I need to clean up the display interface. Right now the debugging output is too verbose and sometimes completely incorrect. I’m also considering implementing the interface using a Video FOSSIL driver (can’t find a good link…) so that a full-screen interface might be possible.

As an aside, Video FOSSIL drivers are meant to provide a similar functionality to communication FOSSIL drivers, only instead of providing an API for accessing serial ports, they provide an API for accessing the screen in character mode. The Rainbow, of course, has a Video FOSSIL available since its video access routines are substantially different than IBM PC routines. The functions are, if I’m remembering correctly, based on the OS/2 character addressing routines. Most of the FidoNET BBS packages supported Video FOSSILs, including Opus CBCS and BinkleyTerm.

  •   •   •   •   •
January 22, 2009 by Jeff

uIP Loves the Watcom and Hates the Turbo

I haven’t had a post since I finally released the final version of the Rainbow Disk Image Utilities. The utilities, while small and rather straightforward, were the product of a good amount of research, especially disk skewing and disk geometry in general (I know what a track is now!). I think the utilities could probably stand alone as my completed RetroChallenge, but what’s the fun in that?

Over the last few days I’ve been dealing with the uIP version 1.0 source code. The last time I dealt with uIP, I used the source code for version 0.6, which has since disappeared from the uIP home page. Version 0.6 was substantially simpler, but was encumbered by an original BSD license. Version 0.9 was relicensed, however, into a new BSD license, making it a GPL-compatible software product. For those of you unfamilar with the perils of the original BSD license, check out the Wikipedia article. In summary, the original BSD license, as used in version 0.6, required an “advertising clause,” stating that the program contains software by Adam Dunkels. I realize it may seem like nitpicking, but I’m a bit of a free software junkie and the distinction is important to many people. Also uIP version 1.0 is a bit more “supported,” and there is something to be said for using the latest version of a package.

The first problem I encountered in the uIP source code was that version 1.0 no longer shipped with a SLIP (Serial Line Interface Protocol) device driver. SLIP is a protocol for IP over serial connection, similar to PPP. SLIP was popular in the early days of internet dialup, but PPP has eclipsed it for a variety of reasons. In fact, Windows hasn’t shipped with a SLIP driver since Windows 98 (I believe). Finding a GNU/Linux distribution supporting it can also be a task. To save some trouble for those interested, Debian x86 (both Sarge and Etch) supports it. I’ve also had luck with SLAX version 5.

I’ve decided my goal should be parity with my version 0.6 port. I’m going to attempt to get the simple web server running on the Rainbow and serve pages to a modern GNU/Linux box. I started by pulling out the source code I needed from the substantial amount in the uIP distribution. In particular, I dropped all apps other than the web server (uIP also provides a web client, a telnet server, dhcp, and a resolver) and I removed code referring to the unix port. Next I needed to build some custom makefiles for Turbo C just to check if I could compile everything; this is where the fun began.

uIP version 1.0 is pretty heavy on the directives. The code contains a ton of the #if and #define pragmas. Also, uIP, along with its application, is meant to be built in a monolithic manner. The complex interdependency of header files does not allow for constructing a uIP library per se. Finally, the application code was incomplete. The task of writing the actual int main() routine is left to the programmer.

First off I attempted to build the core uIP files using Turbo C. This task failed miserably on Turbo C, even with the proper makefile. The problem turned out to be that Turbo C was not particularly fond of UNIX line breaks; it only accepts lines ending in the full \
\
MSDOS format. Turbo C was also having a bit of trouble coping with the enormous amount of pragmas in the source code. For these reasons, I decided to do development on Open Watcom instead.

Before anyone calls shenanigans, let me just say that Open Watcom definitely appears retro. The IDE that ships with OW reeks of Windows 3.1, and the toolbar icons are just horrific. Of course, I never said I was also doing the endurance challenge, so I feel good about using Open Watcom on a modern box. Besides, OW requires a 386 or higher to do anything, even from the command line. The beauty of OW is that it allows the programmer to target a variety of operating systems, including 16-bit DOS. Also the uIP source code is complex enough that even the simple OW IDE makes working with uIP infinitely easier.

So far I have managed to do the following with uIP:

  • Write a FOSSIL SLIP device driver based on the device driver in Contiki and my previous 0.6 port
  • Write a driver program that should initialize everything a serve web pages
  • Rip out the uIP server code that served in-memory web pages and replace it with code to serve pages from disk
  • Compile everything in Open Watcom

Compiling everything under Open Watcom was quite a task considering the complexity of the code. uIP is difficult to work with because a) many portions are just a “black box” to me and b) it suffers from “someone else wrote this” (it doesn’t match my style of programming).

Last night I decided that, since everything was compiling in Open Watcom, it was time to update the makefiles to build everything under Turbo C. I really wanted to try to build it on the Rainbow itself. After working on the makefiles for a while, I finally had everything ready to compile. I tested them under DOSBox, and, after a lot of incremental fixes, it finally proceeded to the final link step. This point is where Turbo C showed its age.

I prepackaged all the object files into a library in the makefile prior to linking. The reason for the library was to avoid a command line overrun due to the large number of object files (ok, maybe there are only 15 or so…). The linking, however, complained about 5 or so undefined symbols, mostly originating from slipdev.c and httpd.c. I figured that I must have neglected to include the object files, but the makefile clearly showed they had been included. I printed out a listing of the library modules. Lo and behold, the two object files were present, but contained no public modules.

I had run into this problem before on the previous RetroChallenge. Occasionally, Turbo C, during compile, decides that the functions in a file don’t actually need to be compiled and simply ignores them. I have no other explanation for the behavior. Turbo C is not exactly the most robust compiler in the world.

I may have a look at the problem for a bit longer, but I have two other choices. First, I could try compiling everything using Microsoft C, which will run the Rainbow fine. Microsoft C, however, comes with the most braindead Make.exe I have ever observed, however. The problem can be overcome by using Borland Make.exe instead, but the library lines in the makefile will need to be replaced as well. Second, I could continue to use only Open Watcom. This option makes me nervous, however, as I’m not positive that Open Watcom is generating strict 8086/8088 machine code. I think it might be lying…

Hopefully, though, everything gets up and running prior to the end of the month. If so, I plan to have a real Rainbow serving up pages onto the net for everyone to try out.

  •   •   •   •   •
January 19, 2009 by Jeff

Rainbow Disk Image Utilities

The first half of RetroChallenge Winter Warm-Up 2009 has resulted in the Rainbow Disk Image Utilities. These utilities can be used to generate disk images from RX50 floppy disks on a Rainbow 100 and write images back to disks. The image utilities are compatible with the RAW image format, a popular standard in GNU/Linux and *BSD (via the dd utility) and on MS-Windows/DOS (via the Rawrite utilities). While the programs were tested with Rainbow-formatted floppies, the utilities should work with any disks or images conforming to the RX50′s 80 tracks – 10 sectors/track – 512 bytes/sector format. The archive containing version 0.1 is located at:

The utilities are, coincidentally, also compatible with the output of the wTeledisk utility by Will Kranz. The utility allows the conversion of Teledisk disk images (like the ones found on the Update archives) to RAW image format. The combination of the Rainbow Disk Image Utilities and wTeledisk can be used to recreate boot floppies from any of the Rainbow disk images on Update on an actual Rainbow 100 running MSDOS 2.11 or higher. The utilities were tested and shown to work with both MSDOS and CP/M-86/80 boot disk images. The wTeledisk utilities can also be run on the Rainbow for purists.

Share and enjoy!

  •   •   •   •   •