Fun with Compilers

I believe in my last post I claimed I would be working to create a simple window within GEM.  Things did not go quite so smoothly, however.  My first attempt involved transcribing a sample clock program on my Atari, but removing most of the “clock” portions for simplicity.  My second attempt, as I will explain, involved simply compiling a GEM sample program as-is under DOS.  I found most of my problems were related to the tools I was using.

Starting with the Atari example, I transcribed the code for a clock desk accessory, stripping out some of the complicated bits to leave me with a simple window.  GEM is driven via an event-based loop, much like Microsoft Windows programs.  On the face, the GEM model seems to be substantially more powerful than the Microsoft framework, but I’m starting to see the severe weaknesses in the GEM model.  First, dialogs are a nightmare under GEM.  While both GEM and Windows have to contain manual redraw code for windows, GEM requires this for dialogs as well  Furthermore, GEM has a nasty issue where the mouse must be disabled by the program during redraw (what’s that about?) or an actual cursor might be left behind as the mouse moves away after the redraw.  Second, GEM has a rather complicated redraw system involving passing “dirty” rectangles to any window involved in a redraw.  The program itself is responsible for attempting to draw only within the rectangles.  Third, GEM has very few builtin controls.  Basically GEM supports some text boxes and buttons, which can be morphed into check boxes and radio buttons.  The lack of controls is quite nasty, and dealing with them in a dialog is a bit of a nightmare, involving checking the memory where the dialog lives for values rather than function calls.

One great thing about GEM is that all dialogs are designed using the Resource Construction Set, or RCS.  RCS actually differs considerably between Atari and PC platforms, and, again, multiple 3rd party alternatives are available on the Atari (Interface, Resource Master, etc.).  A screenshot of Digital Research’s RCS on the PC appears below:

While being able (almost required, actually) to graphically design dialogs is a big plus, one can see from the screenshots that there are a severe lack of options available for dialog controls.

Now I’ll move on to compilers.  I wrote a simple application on the Atari using some sample code and attempted to compile the package with AHCC, a relatively new compiler suite based on Sozobon/X C.  I received piles of errors from the compiler, most focussing on some globals that had been declared as WORDs.  Some searching of header files revealed that AHCC did not seem to be defining a WORD in the header files.  I defined WORD as an unsigned int and moved on.  However, now I was receiving cryptic errors involving some functions that were declared as returning BOOLEAN values.  I normally avoid anything resembling booleans in C because their implementation seems quite inconsistent.  In this case the example I was basing my program on used them, so I did as well.  At this point I was quite frustrated, so I moved on to Pure C.

The Pure C compiler was the standard for years on the Atari, only recently being phased out by most surviving projects in favor of GCC.  Pure C is an odd package, in my opinion.  It is clearly based on Turbo C, as you can often find Borland copyrights sprinkled in the headers.  The system also provides BGI headers, which, I believe, is the ol’ Borland graphics interface.  It seems to have been developed at least partially by Gribnif Software, an old Atari company, but was marketed by ASH, another company.  Because the Pure C package is based on Turbo C, the GEM headers are much closer to Turbo C GEM bindings on the PC.

Anyway, I attempted to compile my program again using Pure C and was met with more problems on the declarations using BOOLEAN return values.  At this point I was convinced that the issue is the use of booleans and changed everything to integer return values, as I usually do in my programming.  Suddenly the program compiled and linked successfully!  I first needed to change the extension of the resulting .prg file (GEM apps on Ataris have .app or .prg extensions) to .acc to mark it as a desk accessory.  Because I’m running MiNT, I can double click desk accessories to start them rather than require a reboot with the .acc file on the boot drive.  Success!  Well, sort of…  A window of basically zero width and zero height was created, although some of the window decorations were drawn.  This horrible example proves that I’m at least to the point of being able to compile GEM programs on the Atari.

Next came the task of compiling this program on the PC.  Things became ugly very quickly.  There is a severe lack of C bindings for GEM on the PC.  My personal theory is that its a product of a closed environment from years previous.  In other words, since GEM and its programmer’s toolkit cost money, they were not generally available.  Microsoft’s nearly completely open (at this point, at least) availability of development tools is and was probably a major contributor to its popularity.  A GEM SDK is “available” from Shaneland, but the archive contains bindings for DJGPP and Pacific C, neither of which I’m keen on using.  For DOS development, I always prefer OpenWatcom, but I’ll deal with Turbo C as well (see my earlier challenges…).

I finally ran across a forum post that had links to the original GEM Programmer’s Toolkit from DRI, which contains language bindings for Turbo C, Microsoft C, and High C.  However, more fun came along with this kit.  The toolkit contains a GEM installer application that apparently assumes everything is on A: with no exceptions.  I finally realized that I could force DosBox to use a directory as A: as well.  I started GEM and selected Turbo C as my desired language.

The next step wasn’t quite so obvious.  I next needed to compile the libraries using Turbo C itself.  A handy batch file was provided, MAKELIB.BAT, that did produce a lib, but the lib contained nothing.  Looking at the documentation, I needed to make a special makefile “include” called BUILTINS.MAK which contained directions for actually compiling.  After creating the proper file and copying the file into every binding directory (AES, VDI, EMS for memory, XFM for file access, DOS for DOS interfacing, and MISC), the library successfully built.

Finally I compiled a simple hello world application.  Much to my surprise, it worked:

My next step is to fix up the code I personally wrote to get a non-zero sized window, and attempt to compile it on both the Atari and the PC.

RetroChallenge Begins!

RetroChallenge quietly began today, and my challenge got off to slow but productive start.  As my RetroChallenge entry says, I’ll be tackling some programming using GEM, an alternative windowing system originally developed by Digital Research.  GEM enjoyed moderate success in the PC world in various incarnations, but is probably most famous for being the operating system (along with GEMDOS) on the Atari ST line of computers.  A great rundown on how Atari came to choose and port GEM to their rapidly developed hardware is available from DadHacker, an Atari developer at the time.  While PC and Atari GEM are theoretically compatible, there are a few show-stopping differences in porting software to each.

The most obvious issue is the CPU differences.  The Motorola 68k series found in Atari machines is big endian, while Intel and compatible chips in PCs are little endian.  While not an obviously large difference, it does make the use of macros in C a necessity for porting software.  However, the macros themselves are problematic as compilers diverged from each other on the two systems.  Little, tiny differences, which I’ll outline in later posts, appear in different compiler suites.  Furthermore, header files are not named similarly or present at all on one or the other.  The other nastiness related to system architecture is the inherent ugliness of far pointers and the like on Intel systems.  The Atari does suffer somewhat from a two-stage memory system (ST and TT RAM addressed using 16 and 32 bits respectively), but the Atari’s 14MB of addressable ST RAM is accessible without any trickery, unlike Intel architectures, which require EMS/XMS/whatever else you can imagine.  Yuck.

The other major difference is the differences in GEM itself.  It appears that Atari licensed the GEM source code and then never spoke to DRI again.  PC GEM was basically dead by 1990 as Microsoft Windows was clearly becoming the winner of the GUI wars on the PC.  DRI more-or-less abandoned GEM as a standalone environment, relegating it to a shell in DR-DOS called ViewMAX.  Furthermore, Apple sued the pants off DRI for GEM and its functionality, severely crippling its usefulness on the PC.

Atari GEM, on the other hand, continues to be developed (although the pace has slowed considerably).  GEM is partially derived from GSX, the Graphical System Extension, and retains the modularity of the older system.  In fact components of GEM can be swapped out with ease, replaced with significantly better software.  Atari itself released its last version of TOS, which is stands for “The Operating System” and is comprised of GEM, GEMDOS, the BIOS, and the XBIOS, in 1993 with the release of the Falcon030.  However, TOS itself continued to be developed, released again in the late 1990s with the Milan computer, a 68040 or 68060 Atari clone.

Various additions and alternative components have been released both commercially and as open-source software.  For example, my Atari runs FreeMiNT, which replaces much of the GEMDOS and provides the ability to use preemptive multitasking and modern file systems.  On top of FreeMiNT I run a replacement open-source Video Device Interface (VDI), commercial Graphic Device Operating System (GDOS), a commercial Application Environment Services (AES), and a commercial desktop.  The gist of this description is that Atari GEM is considerably more advanced than PC GEM.

As an example, lets take a look at two screenshots.  This first image is a capture from ShaneLand OpenGEM, the current state-of-the-art PC-GEM implementation:

Note that the screenshot shows monochrome icons and a relatively Win 3.1-like feel.  The system pallete is 16 colors at 640×480.  Contrasting OpenGEM is a screenshot from my current Atari TT030:

The screenshot above was taken at a resolution of 1024×768 in 16-bit color.  The icons are all 256 colors, and the window decorations are decidedly mroe attractive.

Tomorrow I’ll try to start postin about the wonders of GEM programming once I understand a little bit more about it.  From my initial glances, GEM seems a lot like Win32 C programming, relying on messages processed in event loops.  I plan to attempt my first “application” tomorrow.

Upgrading the Atari TT030

I’ve decided to use my Atari TT030 in this year’s RetroChallenge, which begins in July.  Lately I’ve had a love-hate relationship with the venerable workstation mostly due to its age and wackiness.  I tend to get a little nostalgic for the computer, drag it out, set it up, become highly frustrated, and throw it back into storage.  So this time I’m trying to get my mindset right for using it.

My Atari has seen some serious upgrades over the base TT030 specs.  Notably, the system features 64MB of TT-RAM (meaning its 32-bit addressable only) and a Galaxy VME graphics card, a relatively recent hardware design.  The Galaxy card was designed by Mario Becroft in the early 2000s and is capable of resolutions up to 1280×1024 at 65k colors.  The card itself is a pretty neat design as it uses a FPGA that is configured at boot.  The later models feature ethernet as well, although mine is an early prototype (serial #3, I believe) without the ethernet chip.

Anyway, I’ve always used the FreeMiNT operating system on my Atari.  I’ll talk more about that operating system as the challenge proceeds, but the most up-to-date distribution of the FreeMiNT operating system these days is called EasyMiNT, which purports to have a simple installation procedure.

I did have a 2GB SCSI hard disk in the TT up until a few days ago.  The hard disk partitioning scheme I had chosen did not appeal to me, however.  Also, I seem to have a plethora of 10K rpm 9.1GB SCSI-3 hard disks for some reason, and I thought one of these drives might be much more appropriate in the Atari.  I fought with some paritioning software for a while and finally got the Atari to boot from the hard drive (not strictly necessary on Atari’s as the OS is in ROM).  I set up to simple 512MB partitions that the Atari can see without any additional software, and partitioned the balance into 2GB partitions.  To see beyond 512MB, the TT needs to have an OS installed that is capable of addressing these partitions, which is where FreeMiNT comes in.

Installing EasyMiNT is relatively simple.  I loaded the distribution onto a CD (I believe it measures in at around 128 MB or so with all components), configured the Atari MetaDOS CD driver, and got started with the install.  EasyMiNT needs MiNT to boot in order to run the necessary tools to complete the install, so the installer modifies the boot folder to start MiNT to allow the installer on reboot to complete the installation.  Atari TOS (which stands for many things, nobody is really sure…) looks for a folder called AUTO on boot.  If the folder is present, it proceeds to execute the programs in the folder in the order they were placed there.  This design seems simplistic, but makes installing drivers on an Atari a snap.  EasyMiNT simply modifies the AUTO folder to make sure MiNT is started and the installer can proceed.

Things went south at this point.  When the computer rebooted, MiNT was started as expected.  However, MiNT attempts to start XaAES, an alternative AES (AES is the Application Environment Services, which is basicly the graphical interface similar to a window manager in the GNU/Linux world).  However, XaAES failed on the TT; the machine simply halted.

Instead of trying to figure out why XaAES wasn’t working, I decided to go with a commercial alternative, N.AES.  N.AES is a commercial AES designed to work with MiNT perfectly, and it is my AES of choice.  I believe I purchased it new in the early 2000s as well.  I editted EasyMiNT’s configuration file MINT.CNF to start N.AES (which I had already installed) rather than XaAES and rebooted.  Tada!  The EasyMiNT program was able to continue the installation.

Trying to troubleshoot XaAES, or anything on an Atari, is more difficult than ever.  The community surrounding the Atari 16/32-bit line of computers is always dwindling, and a lot of the remaining people have wildly differing hardware setups.  I myself basicly disappeared from the Atari stuff around 2002 when I just couldn’t take the lack of interest in the platform anymore.  Hopefully the RetroChallenge will convince me that I should keep the ol’ TT030 set up.

My upgrade to a 9.1GB harddrive was a success.  The system now boots into MiNT 1.16.3 Alpha with N.AES on top running the Thing desktop.  I still have the following to do before the RetroChallenge starts:

  • Install the Galaxy VME graphics card drivers
  • Install the Jinnee desktop
  • Possibly install NVDI for access to fonts
  • Setup the new AHCC development suite
  • Transfer all my junk off the original 2GB boot drive

As the RetroChallenge proceeds, I plan to explain a lot more of the terms I’ve used above, like what are the VDI, AES, and GEMDOS components of the Atari TOS operating system.  Also I plan to grab some nice Atari screenshots along with pictures of the actual hardware.

A New Rainbow Home Page

It has been slow in coming online, but my new Rainbow 100 home page is now online.  The site’s name is Drive W, refering to “Start from Drive W” on the Rainbow 100B’s boot menu.  Of course everyone knows that Drive W refers to the Winchester hard disk…

The site took a good amount of time to create, but much less than I originally thought it would.  My first plan was to build a site based on Ruby on Rails as it is often heralded for being easy to develop with and I know some Ruby.  However, I was not once ever able to get Rails to start, let alone work on any of my systems.  While I could have worked to debug the problem, it truly wasn’t worth it to me.

For my next attempt, I had a look at Django, a python framework similar to Rails.  I would say that I’m far more adept at python programming, so it seemed to be a great alternative.  The main goal was to create an online, indexed file archive of Rainbow programs, complete with descriptions and categories.  Django proved to be simple, yet powerful.  Furthermore, it worked on every system I had installed it on.

So the website is now up, and it actually contains more features than I originally envisioned.  While the archives section is working flawlessly, I’ve also added news items and the entire Rainbow 100 Frequently Asked Questions.  The only thing I’m not particularly pleased with is the CSS and page layout.  I’m currently using a very basic CSS set that I authored years ago as I can’t stand trying to get CSS to work.  I would say that the site is presentable, but it could be much better.

Have a look if you’ve got some time!

In other news, I completed yet another PyWeek and I’ve started playing Legends of Zork, an online browser-based game.  My PyWeek entry did not place particularly well, but there were quite a few positive comments in the ratings section.  Most people had complained about the horribly drawn graphics that were used in the game.  Everything was pretty ugly with the exception of the alien enemies, which I managed to render and animate in Blender.  One week just isn’t enough time!

Legends of Zork is a fun, casual game I just started playing two days ago.  It seems to be rather tightly linked to the Zork world, although it is not necessarily building on the previous games (and it shouldn’t…).  I highly recommend it if you’ve played the old Zork games before.  Best of all, its free!

RetroChallenge Winners Announced!

The fine judges of RetroChallenge have announced the winners of this years Winter Warm-Up! Congratulations to the winners and all the participants! I think the right people won the contest, and everyone had a wonderful time (at least as far as I know).

I have to admit that my personal favorite this time around was urbancamo’s entry.  The difficulty of attempting to use a VAX fulltime was pretty amusing.  I actually leaned a great deal about the ol’ DEC software from reading his retro-created blog.  Wgoodf’s controller was an absolute riot, and the hardware challenge always amaze me.  I didn’t follow billdeg’s challenge too closely, I have to admit, but very impressive in the end!

I’m especially happy with what I was able to generate over the course of the challenge.  There seems to be some authentic interest in both my disk imaging software and the uIP port over at the Vintage Computer Forums.  Most importantly, I’m finally able to write Teledisk images in a reliable manner again!

I think for next challenge I’ll need to change things up a bit.  Specifically, I should really move on to another retro computer just for the sake of learning.  I was considering possibly pulling out my souped-up Atari TT030 for some possible programming.  Maybe I could play around with some of my 8-bit Atari and Apple systems instead.  We’ll have to see; I have plenty of time to think it over…

And I’m Done.

Well, that about wraps up RetroChallenge Winter Warm-Up!  I achieved the two goals I originally set forth, so I can say that everything was a success.  The products of this challenge are:

More importantly, I did learn a considerable amount over the course of the month.  First, I finally learned the physical layout of disks, including the meanings of “sectors” and “tracks.”  Second, I gained a good deal of knowledge about programming the Rainbow through BIOS calls.  Third, I finally cracked the nut that is uIP; I understand a great deal more about how a TCP/IP stack works.

Great work to all the other participants this year!

uIP/FOSSIL Version 1.0

Version 1.0 of uIP, the portable, tiny TCP/IP stack, is now available for MS-DOS supporting Serial Line Interface Protocol (SLIP) connections via a FOSSIL driver.  The source code is available below along with a sample http server application.  The source code is released under a modified BSD license.

The web server application, and any other apps compiled in the future using the stack, should run on any FOSSIL-compatible systems, including the IBM PC, DEC Rainbow 100, PCjr/Tandy 1000, Tandy 2000, and many others.

This port implements a SLIP device driver taken from the Contiki Operating System and modified to use the FOSSIL interface.  Most uIP-specific code should not differ from the actual uIP distribution.  All testing was performed using a GNU/Linux system as a gateway, although SLIP on other OSes should work as well.

The sample web server contains highly modified code to allow hosting files on disk.

uIP Web Server Online!

The uIP web server on a DEC Rainbow 100 is now online!  Check it out here:

http://uip.rainbow-100.com/

Beware of slow loading speeds!  The uIP SLIP interface seems to be capable of about 400 Bytes/Second.

The actual source code and the web server executable will be posted shortly as well.

One Day Remains

I have just a single day left to complete the RetroChallenge 2009 Winter Warm-Up.  Luckily, I fixed the last remaining uIP issue with some luck.  I had changed the some of the HTTP filesystem code early on in an attempt to build a standalone uIP library.  However, uIP exhibits a strong interdependence on modules and application code.

The short explanation of my problems is that I never fixed the minor code change in the uIP code that was meant to support HTTP filesystem changes.  Reseting a single #define in the code seemed to wash out the remaining multiple connection issue.  Some minor 404 errors were still present, but they have been since fixed.

The tasks for tomorrow include bringing the actual Rainbow server online with a tiny selection of web pages, documenting the uIP port a little better (it could use a README), and packaging up uIP for distribution.  Exciting stuff!

I put together a tiny, four-picture album of the equipment used in this challenge.  The album can be viewed here.  The tiny black computer is the decTOP server, and the array of 4 computers is where the uIP port was performed and tested.  The two GNU/Linux desktops are both hand-me-downs, and the laptop is used for compiling uIP running OpenWatcom via Wine.

Please excuse any of the mess, as the equipment is all located in my unfinished basement…

Fun with SLIP and the Linux Kernel

I yanked out a beautiful Rainbow 100B last night to act as a web server.  I stripped the hardware down to the minimum for that authentic “unnecessarily obsolete” experience.  The Rainbow I chose was in pretty good condition, with a little bit of dust buildup on the motherboard.  To make sure everything was okay, I pulled the motherboard assembly out (a trivial task requiring no tools on a Rainbow), removed all her daughterboards, and cleaned off the caked-on filth.  This particular Rainbow had a fully populated memory expansion board, bringing system memory up to 896KB, a graphics option, and a hard disk controller.  I decided not to replace the graphics option and hard disk controller while I reassembled the system as they were unnecessary for this application.  After reassembly, I ended up with a Rainbow 100B with 896KB RAM and a single RX50 floppy drive unit.

I set up the box on the table that holds my decTOP box, replacing the giant HP Visualize workstation.  For a monitor, I chose to use a VR241, which is the color monitor for Rainbows.  The VR241 is giant and bulky, comparable to the monitors IBM shipped with the XT and AT computers.  Unlike the Rainbow’s quaint VR201 monochrome monitors, however, the VR241 is independently powered and can be switched off (VR201 monochrome monitors are powered by a 12V power line in the monitor cable).  The whole setup looks a little silly, but I’ll post a picture tonight or tomorrow.

The Rainbow is connected to the decTOP server using the USB-to-Serial adapter and a null modem adapter (and a selection of cables and gender changers).  Once everything was plugged in, I turned on the Rainbow and fired up the SLIP connection.  I’m booting the Rainbow off an MS-DOS 3.10b boot floppy created with my imaging program, the other half of my RetroChallenge. The disk in drive B hosts the http server and all its web pages.

The first attempt at creating a SLIP connection failed for reasons I’m not quite sure about.  The usual way of creating the SLIP connection on GNU/Linux is to:

  1. Run slattach -p slip -s 19200 -L /dev/ttyUSB0 as a background process
  2. Use ifconfig to bring up the sl0 network

Usually, to kill the SLIP connection, the slattach process needs to be killed.  This method works like a dream on SLAX where I’ve been testing this, but on Debian Etch, this process doesn’t quite work. The only way to bring down slattach is to issue a nasty SIGKILL signal, which simply destroys the process. Apparently using SIGKILL does not seem to allow slattach to shutdown elegantly because it never starts up again successfully afterwards.

After trying to get it to reopen the SLIP connection, I finally had to reboot the GNU/Linux box.  I was truly dissapointed because the uptime was at 138 days.  The problem, of course, is that SLIP lives in the kernel.  The unsuccessful attempt to restart slattach, at least in Debian, meant that the whole kernel needed to be restarted.  Sigh…  Rebooting a 400MHz Geode-based system is not a quick process either, especially when it is attempting to run Apache 2.2, a MySQL daemon, and a Postgres daemon (don’t ask…).

After about four reboots and some cable switching, I finally found a nice configuration where the SLIP connection was up and running in a reliable manner.  I made the necessary changes to my Apache configuraiton to pass all requests for a given server (I’ll tell everyone soon!) onto the Rainbow via the SLIP connection.

Testing revealed two bugs in the current web server, one easy to fix, one rather difficult.  The first problem was that uIP wasn’t marking files ending in .htm as HTML files.  Requesting the same file as .html works fine thanks to an undocumented Watcom function, _lfntosfn, which converts long filenames to short filenames.  The change was a quick addition to an if statement to ensure that the three-letter extension was also recognized as valid HTML.

The second issue seems to be that uIP is not dealing with multiple connections correctly.  I observed similar behavior when working with version 0.6 years back, but version 1.0 should not be experiencing these problems.  For example, if a web page contains two images, most browsers will request both at the same time.  What currently happens in my implementation of uIP is that the first request received is honored, while the second is treated to a fine sampling of garbage data.  I’m still trying to figure out how the problem is occuring as I don’t see any points in the code where they should be stepping on each other.  Under a worst-case scenario, I could limit uIP to only 1 connection at a time, which would also fix the problem.

Hopefully, a completed version will be available tomorrow!