Software Portability in the Linux Age
A friend of mine has been having difficulty for some time, now, getting KDE2 to work on an SGI running the latest IRIX and on a DEC Alpha running the latest NetBSD. This software is a product of the Linux community, and as such, it seems only to work on Linux. There are many other examples of this phenomenon. Not that that helps him, of course. Right now, he feels like giving up and going back to fvwm.
After a particularly frustrating round of battle with the code, he wrote me with the error messages. At first, I was speechless. This code seems to sink to new depths of low quality. Then, I pondered some more, and wrote:
Software Quality Cycles
Actually, on second thought, I do have something to say about this, but it’s a bit incendiary. I’ve noticed a “Linuxization” of the open source base in recent years. That is, there’s a mentality that free software == Linux, and both are the end-all be-all of software. This leads to people writing software that just barely compiles on Linux, and works on their x86 box, and they release it as a package.
Of course, this does improve the world in its way, since the work they’ve done is something, and it can be used as a stepping stone to creating a workable solution. Take the X-10 Firecracker code, for instance. Someone released his code that worked for him on Linux on x86. Someone else came across it, hacked on it for a while, and released patches that made it work on FreeBSD. Fortunately, the original author incorporated the patches, and the result was something that with minimal effort works on NetBSD (and that I use to this day). This really can work.
However, while this can work for the forces of Good, it means that there are an awful lot of projects out there in this intermediate stage of works-on-Linux, but that haven’t been made generally useful, yet. This gives those of us who don’t happen to run Linux the impression that the quality of code in the world generally sucks.
Of course, it would be best if the people writing and releasing code would read a little bit or otherwise learn about writing portable code. It’s not that hard, I think we all agree, to write code according to specs, such as the POSIX standard, or C, or even C89, or the published interface of whatever libraries they’re linking against. (Of course, in many cases, the libraries themselves are poorly documented or poorly implemented.) But, on the other hand, there’s little influence on coders to do this. Sure, it’s The Right Thing, but why bother, when it doesn’t seem necessary? From their perspective, what they’ve produced works just fine. I suspect that many of these code authors don’t even know that they’re writing non-portable code, or even that there’s such a thing. And the rest don’t care.
Right now, the dominant operating system for which programs are written and their source released on the net is Linux. It used to be SunOS, remember? Fortunately, many of us ran SunOS back then, so it was nice for us. If you found a package out there on the net, it was going to compile, and it was going to work on your SunOS box. Almost without fail. Quality was high, and source code was portable. Well, that’s not true, of course; it just seemed that way. When we got our first Alphas at Glue, I spent months porting the hundreds of packages we used to the new machines. The major problems were 64-bit cleanliness and assumptions about the sizes of pointers and integers. There were also assumptions about syslog (OSF/1 used the two-argument 4.2BSD, rather than the three-argument 4.3BSD syslog), signals, and other standard library and syscall features that had always worked just fine on SunOS.
I like to think of myself as the perfect programmer, of course. I write portable code, and I check my code on multiple platforms before committing it, publishing it, or whatever. But, that’s just how I like to think of myself. I have made mistakes, too. Around 1990, I submitted a module to the author of xv that allowed xv to render xpm files. In order to save memory and improve performance, I used a hash table to store the color lookup index. (The color identifiers can be arbitrarily long, and there can be arbitrarily many of them, so an array lookup would require an arbitrarily large array in memory, plus parsing the file twice.) To do this, it turns out that I used a hashtable feature specific to the SunOS libc. The xv author rightly rejected my contribution, presumably because it didn’t work for him. When we got those Alphas, I couldn’t get it to work, either, and finally understood why it had been rejected from xv. (I never did find a good solution, and as far as I know, xv still doesn’t handle xpms.)
Fortunately, within a couple years or so of the introduction of Alphas, most of the code I came across became much more portable. I think this is largely because universities, such as mine, were adopting Alphas in large number, and since most free software came from universities, or was at least heavily worked on at them, it was all made to work. The early ’90s saw a great improvement in the quality and portability of free software. This was also helped, I suspect, by the release of 4.4BSD-Lite and the ensuing rapid growth of adoption of the pure-BSD Unices, which had a vested interest in high-quality, portable, Posix-compliant third-party source code being available.
Then, Linux exploded. Sure, it was there even in the late ’80s, but market and other forces, in particular the popular explosion of the Internet, caused its popularity to rise in the late ’90s to levels none of us could have imagined before. And with it, teeming hordes of newbie programmers eager to contribute. And contribute they have. Sadly, lacking the experience of the years before this surge, they are writing code that’s in many ways like the SunOS-specific code we grew up with during the surge of Internet popularity on college campuses.
A decade ago, Suns were dominant, and closely tied to the rapid academic adoption of the Internet. “The network is the computer,” and all that. And Suns were among the first machines I know that came with an ethernet interface. And we were all writing code for SunOS and releasing it on the Internet. It had to be ported to work on anything else, and eventually, either it was ported, or it fell rapidly into disuse. Today, the Internet is again being rapidly adopted by a fresh group of people, and this time the operating system most often associated with it in people’s minds is Linux. So the same thing is happening again.
So what am I saying, here?
I guess what I’m getting at is that while the average portability and general quality of the extant free software code base has deteriorated rapidly, that’s largely because there is so much more software out there than ever before, and because it’s almost all being written with only Linux in mind. These new coders are, in fact, doing us a service, by providing their work. Porting code is usually much quicker than writing it from scratch. And that’s even presuming that those who would do either the porting or writing from scratch would ever come up with all the ideas that the large number of coders out there have. Which they couldn’t.
And there is hope, of course. There are those of us porting the code bit by bit. And, if the past prognosticates the future, as long as Linux does not remain forever dominant, the existing code base will eventually be made portable by the need to make it work on whatever’s next. And those projects that don’t become portable will die.
I would also suggest that we can improve things on a shorter term, without having to hope for Linux to be usurped, by helping to educate the newbie programmers. This would probably best be done by advocating for portability amongst the Linux crowd. Perhaps writing web pages or other documentation about writing portable code, and making it readily available as a how-to for programmers new to the open source community. I remember that one of the best resources I had when I started writing portable code was “Porting Unix Software,” a book about how to port code and write portable code between SysV and BSD, the two dominant flavors of the time. We need the modern equivalent of that book. And we need people to read it.
- Geoff
13 Dec 2001
(Ten years to the day after the first public demo of the World Wide Web)