Over on slashdot, folks are posting their predictions for technical developments in 2003. Well, here's my prediction. In 2003 the GNU/HURD OS will finally become usable. A combination of ideology, technical interest, and better understood development methodologies is finally going to bring this project to fruition. I don't anticipate a 1.0 release (possible by December but not likely), but I think we will see GNU/HURD in a state similar to Mozilla before 1.0: slow recognition that the project is viable and producing something worth using, a 0.9 level of polish that slowly, almost asymptotically approaches 1.0.
But HURD is dead! We all know this to be true, right? I won't argue. But I will point out two community projects that have variously been pronounced dead and are now leading the way for new developments no one anticipated: Mozilla, and Perl 6. I think all three of these projects share some commonalities: an initial surge of interest that cannot possibly be assimilated into a development process, followed by a general disappointment, abandonment, and outright flaming phase, a shrinking of the group of interested parties to a small dedicated core, lots of unrecognized work behind the scenes, and a gradual awareness in the community that this is Something Good.
We've all heard about the ideological pressures for GNU/HURD. Mercifully, I won't repeat them here. :) That's not to say I don't find them important. One interesting thing you might not know about is binary drivers in the kernel. Basically, someone took a compiled driver, converted it to a bunch of hex bytes in a C struct in a file, added it to the kernel, and called it source. There are some people who are unhappy about that.
Technologically, HURD has a lot to offer. The microkernel underpinnings make for a lot of new features: the whole OS is decentralized, in that authentication can go through the "offical" authentication server (/etc/password or whatever), or through a user-implemented alternative. Nothing except hardware access is really controlled by the kernel. It's like a libertarian , anarchistic OS, where you can ignore all the offical services and provide your own. :) Moreover, you can even mount your own filesystems. As a regular user. These filesystems can be real disks, network entitities, or interfaces to something as yet unheard of. The canonical example is an FTP filesystem. Personally, I've always wanted to be able to mount anything I can ssh to. In my home directory. :) I've even dug into kernel internals to see how that might work, but unless someone wants to buy me off of my current job and put up with me on a moderately steep learning curve, it isn't going to happen anytime soon. It'll be much easier with HURD, though, and I expect to see it.
HURD has a lot of similarities with Darwin, when you think about it. Both are a UNIX like OS running on a microkernel. There's a lot of untapped potential there for new OS possibilities. Microkernels still don't rule the roost, despite being universally recognized in academia as the One True Way. Darwin/Mac OS X has meant the installation of thousands of microkernel based systems across the globe. HURD will mean even more.
Speaking of Mac OS X, work has been done to make HURD work on the version of Mach that sits at the heart of Darwin instead of the GNU version of Mach. GNU/Mach worked on multiple architectures at one time, but that support has been temporarily abandoned in favor of just getting GNU/HURD up and running on Intel. If HURD worked on the Mach from Darwin, it would suddenly have PowerPC to play on, too. There's also a project to port HURD to the L4 microkernel. It's said that Mach is showing its age, and L4 is the next best thing. I wouldn't anticipate seeing L4 play any role until post 1.0, though. I'm not sure what benefits it brings, anyway. Still, interesting that you can take all those servers (read: plain programs) that comprise the HURD, compile them for a different microkernel, and get basically the same OS.
There's been a lot of advances in community development methodology, too. I think we all know how in the early days of GNU RMS kept a tight reign on everything, sometimes to the detriment of the project. Linus, ESR, egcs, and others (you guys!) have shown that this is not the way to run these projects. Frequent releases, complete openness, invited volunteer contributions from anyone, and all the factors in CatB have proved to be the way to run this kind of project. And as that development methodology has become more pervasive, HURD has slowly gained progress.
Nowadays HURD has a very special ally: the Debian project. In the same way that people would like to make the HURD servers run on different microkernels, Debian likes to make their OS (this is OS in the sense of "all the programs and utilities that make a system usable") run on different kernels. Did you know there are several Debian projects that do not involve the Linux kernel? Debian/NetBSD, Debian/Win32. Almost scary. A large part of the important work in making GNU/HURD usable is occurring in the Debian/HURD project. Take the thousands of packages that make up Debian and compile them, one by one, on GNU/HURD. Fix bugs. Send patches back to maintainers. Stress test the system. Build a beautiful apt repository so all GNU/HURD users can be running and testing the absolute latest. Think of how Debian has an almost identical running OS with thousands of packages across so many different architectures: Intel, PowerPC, Sparc, etc. They'll put all that work into making Debian/HURD usable, reliable, and consistent, as well. As a result of this volunteer and mostly decentralized effort, GNU/HURD is going to be a very usable system with thousands of running packages right from the start.
Once there is a running and usable HURD system, optimization will begin. I'm certain this will be just like Mozilla: complaints that the code is a memory hog, complaints that it is needlessly slow. But optimizations will occur. I hope Perl 6 doesn't follow the same pattern. I want it fast from the start. :) But, premature optimization is the root of all evil, and if there's any message to this little essay, it's patience.
One other thing I think the free software community has learned as a whole that will play a prominent role in making HURD usable and popular is how to port an OS across architectures. BSD lite started out as an Intel OS. I think. (Actually, it was a descendant of a VAX OS, which was a descendant of a PDP OS. But who's counting?) Now NetBSD runs on 38 architectures, and FreeBSD is being ported. Linux, the kernel, started out as an Intel only OS, and now runs all over the place. The history of porting that kernel to other architectures is a great lesson in extreme programming and refactoring. Linus didn't care if his kernel ever ran anywhere besides Intel, and in fact he started his work as a chance to learn and practice Intel assembly. He didn't worry about porting his OS at all, because that wasn't needed at the time. Other people came back and refactored the codebase to make it easier to port, then took it to their favorite chips. If it weren't for this history, I'd be upset and scared that HURD is Intel-only right now. As it is, I'm eagerly looking forward to watching people use lessons learned to port HURD and GNU Mach.
The HURD is dead. We've known that for years. We also knew Mozilla was dead, and here I am posting this from its cousin Phoenix. Perl 6 was called a disaster for months, then suddenly one day there was a working Perl 6 grammar and parrot interpreter. Yet, the motivation of some dedicated hackers is unstoppable. We will almost certainly see a usable GNU OS in 2003, and RMS will finally have the fulfillment of his long-delayed vision.
--Nat
Re:I'm skeptical
jdavidb on 2002-10-11T23:05:47
Good answer.
BSD started life as a set of patches to AT&T V7 Unix and it initially ran on PDP 11s. AT&T had that same kernel running on several different architectures at that point. AT&T was bragging in the mid/late 70s at how easy it was to port Unix. The line I seem to recall was that the Unix Kernel was only 700 lines of assembly on a PDP11, replace that part and you had a port (actually, the issues were more complex than this due to various endian assumptions that had to be worked out in the C, but those were largely eradicated from the AT&T Kernels sometime in the 70s).
Now, it may be true that Berkeley only had PDP 11s and later VAXes to work with for a while. I'm not really sure when they had other architectures running, but this page mentions the Power 6/32, whatever that was.
However, by the time BSDlite hit the scene, BSD, in some form or another, had been ported to 68000s, Sparc (Sun OS 4 was BSD), MIPS (DEC Ultrix was BSD), NS320xx, Motorola 88000 (? Pretty sure DG Aviion was BSD based), and numerous others, I think. Pretty much every Unix vendor in the 80s except AT&T were selling their own version of BSD.
The Free Software community did not invent porting Unix, nor did they really extend the science of it. Perhaps they did it more, that's all.
Re:Uhrm... no
jdavidb on 2002-10-12T02:06:02
Well, I knew there was a lot more porting involved in there that I was leaving out. I specifically knew about Motorola chips and some of the other things you mentioned. Yes, UNIX has always been very portable. I consider the "free software community" to be a descendant of that same group that was doing that work back then.
I do know that NetBSD reached new heights of portability. They reached the point a long time ago when a single driver could be used in the OS on different architectures. Very modular, architecture specific elements very encapsulated. And I think Linux is a great example of getting to that point through refactoring, after ignoring the "need" for portability in the early days, when it would have been counterproductive. I was really getting at the extreme programming principles in action more than anything else.
Definitely not trying to discount the portability work in UNIX, which in fact predates not only the modern free software/open source movements, but also predates me!
:) Re:Uhrm... no
jdavidb on 2002-10-12T02:08:11
Let me rephrase again.
:) What the community has learned is not how to port an OS. That's textbook. (Well, sorta.) What the community has learned is how to throw the open, distributed development model at the task of OS portability effectively. Very effectively. I'm anxious to see it thrown at HURD.
The reason I'm skeptical for may be obsolete by now, and I'd be happy to be corrected on this. The reason microkernels didn't use to fly despite being a very nice design, was simple: performance, or the lack thereof. All those layers of abstraction and encapsulation simply took too much time to go through. In other words: the performance of microkernels sucked (compared with monolithic kernels).
That was the state in mid-nineties, when I knew some of the Mach guys (one of the Mach projects was called Lites, and I used to know the core people of Lites). Nice design, but sucky performance, even these very bright guys couldn't make it much better. Then came Microsoft and bought most of the Mach people of CMU, and one of the Lites guys, to do hush-hush projects for their corporate research.
Of course, Mach/Lites is not Hurd (maybe the Hurd people have figured out how to write fast microkernels and the servers around them), and in ten years the Moore's Law might have helped the slowness a lot: just throw enough hardware at the problem. So I don't know what the situation is these days.
P.S. In another microkernel news, if you are interested in that kind of stuff, you might want to look at Chorus/C5: a rather nice and very modular real time OS. Originally a French academic project (just like Mach), then productized, then bought by Sun, and now quite recentely open sourced and reproductized ("get support from us"). Source code free (not GPL), but one of the Sun OS licenses.
Re:I'm skeptical, too...
jdavidb on 2002-10-12T15:59:09
I meant to mention a little more about the performance issue. One thing I was trying to get at but failed to mention explicitly is that many of us are now using a microkernel on a daily basis. You specifically: I know you are because I saw you with an ibook at YAPC.
:) OS X is built on a microkernel, and all you new Mac users can tell me how the performance is. I would not be surprised if the HURD I am predicting has poor performance for awhile. I am optimistic that if Apple can make a usable microkernel OS, the HURD people can figure it out, too. (Ask me in a year; I may take it all back.)
I see yet another parallel here with the Mozilla project. When I finally started hearing good things about Mozilla and tried it, it was painfully slow and ate far too much memory. I gave it up for dead (again...), but it wasn't long until I heard "everybody" (for some value of "everybody") was using it, tried it out, and discovered they had fixed the performance. I expect HURD to go through a similar cycle. I pray Perl 6 doesn't, but if early results give disappointing benchmarks, I'll be there saying "Give it time."
To be honest, I'm still a bit skeptical myself. But having watched some projects go through the cycle of monstrous developer interest, near invisibility, and sudden success, I have an inclination HURD's time may be sooner than everyone expects.
Re:I'm skeptical, too...
jhi on 2002-10-12T23:54:57
> OS X is built on a microkernel, and all you new Mac users can tell me how the performance is.
That's easy: my computers are always too slow
:-) be they 8-bit home micros or 1024-node Crays.
