Software Is Hard

GameArchitect.net

Musings on Game Engine Structure and Design

-->

Home

What It's All About

Who Is This Guy?

The List

Complete Archive

Personal

RSS Feed

People

Software Is Hard

By Kyle Wilson

Sunday, August 19, 2007

"Software is hard," reads the quote from Donald Knuth that opens Scott Rosenberg's Dreaming in Code. The 400 pages that follow examine why: Why is software in a never-ending state of crisis? Why do most projects end up horribly over-budget or cancelled or both? Why can't we ship code without bugs? Why, everyone asks, can't we build software the same way we build bridges?

The framing story for Rosenberg's investigation is the Open Source Applications Foundation's Chandler project. Chandler begins life in the spring of 2001 as an alternative to Microsoft Exchange:

[Exchange] was the least unbearable of available options for small-group scheduling. Still, it seemed crazy, far more powerful than the office needed. Costlier, too: You had to dedicate a computer as the server, you had to pay for a Windows license for that server, you had to license the Exchange software itself.... Before you knew it, you were spending thousands of dollars just to keep a handful of calendars in sync.

OSAF is a non-profit venture funded by Mitch Kapor, who made enough money as the founder of Lotus that he can afford to spend a few million dollars out-of-pocket on software that may change the world. Kapor's munificence makes Chandler immune to the budget pressures that force most software applications to ship before they're ready, but it also means that there's no real pressure for Chandler to ever ship at all. "We're not operating with the mythology of the Silicon Valley death march, with the deadline to ship a product and get revenue, where the product quality goes out the window," Mitch Kapor says in Dreaming in Code.

As a counterpoint to OSAF, Rosenberg presents 37 Signals, which has made a business out of small web-based applications. Jason Fried, the president and founder of 37 Signals, says, "Constraints are key to building a great product. They're what makes creativity happen. If someone said you have all the money in the world to build whatever you want, it would probably never be released. Give me just a month!"

Chandler's saga is an uncomfortable story for me because it's powerfully reminiscent of development at Cyan back in 1999, when we started on what eventually became Myst Online. Mitch Kapor created Lotus 1-2-3, a killer app that brought him surprise fame and fortune at an early age. Cyan CEO Rand Miller created Myst, a killer game that brought him surprise fame and fortune at an early age. Kapor ended up a lot richer, but they both made enough from their early success to leave software development behind and live comfortably for the rest of their lives. Instead, both men spent millions of dollars from their personal fortunes trying to create something even greater. Rosenberg asks Andy Hertzfeld, an early OSAF volunteer, if Kapor's accomplishments aren't enough:

"No, he still needs more glory. We all need more glory as designers--to show we can design another great thing. Everybody who has a first success, especially when it's young, wonders: Was it luck, or was it skill? Well, it's a little of both. If you can do another really great one, it shows the world something."

I don't want to disparage Mitch Kapor or Rand Miller. I know Rand is, and I believe Kapor to be, genuinely good, likeable, and smart. But the determination to make something insanely great is a kind of hubris, the flip side of which is a terrible fear of making something merely ordinary. Myst Online started out as a single-player game of modest scope called D'ni In Real Time, or DIRT. By the time I left Cyan, it was planned to be an MMO with a million subscribers, a driving application that would convince casual computer users to sign up for broadband Internet connections. Chandler starts out as a free replacement for Exchange that's simple enough for casual users. But that's not enough. That would be merely ordinary.

The story of Chandler's ensuing development is, for anyone who's spent time developing software, sadly predictable. There comes a point halfway through relating Chandler's saga, where Rosenberg writes, "By now, I know, any software developer reading this volume has likely thrown it across the room in despair, thinking, 'Stop the madness! They're making every mistake in the book!'"

It's true. Here are a few:

Peer-to-peer architecture. From the beginning, Chandler is conceived as a peer-to-peer application, for little obvious reason except that P2P apps were considered to be a new and nifty thing at the time the project was starting up. This greatly complicates its communication and security requirements.

Simultaneous cross-platform development. Because it's an open-source project staffed by Linux users and ex-Apple employees, Chandler is developed simultaneously on Windows, Linux and...