Why is programming so captivating, yet so agonizing? | elsewhere

Today, AI coding tools are lowering the barrier to programming at an unprecedented pace.

More and more people are building products with their own hands, and revisiting a fundamental question: what kind of craft is programming? Why is it so captivating, and why is it so hard?

Frederick P. Brooks Jr.'s "The Tar Pit," written fifty years ago in The Mythical Man-Month, reads with surprising relevance today.

Brooks is best known for leading the development of IBM's System/360 and its operating systems. He likened building large software systems to prehistoric beasts trapped in tar pits: the real danger was never any single difficulty, but countless problems surfacing simultaneously and entangling with one another. The harder a team struggled, the deeper it sometimes sank.

Programming is both painful and joyful. It mires you in trivialities, yet also empowers creators to turn ideas into reality.

May you pass through the tar pit, and still find joy in creation.

The Tar Pit

In prehistoric times, one of the most vivid scenes was that of great beasts struggling to their death in tar pits.

Dinosaurs, mammoths, saber-toothed tigers... the harder they fought to break free, the tighter the tar gripped them. No beast was powerful enough to avoid sinking.

For the past decade, large-system programming has been the tar pit.

Many a powerful team has thrashed violently in it. Many have emerged with running systems — few have met goals, schedules, and budgets simultaneously.

The difficulties are not inherent to any single aspect. Any one problem seems solvable, like a single paw that can still pull free. What is truly terrifying is that many factors appear simultaneously and interact with each other, ultimately pulling the entire project deeper and deeper.

To solve this, one must first understand the craft of programming itself, and the intrinsic joys and woes within it.

Program, Product, System

We often read stories in the news of two programmers in a converted garage whose program outperformed a giant's.

Every programmer wants to believe such stories, because we know that if it's just writing a program, our speed far exceeds an organization's "1000 lines per year" pace.

So why haven't big-company development teams been replaced by focused garage duos?

In 1976, 21-year-old Steve Jobs and his friend Steve Wozniak founded Apple Computer in a garage

A "program" is something complete in itself, runnable by its author in the development environment. This is the garage project, and what programmers implicitly reference when estimating their own productivity.

But for a program to become truly useful, it must cross two boundaries.

The first boundary is from program to product.

It no longer suffices for the author alone to run it. Anyone must be able to run, test, repair, and extend it. It must function across different operating environments and handle diverse data.

To become a general product, the program must be written in a more generalized way. Especially the range and form of inputs must be generalized as far as the basic algorithm reasonably permits.

Then the program must be thoroughly tested, so that one may trust it. This means preparing, running, and documenting a substantial body of test cases to explore its input range and boundaries. Finally, complete documentation is needed so anyone can use, repair, and extend it.

By experience, a product costs at least three times as much as a debugged program with the same function.

The second boundary is from program to component in a system.

A system is not an isolated program but a collection of interacting programs, coordinated in function and constrained in format.

To become a system component, every input and output of the program must conform in format and meaning to defined interface specifications. It must also be designed to a predetermined resource budget — memory space, I/O devices, compute time. Finally, it must be tested with other components in all expected combinations.

This testing must be extensive, because the number of test cases grows combinatorially. It is also time-consuming. Even if each component is debugged in isolation, unexpected interactions between components can still produce subtle bugs.

A system component also costs at least three times as much as a standalone program with the same function.

When a program is simultaneously a product and a component, its cost is nine times that of a simple program. Yet this is the truly useful result, and the goal of most development efforts.

The Joys of the Craft

Why is programming fun?

First is the sheer joy of making things.

As a child delights in his mud pie, so the adult enjoys building things, especially things of his own design.

I think this joy must be an echo of God's delight in creating — each leaf, each snowflake unique and fresh.

Second is the pleasure of making things that are useful to others.

Deep down,...