Chad Perrin: SOB

24 October 2007

expression vs. construction: how to think about programming

Filed under: Cognition,Geek,Writing — apotheon @ 12:04

Justin James had this to say in his weblog post, The programming paradigm needs an update:

It is all coming together for me now, the conscious understanding of what I really dislike about programming. I got into programming because I like solving problems, but solving problems is really only a small fraction of the work. The rest of it is giving exact, precise details as to how to perform the tasks that solve the problems. It is like painting the Sistine Chapel by putting a seven-year-old boy on a scaffold and dictating to him every stroke you want him to make. It is ridiculous.

Back when I flirted with WYSIWYG web design software for a year or so, I started running up against a similar wall in my enjoyment of the craft. I had many of the same issues with web design that Justin seems to be having with programming — it was tedious, micromanaging work, starting with doing a broad sweeping overall design, then having to subdivide into parts to fix the initial generated code to make it actually do everything I needed. I found myself wondering what I’d ever found to enjoy about web design.

One day, I had to do a fair bit of web design work on a machine that didn’t have Dreamweaver installed. I cracked open Notepad and got to work. Guess what happened:

I enjoyed myself!

I discovered that the major hurdle in my way is that I was treating the process of web design as something akin to bricklaying — I created a grand design document, I threw together sections of wall mechanically and largely automatically, then I fit individual bricks into the spaces to tie it all together. Tedious, really. When I just used an empty text editor as my page, I could compose the code I needed rather than constructing it. Not only did it become something more like finely detailed art, as though writing a character-driven tale with the plot as something that emerged naturally rather than being hammered into the reader at every turn, as opposed to some kind of boring civil engineering task, but the end result ended up being much higher quality, better suited to the specific task I needed to accomplish. The end result got better at the same time that my enjoyment grew.

I had up to that point never used an IDE for any actual programming — I had always just used a bare text editor. I later picked a few up and played around with them for a while, but they never actually accomplished anything any more quickly for me, and worse yet they tended to crush my enjoyment of coding. I never really made a specific, direct connection to the situation with using a WYSIWYG HTML editor before today, and the effect wasn’t nearly as dramatic and notable because I never spent as much time with an IDE as I had with Dreamweaver, but now that I think about it I suspect much the same thing was going on.

Perhaps the problem Justin is experiencing is in large part because he progressed from vi to IDE, rather than from vi to Vim (or nvi, or whatever advanced successor to vi he’d prefer). The IDE culture that has grown around Java’s and Microsoft’s shared domination of much of the software industry turns programmers into manipulators of automated assembly lines, rather than masters of a craft. I think the key problem here is in how we think about programming: when using all those code generation tools to produce masses of scaffolding into which we’ll insert library calls and class APIs, we think of programming as a rote exercise in rearranging pregenerated blocks of reusable atomic substructure; when we break open a powerful text processing application like Vim and start with a shebang line or header includes, then proceed to typing the first instance of a function or class definition, we are actually thinking in the programming language. It’s more like writing a sonnet — structured and guided by rules of a sonnet’s composition, but in some ways more an artistic expression than a step-by-step execution of the instructions for building a suspension bridge.

I’ve heard it said many times that the fun programming involves solving new problems, rather than just solving the old problems all over again. This is why smart programmers usually end up wanting to work in AI, or otherwise pursuing new avenues of applying and expanding their understanding of computational theory. I’ve seen some, however, who are just as happy to construct CRUD (Create, Read, Update, Delete) applications that are a thousand times as good as the standard Java CRUD app. I, too, am not terribly bored by solving old problems in a slightly different way to suit my own preferences. I think the difference for the bleeding edge problems is not necessarily that they’re problems nobody has solved before, just as real, original, good poetry can still be written in iambic pentameter and about the ages-old topic of lost love. Rather, the difference is in the tools that are used. Because bleeding edge problems cannot be solved with code generators and other tools that just rearrange blocks of commonly used code, the programmers “flex” their brains and have to really think in the language they’re using, composing something of an artistic expression of the problem domain in code. You can do that with any kind of problem — even a largely typical CRUD application. Enjoyment of the craft is not limited to solving problems nobody’s considered solving before. I think that usually just doing a better job on an old problem than anyone else has because you don’t let yourself become constrained by the code generation tools available to you is all that’s needed.

Instead of thinking about how to rearrange big blocks of functionality to get the solution you want, think about the problem itself and how to solve it in your programming language of choice — and maybe even think about which language can best be used to express the problem (and its solution) in this particular instance. I find that the best tools for that are those tools that make editing code exceedingly fast and easy once you’ve mastered them to some degree of competency without masking the language you’re using behind automated operations like class builder functionality in your IDE. I’m talking about tools like Vim, of course.

I think this is also part of the reason that daycoders tend to like their Javas and C#s and other verbose, repetitive, pattern-bound, IDE-requiring languages, while the really eccentric and brilliant hackers in the open source community tend to prefer their Lisps and Perls (and Rubys and Pythons and Haskells and Erlangs) and other succinct, flexible, powerful, dynamic languages that don’t lend themselves to IDE abstraction (but, instead, lend themselves to tools like Vim and EMACS, aka Esc Meta Alt Ctrl Shift): one is for people rearranging pregenerated chunks of functionality to solve a business need quickly but not necessarily well, while the other is for people solving problems they personally want solved — and solved as perfectly as possible for their particular needs. One set of languages is the toolset for assembly line work, and the other is the artisan’s carefully chosen set of tools for a craft.

I agree with Justin, that languages like Lisp can be difficult for many programmers. Lisp is in many ways inherently recursive, requiring one to really grok recursion to use the language to anything even approaching its full potential — and recursion is just the beginning with Lisp. On the other hand, dynamic languages in general need not be so difficult. Ruby, for example, is one of the most dynamic of widely known languages, and yet it’s one of the easiest to grasp. In fact, while I’ve long understood what the map operator in Perl does and have explained it effectively to others who, despite years in the Perl biz, never quite grokked it, I always had to think about it for a few minutes every time I used it to remember the specifics of how it’s used — but working with iterator blocks (such as when using the Array#collect method) in Ruby has provided me with a greater understanding of how language constructs like map work at a deeper, more intuitive level, and Perl’s map operator (as well as its grep operator, for that matter) is second nature to me in a way it never was before. Perhaps better yet, in many ways Ruby acts as a simplified (yet more flexible and rich) Java for the Java daycoders, as it’s a really solid object oriented language that nonetheless eschews much of the repetitive scaffolding and cruft necessary to perform even the simplest tasks in Java (or C#, or C++, et cetera).

In short, I’m not sure we really need to move more toward “intentional languages”. I think the key is that programmers must be able to express their intentions in the languages they use — which requires thinking in the language, and not treating the language as bricks and mortar which must be manhandled into place. A (good) programming language is a language — a means of expressing oneself — and not just a pile of raw materials. If you can’t use a programming language as an actual language, you’ll find it very difficult to express the problem domain and its solution, just as it’s difficult to express one’s intent elegantly by tearing pages out of books on your shelf and attaching them to form a new book.

All original content Copyright Chad Perrin: Distributed under the terms of the Open Works License