Chad Perrin: SOB

11 March 2008

insomnia and productivity

My insomniac productivity a few days ago wasn’t just geared toward writing lengthy rambles about the differences between various forms of philosophical libertarianism. I also wrote code — in particular, I wrote an entire program from start to finish, inspired by code I found online for doing the same thing that (frankly speaking) sucked.

I wonder why it was such a productive night. As I think about it, I remember that my experience in years past mirrors the stereotype: geeks that stay up all night hacking away at something that totally consumes their attention at the time, producing far more in a couple nights’ work than your average professional does in a month.

The phenomenon has been termed hack mode:

a Zen-like state of total focus on The Problem that may be achieved when one is hacking (this is why every good hacker is part mystic)

More about it, from the same source:

Being yanked out of hack mode (see priority interrupt) may be experienced as a physical shock, and the sensation of being in hack mode is more than a little habituating. The intensity of this experience is probably by itself sufficient explanation for the existence of hackers, and explains why many resist being promoted out of positions where they can code.

Recently, Sterling “Chip” Camden (of Chip’s Quips and Chip’s Tips) recently wrote on the subject of geek productivity. It was cast as consultant productivity, of course, because that’s the sort of geek he is — but it gets into the same concepts as the Jargon File definition of “hack mode”.

Compare his words:

Focus. One reason why IT pros don’t always do so well in social situations is that they have the power to concentrate intently on one problem to the exclusion of everything else. (No, honey, I wasn’t ignoring you — I actually didn’t hear you, even though you were standing right beside me and speaking clearly. I might have even said “uh huh,” but that was just a 202 response.) We geeks need to maximize this ability in order to be effective. We can’t allow ourselves to be interrupted for “just a minute” to answer a question. The instant we break concentration, the entire problem that we were modeling in our head drops on the floor and shatters into a million pieces. This is why you shouldn’t answer the phone, e-mail, or chat during these periods.

. . . with these from the “hack mode” entry at the Jargon Wiki:

Some aspects of hacker etiquette will appear quite odd to an observer unaware of the high value placed on hack mode. For example, if someone appears at your door, it is perfectly okay to hold up a hand (without turning one’s eyes away from the screen) to avoid being interrupted. One may read, type, and interact with the computer for quite some time before further acknowledging the other’s presence (of course, he or she is reciprocally free to leave without a word). The understanding is that you might be in hack mode with a lot of delicate state (sense 2) in your head, and you dare not swap that context out until you have reached a good point to pause.

Crazy-late hours of intensive hacking lend themselves naturally to entering, and sustaining, a deep state of “hack mode”:

  • Other (saner) people aren’t around to interrupt. They’re all asleep. This provides additional safety when juggling eggs.
  • The requirement of keeping from disturbing others inhibits the tendency to engage in distracting entertainments (like TV). This almost forces one to focus entirely on one’s work. Only something like reddit stands in the way.
  • Subjectively speaking, it seems that the wee hours of the morning lend themselves to an almost surreal, abnormal perspective. Inspiration can be found in odd places under that sort of influence — which leads to new ideas coming to mind and being explored that, by the light of day, may have been dismissed immediately as “crazy talk”.

Discovery of these benefits to late nights coupled with the heady experience of “hack mode” itself (it’s like a powerful psychologically addictive drug — really and truly), along with other factors, surely contributes to the increasingly nocturnal and insomniac (sometimes to self destructive extremes) tendencies common in “larval stage“:

a period of monomaniacal concentration on coding apparently passed through by all fledgling hackers. Common symptoms include the perpetration of more than one 36-hour hacking run in a given week; neglect of all other activities including usual basics like food, sleep, and personal hygiene; and a chronic case of advanced bleary-eye. Can last from 6 months to 2 years, the apparent median being around 18 months. A few so afflicted never resume a more `normal’ life, but the ordeal seems to be necessary to produce really wizardly (as opposed to merely competent) programmers.

This makes a lot of sense, considering the fact it generally takes ten years to become a competent programmer. The same is essentially true of anything — it takes ten years’ worth of work to become truly proficient (what the unambitious might call a “master”) in any skill that embodies great technical challenges. It’s not the actual chronological period of ten years that matters, of course — it’s the amount of time and effort invested in learning and practicing the skill that matters. It just observably tends to take about ten years for most people who consistently pursue such skills to reach a level of significant proficiency.

I’m prone to insomnia — it comes and goes, sometimes afflicting me for a week at a time, occasionally longer, sometimes only one night here and there. I try more and more all the time to adopt a more “normal” sleep schedule, but I’m simply not nearly as effective at this as I’d like. The insomnia keeps coming — and the definition of larval stage holds a clue to one of the reasons for it:

A less protracted and intense version of larval stage (typically lasting about a month) may recur when one is learning a new OS or programming language.

This obviously isn’t the sole reason for my bouts with insomnia, or even the primary reason. It is, however, probably a contributing factor.

I don’t think I have a perfect handle on the forces at work, however — even those in my own head. The above-listed benefits can’t be the only reason that, when I allow it, insomnia can be so productive, for instance. It’ll bear more thought.

By the way, the program I wrote was a command-line utility that retrieves currency exchange rate data from the Internet and provides up-to-date translations between currencies. It was inspired by the train wreck of a currency exchange utility I found here. As of this writing, that program I found is hopelessly broken, in addition to being poorly written and essentially unmaintainable — thus my desire to write something better.

31 May 2007

10 tips for developing deployment procedures (or: Deployment Is Development)

Filed under: Geek,Popular — apotheon @ 07:56

People* have a tendency to think of deployment as something the admins do — if developers put any thought into it at all, it’s usually an afterthought, and not one that is considered very important. That is, of course, a huge mistake: deployment is a development concern, too.

  1. Have at least two (types of) boxes on which you test deployments at every stage of development. One should be an example of a standard machine in the production environment, and the other should be whatever close approximation you have as part of the development environment. That means that during development of a desktop application, for instance, you should at every major development stage test it on both an end user desktop system and on your own development workstation (or close equivalent). Make sure the thing deploys under wide-ranging circumstances, and ensure that if it doesn’t deploy properly during testing it hurts enough to make you consider deployment matters very carefully. For independent consultants, this may mean requesting a test box from the admins at whatever organization is paying you for development.
  2. Make sure you actually test deployment at every stage. It is becoming ever more fashionable to develop in stages so that at every interim goal you have something that actually runs when it comes time for testing and project specification assessment. When those interim goals are met, make sure you deploy and test as though it were the final deployment to the production environment, and make sure you don’t skip either the production environment test box or your own box in the deployment test (though of course you might want to try the test box first in case it blows up). Don’t consider the milestone reached until it deploys smoothly to both.
  3. Make sure deployment/installation is a largely automated process. Use scripts where necessary, employ the system’s software management applications for deployment where at all possible to integrate it with the environment and reduce confusion, and ensure no steps of deployment are missed. Have a complete, functional, automated installation process working and tested before you declare your interim goal met.
  4. Use automated testing scripts to verify that everything is where it needs to be after a test deployment, and make sure that it’s an automated, idiot-proof verifier so you can incorporate it in the final product. Ensure that, at minimum, it logs any discrepancies it finds so that you have a record of what went wrong during deployment, providing you with a neat and orderly to-do list of things to fix. Make sure that the end-user or sysadmin that will be doing deployment can use the verification system without your help, since somewhere down the line someone you’ve never met will be doing exactly that — guaranteed, assuming your software is good enough that it’s used more than once. Your verification should check locations, version numbers, and file permissions. Integrate your testing scripts into the automated deployment process. Actually use it yourself for test deployments, even on your own machine, and design it so that’s possible without a bunch of special-case handling.
  5. Make sure real human beings test the software after every test deployment to make sure it operates as expected at this stage of development.
  6. Keep necessary configuration to a bare minimum. Best of all would be to have no necessary configuration options at all, with the possible exception of asking whether custom configuration options are desired. Provide opportunity for custom configuration, though, in case something nonstandard is desired.
  7. Provide easy rollback in case of a failure. Test the rollback every time. You should keep track of original system state, and include an automated “undeployment” process for returning the system to that state. This should, just like all other operations, work on both the test machine and your own machine, without any special case handling necessary. Provide opportunities for custom configuration of the rollback procedure, and for some sysadmin you’ve never met before to use only parts of the automated process in combination with manual changes if necessary, but make sure that no user interference is actually required for the rollback procedure.
  8. Follow these guidelines every time. You may think a given project is not big and important enough to bother, but if it’s small and unimportant that just makes it easier to follow these guidelines. I’m not perfect — I don’t always follow these guidelines myself. I should, though, and I intend to do so a lot more in the future. Practice makes perfect, and you definitely want things to be as close to perfect as possible — right?
  9. There’s no excuse for deployment failure. If deployment fails, it means you forgot to account for something. If someone tries (and fails) to deploy it on the wrong type of system, the problem is that you did not account for that type of system — not that the choice of system on which to deploy it was wrong. If some other piece of software changed things so that your deployment didn’t work, the problem is that you did not account for the potentially varied circumstances in deployment. You may not want to tell them that it’s your fault, but you should think of it as your fault anyway: even if it’s because the guy deploying it is an idiot, the problem is that you didn’t make deployment idiot-proof enough.
  10. Extra Credit: Voluntarily test-deploy other people’s software on your system all the time, and help them solve their deployment issues as much as possible — but without actually doing development for them unless it’s specifically appropriate for you to do so. Being on the other end of the developer/deployer divide can help you hone your sense for how to develop high quality deployment procedures for your software. The more you know about what your software is going to be doing, and the conditions under which it’s running — and the more you know about the problems of others’ software deployments — the better you’ll be at developing quality deployment procedures for your own software.

That all adds up to being a better developer with happier clients. While you may consider deployment an “admin problem”, I guarantee most admins and end users think of software that’s difficult to deploy as a “developer problem”, and whether you think that’s fair or not, it affects your success and reputation as a developer. After all, if it doesn’t deploy, it doesn’t work.

* We’re assuming developers qualify as “people”, here.

(Credit where it’s due: This was in part inspired by Justin James’ Time to improve application deployment and a comment to it by Wayne M. titled Deployment Philosophy.)

4 April 2007

the problem with GPL thinking

Filed under: Geek,Liberty,Metalog,Popular — apotheon @ 12:11

I’ve noticed an influx of new visitors to SOB this morning, leaving comments on and linking to my entry the decay of the Debian distribution. It’s kind of a strong title, as one commenter pointed out, but I think it fits as one person’s view of what’s going on. I can’t help but wonder if what looks to me to be a first year of a slow death (and oh, how I hope I’m wrong) is somehow tied in with the controversies over paid release development that arose in the Debian community last year, and the rush to get a new Stable branch release out a mere 18 months (give or take) after Sarge hit the Stable release.

Among the sources of incoming link attention is More rants, a second coffee from a weblog called Planète Béranger. Beranger himself quoted my brief mention of my discomfort with the GPL as the point he found most interesting in my SOB entry, which kinda warms the cockles of my heart — it was a throw-away statement, but in some ways I think the most important part of what I said. Predictably, between that and his comments about GPLv3, some people took issue with his position in comments posted to his own weblog entry.

The very first sentence of the very first comment, in fact, was “One thing to remember about the BSD license is that people will exploit it for their own benefit.” This is a common sentiment, and one that I find not only silly in practice but downright disgusting when one starts examining the psychological roots of that sort of comment. Here’s the bit that immediately followed that sentence:

From time to time Theo de Raadt will rant online about the large corporations who profit from his hard work (things like OpenSSH) and don’t give one penny to help fund these projects. Granted, Theo has a hard time not annoying others as he is very opinionated and does not care how others perceive him. The point is, the spirit of Absolute Freedom associated with a BSD style license can work against a project. Money aside, the idea that software can be better through a community effort might sound like nothing more than a pipe dream, but I believe that this software development model can and will work. A BSD style license creates an opportunity to fracture community development. I don’t believe in an altruistic human nature (at least not in our daily lives).


Basically, what the originator of these words (identified as “Patrick”) seems to be thinking breaks down thusly:

  1. Theo de Raadt, as the head of the OpenBSD project, is in a position to know how bad things are on the BSD side of the fence.
  2. As evidenced by Theo’s occasional rants, people are taking ideas from open source projects and not giving anything back. Shame on them.
  3. The BSD licensing model is obviously broken, based on this evidence.
  4. The GPL is the One True Community Open Source Project License.
  5. Community development falls apart at the seams with that weakling BSD license.
  6. Because humans aren’t naturally altruistic (at least in their daily lives), we should force them to behave as if they were with regard to use of open source software.

Obviously, I’m using a little bit of hyperbole to make my point. I’m sure that Patrick will/would take some issue with my paraphrase, but my response to that is simply that if that’s not how he meant it he should have been more careful with the composition of his sentences — and I suspect that, once all is said and done, he’d either recant or end up being shown to hold those opinions to some nontrivial degree anyway. Seriously. Compare my breakdown with the implications of the quote. See if you don’t agree.


Here’s my take on those points of the breakdown:

  • Theo de Raadt is one person with some very inconsistent ideas (in that sense, bearing a glancing similarity to RMS). For instance, thanks to his guidance, everything in the OpenBSD community is about openness, except the format of the official installer CD image. No, that’s proprietary, “owned” by Theo, and you have to pay for it to acquire it legally. Yes, really. Imagine that. Gee, and people wonder why OpenBSD isn’t more popular. Based on that I’m really not at all surprised to discover that Theo seems to think someone owes him a living.

  • People make use of open source software of every stripe, including GPLed software, without “giving anything back” in terms of development or money. So what? There are a whole lot of people reading these words without clicking on the ads on SOB or posting thoughtful comments, too. News flash: I don’t hate them for it. I license all my original content on this site using the CCD CopyWrite license (edit: These days, I use the Open Works License instead — which is even more permissive than the CCD CopyWrite license.), not because I want some quid pro quo or something, or because I’m too stupid to realize someone will likely use my words without giving me money or contributing beneficial modifications back to me, but because I have a real, well-reasoned belief that it’s the ethical thing to do — that the only claim I have on the product of my intellect once I convey it into the minds of others is due credit for originating the ideas. I’m not interested in cynical, scam-like ploys to “force” people to “contribute” something. Ultimately, people should decide for themselves whether what I’m creating is valuable enough to support it somehow — or whether there is some other value to be gained from “contributing” somehow. More on that in the last point of this list.

  • The BSD licensing model is not broken. It works great. There are strong communities for both FreeBSD and OpenBSD, and since (at least in part) abandoning its focus on portability the NetBSD project seems to have ceased to maintain the same strength, probably because people aren’t seeing as much point in moving to NetBSD as they used to. That’s the way it goes. Technology evolves, and the bits that don’t evolve well die out. That’s the way it should be, as opposed to the proprietary model of “The law evolves, and technologies that cannot manipulate that die out regardless of their intrinsic value.” Good. That means the system works. Freedom works — freedom for developers, distributors, and users, rather than freedom for code with the developers, distributors, and users leashed to the code (which is more the FSF/GPL model). If anything, it’s the GPL that’s broken as a community licensing model, which I’ll address in the next two points.

  • Obviously, the GPL isn’t the One True Anything to anyone but the faithful of the Church of the FSF. The zealous pushing of radical anti-corporatism (which isn’t necessarily bad — I’m of the opinion that corporate law is a direct violation of the principles of free market capitalism) actually alienates many potential users and contributors for free/libre/open source software projects, and causes legal problems for community software development and distribution efforts. In short, the GPL is broken by design, as proven by the simple fact (among many) that the GPL actually violates the FSF’s own Freedom 2 (the third freedom) of the four freedoms it claims are necessary to actual “software freedom”. Yes, really.

  • I don’t know where people get the idea that the BSD license cannot allow a community to hold together, or that it somehow opposes the community building spirit of open source projects. The FreeBSD and OpenBSD communities are stronger (in terms of the actual community integrity, though not necessarily in terms of numbers) than most Linux distribution communities by a pretty wide margin. Even better, in the BSD community, you don’t see the BSD cops running around threatening small grass-roots *BSD projects with lawsuits for obscure violations of half-baked anti-corporate clauses of the BSD license. On the other hand, you do see the GPL cops running around threatening small grass-roots Linux distribution projects with lawsuits for obscure violations of half-baked anti-corporate clauses of the GPL. The FSF, with the GPL as its weapon, is so intent on its idea of the One True Way that it’s willing to destroy people’s attempts to do good work in its tunnel-vision focus on destroying all proprietary software. I’m for eliminating proprietary, closed source software development, and for ethical as well as technical reasons, but unlike the FSF I take a “first, do no harm” position on the matter and do not in any way condone collateral damage to the “good guys”. I do not subscribe to the “to make an omelette you’ve gotta break a few eggs” notion of utilitarian, collectivist ethics that seems to guide the FSF hand. Collectively (pun not intended in this case), the FSF, the GPL, and RMS may be the biggest threat to an eventual ethical solution to the problem of software copyrights and patents, in the “good enough is the enemy of perfect” sense — it’s the de facto flagship for open source software licensing already, and as it gets more entrenched it continues to make things worse for open source development even as it provides a focal point for open source software mindshare.

  • I don’t even want people to be altruistic. Altruism, as an ethical or moral system, is broken by design (there’s that term again): it guarantees that the “bad guys” win, because the “best” people — the most altruistic — are the losers every time, by definition. I don’t want people to provide me with stuff, whatever that stuff may be, at their own expense, out of the sheer goodness of their hearts. I want them to gain value for what they give me, whether it’s helping to propagate ideas of mine with which they agree, promoting more thoughtful exchanges, or some tangible benefit. In the case of open source software, the first and primary benefit we gain from contributing is that the software we use without paying licensing fees and the like continues to improve. Good! Idiots who think they’re “getting away with something” when they copy open source software source code into proprietary, closed source software and try to sell it are actually doing themselves a disservice, cutting themselves off from effective widespread community help with further development. It’s poetic justice, and they’re welcome to it. I’d rather people give me what they feel it’s in their best interest to give than that they feel forced to give something to me by the metaphorical (and, ultimately, very real, if somewhat abstract) gun to their heads of copyright law.

So, people exploit BSD-licensed code for their own benefit. One response: Good! Would you prefer they were forced to use it for your benefit, instead? There’s a term for that — “antisocial personality disorder”.

Another response: If you really think that the world is going to come crashing down around your ears because your open source license doesn’t have a “pass on the source code OR ELSE” clause, you haven’t spent much time in the *BSD communities. The point here is to encourage cooperation and contribution, not to punish those who don’t want to cooperate and contribute. At least, that’s how I view it. This brings me to the part that disgusts me about FSF-ish attitudes toward open source software:

I find spite utterly without redeeming quality. If there’s real evil in humans, spite is a good-sized chunk of it. The impulse to somehow visit punishment upon people who use your software creations, and pass them around to others, without passing on source code is spiteful, pure and simple. That’s all punishment really is, in general. Oh, sure, you can use it as a stick to deter people from behaving in a way you don’t like, but if the end result of that is to do harm to those that behave as you do like, as well, then your stick is being swung far too wildly. I’d rather that a hundred eeeevil corporations “get away with” using a fork of OpenSSH in their proprietary software offerings than that a single end-user find that passing it on to someone else as a friendly gesture without source code has landed him in court. Why not work on encouraging good behavior rather than trying to force it at gunpoint? From a purely engineering-based perspective, weighing the positives against the negatives, this should be a no-brainer. If you want to give something away, give it away. If you want to control how other people can use and distribute it, start a corporation and use copyright and patent lawyers to enforce your will along with the rest of the ethically unsound jackasses out there. Your spiteful focus on hurting people because they didn’t adhere to your strict rules for source code distribution makes my skin crawl.

Besides, all this nonsense about the BSD license being roughly equivalent to the Public Domain in the ability of eeeevil corporations to bury it in proprietary software is obvious poppycock. Check out the PDF, “Reading the BSD License in Isolation”, at Open Source Law of Sydney, Australia if you don’t believe me. The ideas in it about forced contribution of source code are stretched pretty thin, I think, and wouldn’t hold up in court (thank goodness — don’t need another GPL), but the rest of it about copyleft, et cetera, is excellent reading and appears to be spot-on. (IANAL, but I’m not a complete legal dunce either).

Older Posts »

All original content Copyright Chad Perrin: Distributed under the terms of the Open Works License