Chad Perrin: SOB

31 May 2007

10 tips for developing deployment procedures (or: Deployment Is Development)

Filed under: Geek,Popular — apotheon @ 07:56

People* have a tendency to think of deployment as something the admins do — if developers put any thought into it at all, it’s usually an afterthought, and not one that is considered very important. That is, of course, a huge mistake: deployment is a development concern, too.

  1. Have at least two (types of) boxes on which you test deployments at every stage of development. One should be an example of a standard machine in the production environment, and the other should be whatever close approximation you have as part of the development environment. That means that during development of a desktop application, for instance, you should at every major development stage test it on both an end user desktop system and on your own development workstation (or close equivalent). Make sure the thing deploys under wide-ranging circumstances, and ensure that if it doesn’t deploy properly during testing it hurts enough to make you consider deployment matters very carefully. For independent consultants, this may mean requesting a test box from the admins at whatever organization is paying you for development.
  2. Make sure you actually test deployment at every stage. It is becoming ever more fashionable to develop in stages so that at every interim goal you have something that actually runs when it comes time for testing and project specification assessment. When those interim goals are met, make sure you deploy and test as though it were the final deployment to the production environment, and make sure you don’t skip either the production environment test box or your own box in the deployment test (though of course you might want to try the test box first in case it blows up). Don’t consider the milestone reached until it deploys smoothly to both.
  3. Make sure deployment/installation is a largely automated process. Use scripts where necessary, employ the system’s software management applications for deployment where at all possible to integrate it with the environment and reduce confusion, and ensure no steps of deployment are missed. Have a complete, functional, automated installation process working and tested before you declare your interim goal met.
  4. Use automated testing scripts to verify that everything is where it needs to be after a test deployment, and make sure that it’s an automated, idiot-proof verifier so you can incorporate it in the final product. Ensure that, at minimum, it logs any discrepancies it finds so that you have a record of what went wrong during deployment, providing you with a neat and orderly to-do list of things to fix. Make sure that the end-user or sysadmin that will be doing deployment can use the verification system without your help, since somewhere down the line someone you’ve never met will be doing exactly that — guaranteed, assuming your software is good enough that it’s used more than once. Your verification should check locations, version numbers, and file permissions. Integrate your testing scripts into the automated deployment process. Actually use it yourself for test deployments, even on your own machine, and design it so that’s possible without a bunch of special-case handling.
  5. Make sure real human beings test the software after every test deployment to make sure it operates as expected at this stage of development.
  6. Keep necessary configuration to a bare minimum. Best of all would be to have no necessary configuration options at all, with the possible exception of asking whether custom configuration options are desired. Provide opportunity for custom configuration, though, in case something nonstandard is desired.
  7. Provide easy rollback in case of a failure. Test the rollback every time. You should keep track of original system state, and include an automated “undeployment” process for returning the system to that state. This should, just like all other operations, work on both the test machine and your own machine, without any special case handling necessary. Provide opportunities for custom configuration of the rollback procedure, and for some sysadmin you’ve never met before to use only parts of the automated process in combination with manual changes if necessary, but make sure that no user interference is actually required for the rollback procedure.
  8. Follow these guidelines every time. You may think a given project is not big and important enough to bother, but if it’s small and unimportant that just makes it easier to follow these guidelines. I’m not perfect — I don’t always follow these guidelines myself. I should, though, and I intend to do so a lot more in the future. Practice makes perfect, and you definitely want things to be as close to perfect as possible — right?
  9. There’s no excuse for deployment failure. If deployment fails, it means you forgot to account for something. If someone tries (and fails) to deploy it on the wrong type of system, the problem is that you did not account for that type of system — not that the choice of system on which to deploy it was wrong. If some other piece of software changed things so that your deployment didn’t work, the problem is that you did not account for the potentially varied circumstances in deployment. You may not want to tell them that it’s your fault, but you should think of it as your fault anyway: even if it’s because the guy deploying it is an idiot, the problem is that you didn’t make deployment idiot-proof enough.
  10. Extra Credit: Voluntarily test-deploy other people’s software on your system all the time, and help them solve their deployment issues as much as possible — but without actually doing development for them unless it’s specifically appropriate for you to do so. Being on the other end of the developer/deployer divide can help you hone your sense for how to develop high quality deployment procedures for your software. The more you know about what your software is going to be doing, and the conditions under which it’s running — and the more you know about the problems of others’ software deployments — the better you’ll be at developing quality deployment procedures for your own software.

That all adds up to being a better developer with happier clients. While you may consider deployment an “admin problem”, I guarantee most admins and end users think of software that’s difficult to deploy as a “developer problem”, and whether you think that’s fair or not, it affects your success and reputation as a developer. After all, if it doesn’t deploy, it doesn’t work.

* We’re assuming developers qualify as “people”, here.

(Credit where it’s due: This was in part inspired by Justin James’ Time to improve application deployment and a comment to it by Wayne M. titled Deployment Philosophy.)

All original content Copyright Chad Perrin: Distributed under the terms of the Open Works License