Chad Perrin: SOB

21 August 2008

an interview question every IT pro should have to answer

Filed under: Geek,Profession,Security — apotheon @ 11:02

Just this morning, I published a new article at TechRepublic’s IT Security Weblog. I’m basically the “headliner” there, a twice-a-week columnist of sorts pondering the imponderables of information security as they apply to professional computer geeks and the world at large. In the terms used by the TechRepublic staff, I’m the “IT Security blog host”. If that’s a topic of interest to you, you might want to swing by and see what I have to say. Take note of the fact that though I’m the “host” (kind of a misnomer, but whatever), mine is far from the only name to appear in a byline. Examine each article for its author before assuming I wrote it. I maintain a mostly up to date list of IT Security articles I’ve written for TechRepublic, if you’re interested.

Anyway . . . this new article, How do you interview security experts?, takes on the question of what sort of questions one should ask a candidate for a security focused job. In addition to what I said there, however, I felt like offering the world a bonus example interview question here at SOB.

This question isn’t just for candidates for a security focused job, however. This is basically for any IT job candidate at all. Anyone with any degree of responsibility for the smooth operation of your IT infrastructure at all should have at least the most rudimentary understanding of computer security, and this question is something I would consider a must-ask for any IT job interview.

Unfortunately, the formula for this question kinda flies in the face of the advice I give in the article: it’s the sort of question that has a “right answer”. On the other hand, I wouldn’t just reject a candidate out of hand because (s)he gave a different answer. As suggested in the article, if I get an answer that disagrees with what I expect as a “right answer”, I would explain the answer I expected and ask why the candidate’s answer differed.

Without further ado, the question is:

What do you do if you discover your personal computer at home was compromised by malware or a malicious security cracker?

The “perfect answer” is something like:

I’ve never had to deal with that state of affairs on my own computer, but as I’ve done when helping others, I would probably just make sure my backups were up to date, ensure there’s no sign of infection or compromise in the backups, then nuke and pave the system, followed by restoring important data from backups. There’s just no way to be certain, once a system has been compromised, that there’s no further infection or compromise after cleaning up the known problem.

The otherwise “right answer” would be much the same, but perhaps without having actually done the above when helping others, or with personal experience involving the candidate’s own computer as an example of a compromised system in the past (hopefully not often).

Other variants on this approach surely exist, such as restoring the system to a previous known good state from a disk image maintained on another computer that has not been compromised. Keep an open mind for different ways of solving the same problem of uncertainty that the system can be cleaned up after infection; a candidate’s answer may not seem the same as yours at first blush, but may achieve exactly the same end result — or at least a security-conscious approximation. For instance, I wouldn’t necessarily exclude a candidate for failing to keep regular backups, as long as (s)he knows that’s a failure and mentions some reason that (s)he doesn’t regularly back up the system that I could reasonably expect would not translate to similar failures on the job. I knew an excellent accounting clerk who couldn’t manage her own finances worth a damn, after all; she knew exactly what she was doing wrong with her personal finances, and didn’t suffer the same failures professionally, which is sorta the point.

Assuming a wrong answer, like “I’d run some malware and rootkit cleaners to make sure the infection is gone,” I’d have to ask about why the answer wasn’t what I expected — maybe with something like this:

I would have expected you to say you would wipe the system and reinstall, in case the infections or intrusions you detected were merely the tip of a mostly undetectable iceberg. Do you think it is possible to be sure the computer is really clean of all security compromises?

If the candidate is not being interviewed for a specifically security focused position, an answer like “Oh, I hadn’t thought of that. Of course, now that you mention it, that’s what I’d do,” might be acceptable. This suggests not disagreement, but simply that the candidate had not had occasion to think the matter through completely, and is willing to change his or her approach when exposed to new information.

A candidate with a rigid and obstinately wrong approach is a bit more worrisome, on the other hand. If a candidate’s answer was something like “Oh, I personally own licenses for a full suite of Symantec security and recovery tools. I’d be able to clean it up, no problem!” I’d be tempted to say “Thanks for your time, but I don’t think you’ll be a good fit for our organization,” and show him or her to the door right then and there. Depending on how desperate I was for candidates, I might give one more chance to correct the problem by prodding a little more, but at that point the candidate may become suspicious that (s)he is answering incorrectly and change the answer just to try to get the job without actually believing what’s coming out of his or her mouth. In this case, I would definitely focus more on the “why” of any further answers rather than the “what”, because that’s the kind of answer that’s more difficult to convincingly fake.


  1. Crap — it happened again. Something in moderation that I intended to approve ended up getting deleted. I’ve really gotta do something about this.

    Much like the stupidities of Firefox development, WordPress never ceases to find new ways to piss me off.

    Back in WordPress 2.4.x, the moderation interface was pretty good — other than the clock cycles it chewed up and spit out. Now, the interface sucks, and it consumes CPU time even more voraciously.


    Comment by apotheon — 21 August 2008 @ 10:45

  2. “Nuke and pave” — definitely my approach. I had to do it to my own system only once (for an infection, that is) back in 1991. That’s when I learned not to stick any old floppy in the drive. But I’ve had several friends and relatives whose systems were compromised by email-borne viruses. My approach has always been (a) wipe and load, (b) install AV software with auto updates, (c) lecture the user on never opening unanticipated attachments or clicking links in email.

    Comment by Sterling Camden — 22 August 2008 @ 11:01

  3. Nuke it from orbit, it’s the only way to be sure. As stateless computing becomes more ubiquitous, I have a feeling we’ll see the end of the Window era of nuke the system and start over, and move more into an era of it’s always nuked, no matter what.

    Let’s not forget that with the advent of Citrix, many times we don’t need a thick client any more and can function simply with a thin client. I’ve gotten to the point where I just want to remove user stupidity from the equation and move into the work of security on the servers and leave the clients a a non-issue (more or less).

    Comment by jmgarvin — 22 August 2008 @ 12:39

  4. I’ve been infected by Sub-Seven a few times (by my friends and a malicious file), but never had an intrusion I knew of and while running Linux.

    Comment by Joseph A Nagy Jr — 22 August 2008 @ 01:56

  5. Erm, last comment was unfinished.

    And while running Linux I kept a pretty good eye on what was going on with my system. Especially with incoming connections.

    Comment by Joseph A Nagy Jr — 22 August 2008 @ 01:57

  6. Nuke and Pave is absolutely the way to go – in the corporate world or a perfect scenario. We can recommend to those we help that they nuke their infected systems and start over, but ultimately, it is the system owner who makes the decision. They give various reasons not to nuke – from poor back-up habits to “I lost my system disks”. I’ve cleaned more than a handful of machines after recommending a wipe. Of course, I tell the system owner the system should no longer be used for banking, paying bills, etc. I won’t guarantee a cleaning, ever.

    My own system was infected once, a few years ago. But I’ve always been in a position where I could nuke on a moment’s notice.

    Thanks for the question – I’ll remember to ask it the next time interviews come up!

    Comment by Robbi — 26 August 2008 @ 08:18

  7. Robbi:

    Thanks for commenting.

    We can recommend to those we help that they nuke their infected systems and start over, but ultimately, it is the system owner who makes the decision.


    They give various reasons not to nuke – from poor back-up habits to “I lost my system disks”.

    I think the technical term is “excuses”. I don’t really consider them “reasons” unless they’re actually reasonable. If there isn’t a good backup, that just means it’s time to boot off other media and back up everything that I can pretty much guarantee isn’t infected when backed up (e.g., plain text files or anything I can compare against a hash/signature to make sure it doesn’t contain any unauthorized bits). Then, of course, nuke and pave.

    This, of course, is in dealing with stuff over which I exercise final authority.

    I’ve cleaned more than a handful of machines after recommending a wipe.

    So have I. In fact, cleaning machines that should have been wiped was my primary source of income for a while — and, while I’m pretty sure all the machines I cleaned were actually clean when I was done (I was really good at it), I too wouldn’t give an actual guarantee.

    Thanks for the question – I’ll remember to ask it the next time interviews come up!

    You’re welcome. Anything I can do to help improve the security of the world (for a sane definition of “security”) is a win as far as I’m concerned.

    Comment by apotheon — 26 August 2008 @ 09:33

  8. I’m right there with both apotheon and Robbi: Sometimes people just don’t want a wipe and reinstall.

    If I’m getting paid to clean the system, I tend to be a bit more of a hardass about the merits of the nuke and pave approach, and that I won’t guarantee a cleaning, even when I’m sure it is clean to the best of my ability. Since I’m not a top-notch malware writer who also knows every vulnerability, the limit is quite obvious, regardless as to how good I might be.

    I tend to be a lot less insistent if the computer is never used for anything sensitive, however, I really don’t like the idea that something could be missed that may potentially propagate malware to other systems. I’m not using that system all the time, i can’t see how it behaves.

    Strangely enough, I find a fraction of the people I know prefer to just have an infected system nuked and paved. It may partially be from a security mindset, but I think that preferences may be more informed by some personality traits than they are by knowledge. It’s a non-rational preference to build new or to renovate (as opposed to a rational one).

    Comment by seanferd — 26 August 2008 @ 10:46

  9. Welcome to SOB, seanferd. I don’t believe I’ve seen you comment here before.

    Since I’m not a top-notch malware writer who also knows every vulnerability, the limit is quite obvious, regardless as to how good I might be.

    It’s not just that, of course — even a malware writer isn’t going to know whether someone else’s malware is squatting on the system because there simply isn’t any reasonable way to be sure other than by nuking and paving.

    As jmgarvin and Ripley put it, the best option is simple — nuke the site from orbit; it’s the only way to be sure.

    Comment by apotheon — 27 August 2008 @ 10:25

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

All original content Copyright Chad Perrin: Distributed under the terms of the Open Works License