The Case for ... Investigating

Lee Whitfield posted on his corporate blog earlier today about reasons not to bring cyber investigations for litigation (specifically forensics and eDiscovery) in-house, in response to this article making a case for the opposite. I replied on twitter, and was promptly lambasted by Kyle Maxwell for not blogging about it instead. You know how it is, twitter just doesn't provide a good platform for detailed response, and Kyle seemed to feel that sending several tweets was inappropriate. If it went any further, I think he would've called me old and feable. Again. :( So here's my detailed post.

I think (hope) that my perspective is somewhat unique, as I have been on both sides of the fence - in consulting and corporate roles. I've spent a lot of time scoping these types of matters, talked with GCs, OCs, IT, InfoSec, and eDiscovery folks. Full disclosure, I worked for several years at the consulting firm where Lee now works (although I've never had the pleasure of working with him), and in my current corporate position I'm responsible for building out the in-house programs. Lee's company focuses on reducing datasets so that eDiscovery costs are lower. That service is nowhere near the eDisco costs, but it ain't free either. ;) I was brought on where I am now specifically because they wanted the internal capabilities, having dealt with the difficulties of not having that kind of expertise in-house, and the tremendous cost of paying outside consultants and vendors.

In the original article, which was based primarily on an interview with Greg Thompson of Scotiabank, the essence of Thompson's argument is that it's totally worth it. He stated that they can easily spent $2000/day with an external vendor, compared to internal costs of $800/day. That's a pretty obvious ROI. My guess is that they're talking about an all-in-one vendor that collects, processes, and produces the data, with reductions coming only from deduping and deNISTing, and that probably occurs only after loading in the eDisco software.

Lee has a three-point rebuttal that centers on: Cost, Impartiality, and Skill set (not Skil saw, that's very different). Lee's company is not an eDisco processor, they're a forensic shop whose bread and butter is large-scale collection and data reduction for litigation. No, I'm not selling them, but as I said, I used to work there, so I'm very familiar. Because of the culling process they employ, data sets can be significantly reduced compared to what was collected, with associated lowering of eDiscovery processing costs.

My basic response to Lee was, "It depends." Trying to expound on that in twitter is pretty much fail, so I'll attempt to do so here. Basically, you've got a "party line" on either side. No, I'm not talking about the old phone system where you could pick up your home receiver and hear your neighbors' conversation. I'm talking about the territorial/turf war approach. In general terms. The consultants say if you're not out-sourcing, you're not doing it right and risk sanctions. The corporate folks say that they don't need somebody coming in "administering" their network and charging money for something they can do just fine. From my experience on both sides of the fence, here's how I think it breaks down.

Cost:
In a nutshell, Lee points to the salary of the kind of experienced expertise you'll need, software/hardware, training, and certifications. That's true, it's not cheap to get that kind of personnel. There's a couple things with this though, that I think bear more discussion. As with any such position (even in consulting), it's probably not a dedicated role, so the actual percentage of salary that applies to the forensics/eDisco work is not anywhere close to the six figures Lee mentions. Anyone in IT (much less InfoSec) is going to require ongoing training and certifications, and any employer that places value on professional development will support that anyway. So that only leaves us with hardware/software, which up front may be a sizable layout in cost, but will pay for itself very rapidly. How so? Well, a single case with 30 or more custodians could quickly cost over $100K. If you have one or more of those per year, your internal programs are covered. ROI's easy there.

Impartiality:
This may - in my opinion - be the best argument, up to a point. The corporation is paying the internal resource's paycheck, so those individuals have to support the corporate position, right? Not a bad assumption, but the same can apply to a consultant firm - they're paid by some organization to act in support of that org; if they don't give "good" results, they're out, right? So that knife cuts both ways, I think. But to the original point, I think it comes down to ethics of the investigator, just as with any case; we have an ethical, professional, and moral responsibility to do what is right, no matter what. Since the core of our work is based on facts in evidence, this shouldn't be an issue (at least not theoretically, but again, that cuts both ways). I think in most cases, an internal investigation is acceptable; there may be times that is different, and those should be addressed accordingly. The company - and its investigators - need to be able to determine when it may not be appropriate for the investigation to be handled internally. I know Kyle has mentioned having to deal with that where he works.

Skill set:
I'm a little confused on this one, to be honest. Lee says that most in-house investigators come from security or investigative backgrounds, discusses that network forensics has little to do with host forensics or eDiscovery, then goes on to say that while having "IT" staff involved, they shouldn't necessarily collect data themselves, as that could stomp on its evidential value. Okay, that's a long sentence, and a paraphrase of several combined. My confusion comes in from his starting out talking about computer security, network security, investigation background, and network forensics, then pointing out that IT staff aren't trained to know about file system changes, timestamps, and so on (all the yummy metadata stuff that forensics thrives on). I don't disagree with the latter, but I don't see the correlation with the former. The former is more the Incident Response (IR) type, it seems, and in my experience those folks are rather well versed and cognizant of maintaining evidence integrity (such as all that yummy metadata) and chain of custody. Pure IT folks - sysadmins and such - not so much; that's not to place blame, it's just not their area of expertise.

So here's my summary:
If your company is under regular litigation - large or small - and perhaps if you have regular threats to your intellectual property (thinking internal threats here, not external), it may well be a wise move to look at developing in-house capabilities. You need to really take some time to determine your internal needs and requirements, and remember these matters are about more than just email (systems, network, database, etc), and you must have a good grasp on your environment variables. You need to determine how much of the process you want internal - you may still want to outsource final production and hosting, for instance. Make sure you get the right expertise, and be aware that there will be an up-front cost (ongoing costs for software, hardware, training and certifications are minor in comparison). But the savings can be significant, and it is possible to come out ahead, if you compare against the money you would have been spending with outside vendors. ROI, the language of C-levels... :) Bottom line is, be informed, and make intelligent choices - don't just take action based on what either "side" is telling you.

I do think it may not be the best decision to try to convert your IT staff. In years of dealing with IT departments, and knowing how those personnel tend to think/approach these matters, your up front difficulties and costs are much higher, and you have a much steeper "learning curve," if you will. It pays to get someone who already knows how to do the work, has solid experience, and I would even add, has provided expert testimony in court. That is the bottom line for this field, whether one - and one's work - stands up well in court. But do be careful, as not all consultants are suited for corporate life; it's a different style of work, and you need someone for the long-term, not short-term, or your ROI decreases. You also don't want a "push button" forensics person, but someone who truly understands what's going on behind the scenes; they're going to be able to provide better development and support for your internal programs.

Let's face it folks, litigation isn't going away, nor is electronically stored information (ESI). Thus, ESI will have to be produced in litigation, and in comes eDiscovery. Orgs large or small feel the sting of the associated costs (which seem to be rather unreasonable at times), and - just being realistic - people are going to look for ways to bring it in-house. Sometimes that's just not feasible, and in those cases I think it's important to look for help in pre-culling to reduce costs. But for many organizations, having an internal program makes perfect sense and is not a mistake - when approached carefully and done right.

Okay, I think that's about it. No tech stuff this time, sorry to disappoint those who might've hoped otherwise.