Eric Huber recently posted an interview with Hal Pomeranz on his blog, here. I'm not typically a person who reads interviews, but I had Hal as one of my SANS instructors. I think Hal's one of the many sharp forensic folks out there, and I've certainly learned a lot from him. So anyway, I was quite a bit more interested in “hearing” what he had to say. And because of this posting of Eric's, I've actually started reading more interviews. So hey, a positive impact!
I thought the interview was very good (thanks, Eric!) and there were a lot of good take-aways in there for me, and the company where I work (a boutique firm where we work primarily for lawfirms on behalf of corporations). If it was that good for me, then surely it's good for others, too. So I thought I'd share my – well, for lack of a better word – thoughts on what was said. I've been working on this post ever since the interview first came out (I know, it's taken a while – I'd like to blame my work schedule, if I may). I first wrote a version of this for my company (in a spreadsheet), as I thought we could (and should) take some of these things to heart. Obviously, I've made some changes to post to my own (infrequently updated) blog.
Eric's questions weren't numbered, but I've taken the liberty of referring to them as if they were, for ease of making sure I've covered my points, and so that it's easier to track back to the original.
1. Not much here. Eric asked about Hal's journey into forensics, and Hal mentioned being approached by Rob Lee to be part of Mandiant's Surge Staff. I just think it would be cool – an honor to work with Mandiant. Perhaps some day; a guy's gotta have dreams, after all … ;)
2. This question was about Hal's involvement with SANS' Forensics program, and Hal talked about how challenging it is to prepare to teach the material. I've seen the same thing in the military and business - when you have to prepare to teach a subject, you learn more about it than you probably ever would by simply taking the class yourself, or doing the work. Teaching is hard work, but obviously the rewards are extremely beneficial in this respect. In addition, getting outside of your environment and teaching (not just outside your comfort zone) helps establish your presence in the community at large (both personally and professionally).
Teaching others isn't just about personal (or even professional) accolades. It is an opportunity to learn even more about the craft/science we use, and giving back to the community. Forensics has developed as a community-driven discipline, because there aren't rulebooks, documentation, etc - we are the ones paving the way. As we benefit from the community, we really have a responsibility to give back, or "pay it forward" as Hal mentions. It's also a way we make connections in the industry which could lead to business associations as well. Later on, Hal emphasizes the importance of giving back to the community - doing blogs, whitepapers, articles, etc. I think that plays a part here as well. You don't have to be a Harlan Carvey, Didier Stevens, or Hal Pomeranz to contribute. Simple blog posts about the things you find can be useful, as you may have come across something somebody else hasn't seen before, and you may be the only person (or one of a very few) reporting on it. There's at least one blog I know of where the guy is new to forensics, but he's doing a lot in Linux and posting findings on his blog.
3. Eric asked about skills Hal has developed over the years that help with forensics, and Hal talked about tricks and other things he learned from his years as a Unix Sys Admin, scripting, and problem solving. This whole section speaks to what I think is really part of what goes into our team (where I work). We have varied backgrounds, different business paths, and so on - these all help shape our skills, troubleshooting/problem-solving abilities, and so on. We don't just think from one perspective; even if a single individual did (and I don't think we do, given what I've seen), the "group mind" takes it much further as we collaborate and share based on our personal experiences. Together, we are greater than the sum of our parts, and I think this is one of the things that really sets our team apart and allows us to accomplish more than a lot of others in the field.
4. There are several things in this question about what it takes to be a good forensic examiner. Hal mentions root cause analysis, writing skills, and passion for the gig. Rob Lee has a great article out there discussing what it takes to get started in digital forensics (http://computer-forensics.sans.org/blog/2010/08/20/getting-started-digital-forensics-what-takes). IMHO, it's a very good writeup. One of the things I took away from it is that he looks for a drive (no, not a hard drive) in the person to accomplish new things. He gives the example of a person who decides to learn to play golf one summer and devotes time & effort to doing just that. He said he asks candidates what they've done along those lines and uses that as a barometer to gauge their forensic potential. (Section: "Do You Have A Desire To Be An Expert?") I don't think it's so much about "what you know" as what you want to know and are driven to learn. You can have a good IT person who knows a lot about computers but would not make a good forensicator, as they might not really want to learn more, or get "outside the box" for their current set of knowledge.
The whole writing thing, well this is a bit of a personal pet peeve. Well, part of it anyway - the coherent documentation bit. A lot of the forensic reports (and white papers) that I've reviewed are, in my opinion (humble or otherwise), drivel. I think people are in way too much of a hurry, don't take time to do error-checking, editing, grammar/syntax correction, formatting, or making sure they are communicating their point for the intended audience. There also seems to be a general lack of understanding regarding some basic grammar rules (run-on sentences, for instance, is a big offender). Then beyond that you also have to take the target audience into account, as Hal mentions. Unfortunately, I'm not sure that quality report writing is something that can be taught; it's like the "silver tongue" - either you have it or you don't. Anyone can improve their skill, but to what level? And some, I think, will not ever have it (something that needs to be watched for in the hiring process).
5. The question about how to get into digital forensics. Yeah. There's been a pretty good discussion on one of the LinkedIn groups about this, but Hal hits some different points, focusing primarily on networking. Not sure what a switch and a CAT6 cable will get me but... This section is where Hal mentions doing your own blog posts, articles, etc. He's approaching it from the standpoint of getting your name out there, looking for jobs, and so on. However, this works not only for the individual, but to help promote the company too. I know Digital Detective and Guardian Forensics, for example, both host their own blogs. I understand, though, that if you don't keep it active, you just look foolish as a company (does that count for individuals, too? Thinking about my own here...). He also emphasizes the importance of continuously learning. Again, there aren't any all-in-one rulebooks or documentation for our corner of the InfoSec industry. Operating systems change rapidly; these change available artifacts in ways that are not typically documented. With each change, we have to figure out what to do next. In addition to these "obvious" changes to the landscape, cell phones are now really miniature computers that also make phone calls - not to mention tablets brought back by the iPad. Their operating systems are completely new things, with new data structures, artifacts, and so on. Not only do we have to figure them out, we also have to figure out how to access the data (think Android, iOS, etc) such that we can forensically acquire it in a useful manner. If we don't keep up, we lose...
6. I don't have a lot of input here either, for the question about seeking advanced education or degrees in digital forensics. I'm not convinced that the current programs are actually preparing graduates for the real world, but then that's probably true for any degree plan, and I also have not by any means seen everything that's out there.
7. Question 7 focuses on the “state of the union” so to speak, regarding forensics. The first three things Hal brings up are our general youth, developing solutions to problems, and sharing information. This ties in with my point about us paving the way, and the need to continue learning. Microsoft and Apple, for example, are certainly not in the business of providing information to the forensics community about how their various operating systems work, what changes are being made that would impact our professional lives, and so on. For example, as browsers are designed more and more to "protect" users' privacy, the storing of browser artifacts is something that has changed fairly dramatically in recent months. When it comes to things like the iPhone, as people figure out how to hack it for various purposes, Apple tries to block it in a new release of the iOS. Can't fault Apple, as they're trying to protect their users and interests; however, as we are often looking to utilize the same exploit, this negatively impacts our ability - and then Jonathan Zdiarski has to come up with a new script or process... ;)
From there Hal goes into cross-platform tools. He and I had an email conversation about this very thing a little while back. He'd posted some questions about Mac forensics/tools on the GCFA list, and did not get a lot of responses. I let him know about the tools that we've used, and we had a little back-and-forth off-list. One of the things we touched on then was how it seems the majority of tools for Mac forensics only run natively in Mac OS. While this may be a natural starting point, it can be difficult for folks who don't have immediate access to these platforms, but have reason to need to do analysis.
The final area in this question relates to, as Hal puts it, “bridge-building.” I just think this is a very good point - educate the legal community on what we do, and how we can help them. If we also become more educated on what they need, we will in turn be better prepared to help them in a meaningful way. As Hal mentions later on, he sometimes finds stuff he thinks is cool or interesting, but it may have absolutely no value for the client. I think it's important for us to step out of our comfort zone (or simply take our blinders off) to get a better grasp of what our client really needs (as opposed to what we want to give them).
8. The question here is about difference between IT- and LE-based forensics, and I think Hal has some excellent points (but of course, right?). We tend to get wrapped up in our own "little" world and don't realize that our clients are juggling a lot of projects as well. They may not remember exactly what we told them before, and may have some difficulty grasping the points we're trying to make. A little consideration will probably go a long way to help create a stronger, more mutually beneficial relationship. We should have a "take-away" focus to help us learn how to better do our jobs (not just the way we've always done them).
The second half of this brings up rabbit trails, and maybe a little scope creep. This is one of the biggest issues I think we face, when it comes to actual investigations (by “we” I mean our company in specific, although I realize this may have broader applicability). We are bad about going off on our own tangents, "just because we want to know." Maybe this is useful from the standpoint of educating ourselves, and if we aren't billing the client, what's the harm, right? But if it detracts from us meeting goals/deadlines that the client has, then we're doing everyone a disservice. We have to stay on task, focused, and try to make sure that we are meeting the client's needs. Sometimes I think where this starts is we see something the client hasn't mentioned, or says they don't think they're interested in. We *know* that it is valuable, and that they *need* the info - so we go digging to get it, to have it ready once they "come to their senses." This is probably not the best approach, and we need to do a better job understanding the client's scenario, what their end-goal is, and so on, so we can make valid recommendations to help reach that goal. We also need to do a better job helping the client understand the value of what we are offering - our expertise and potential testimony on the stand. After all, we're a consulting firm; we should *consult*.
9. Aah, scoping, planning, and then communications with the client. Okay, here goes... This ties in with the comments above. Our approach to a situation, the deliverable we're providing, and so on, should really be tailored to the individual needs of the matter. If we have a "fit in the box" approach, we do everyone involved a disservice. From this standpoint, there are probably times (however much we might not want to do so for whatever reasons) that we need to customize (or completely change) our User File approach (we have a proprietary process that starts with deNISTing and goes much much further; this is used in pre-culling for eDiscovery projects). One potential example that comes to mind is if a matter only stipulates that email, or MS Office documents, or PDF files are required, it might be reasonable to only produce those data types and save the client time and cost. Obviously, as Hal points out, communication with the client is key.
This is also true for investigations – while it's good to have an idea of a “standard” approach to a different type of investigation (misuse of resources, theft of IP, spoliation, etc), we need to make sure we understand what the client needs to accomplish, so we can tailor our approach to their needs. This way they get the benefit of the best possible results, and we don't waste time doing things that aren't going to help reach the goal.
So then we come to the communications part of Hal's response. Let's rub some more salt on that wound, why don't we (again, where I work we seem to struggle with this; some analysts don't seem to want to have a lot of interaction with the client)? We as analysts need to be better about direct communication with the client. There's a time to involve the account/project manager, but no one is better prepared to communicate technical details of our investigation than we are. And it's not just the aspect of making sure the client knows how we're progressing - it's the personal touch that makes a difference. The analyst, having ownership of the investigation, is able to make a personal connection with the client, and the client is going to remember them down the road (as Hal notes). Case in point, I have one attorney that sends me Christmas cards every year - and calls our company for more work - because of the relationship that developed while working on a case with him. And yes, I'm proud of that.
10. Wow, there's a lot here in this last section, and it is stuff that can probably get a lot of folks all fired up. I remember a while back we were asked to work defense on a criminal case. Regardless whether we wanted to turn it down or not, we were "required" to do so as a condition of our director's membership in the HTCIA - no criminal defense is allowed. I think Hal has a valid point in that everyone in this country is entitled to a defense, and it doesn't really seem "fair" that they be prevented from doing so by not being able to access the same resources (ie, forensicators) that are being brought against them. Obviously that gets into the political realm, but aside from that I think the most salient point he has is about scientific fact-finding being the foremost goal, not personal or professional agenda. We've certainly been involved in cases where the other side brought in their own experts, and a most contentious fight ensued as they weren't willing to admit that we were doing what needed to be done. These aren't criminal matters (since we don't typically work in that space), but the point is still the same - shouldn't we be working toward a common goal of fact-finding, rather than fighting about semantics (or other irrelevant issues)?
I guess on the other side of the issue, we try our best to do a top-shelf job, being professional and just looking for - and presenting - the facts. However, I've seen a number of firms that appear to be hired guns without ethics or morals, who are willing to say whatever their clients request, for the sake of a buck. To me, that's just sad.
Eric and Hal, thanks for a great interview and blog post. I enjoyed it. I hope others have as well. Just to give you a quick plug, yours is one of the (many) blogs I follow (and Command Line Kung Fu, of course). (Google Reader on my phone is a tremendous help during dead time while imaging at a client site!) I look forward to seeing new posts.
LM
