About Andrew Hay

Andrew Hay is an information security industry veteran with close to 20 years of experience as a security practitioner, industry analyst, and executive. As the Chief Information Security Officer (CISO) at DataGravity, Inc., he advocates for the company’s total information security needs and is responsible for the development and delivery of the company’s comprehensive information security strategy.

Andrew has served in various roles and responsibilities at a number of companies including OpenDNS (now a Cisco company), CloudPassage, Inc., 451 Research, the University of Lethbridge, Capital G Bank Ltd. (now Clarien Bank Bermuda), Q1 Labs (now IBM), Nokia (now Check Point), Nortel Networks, Magma Communications (now Primus Canada), and Taima Corp (now Convergys).

Andrew is frequently approached to provide expert commentary on security-industry developments, and has been featured in such publications as Forbes, Bloomberg, Wired, USA Today, International Business Times, Sacramento Bee, Delhi Daily News, Austin Business Journal, Ars Technica, RT, VentureBeat, LeMondeInformatique, eWeek, TechRepublic, Infosecurity Magazine, The Data Center Journal, TechTarget, Network World, Computerworld, PCWorld, and CSO Magazine.

Interview with Harlan Carvey, Author of Windows Forensic Analysis

interviewAfter speaking with Harlan Carvey on several online communities we both frequent he agreed to be interviewed on Windows forensics and his new book:

How did you get into forensics?

I started in the commercial infosec arena as a consultant doing vulnerability assessments and pen tests. At one point, I started working for a company, and a forensics guy needed some assistance. With something of a security background and a clearance, as well as some technical knowledge, I helped out and began to see the other side of the coin. I began to see the early stages of understanding that Locard’s Exchange Principle applied to the digital world just as well as the physical world.

From there, I had opportunities to not only talk to and ask questions of folks performing “forensic investigations”, but I started performing my own incident response, and looking for ways to do my job better. From there, I grew into the forensics field.

What training would you recommend to those interested in entering the forensics field?

Right now, it’s hard to say much about “training”. Much of what’s needed is specialized…however, I believe that a great deal of what’s needed can be taught to anyone who is interested in learning. Training itself does not suffice…I believe that you have to have an interest in the field. Hand-in-hand with the training is some kind of standard that needs to be met…too much of the forensics field is considered an “art” at this point.

I would say, however, that a great way to get started is to have a broad base of knowledge and understanding of network computing. You don’t have to be an expert in writing TCP/IP-based applications, but you do need to have a fundamental understanding of network communications (peer-to-peer, client-server, etc.) as well as how operating systems and applications work.

What other books have you written?

My first book, “Windows Forensics and Incident Recovery”, from AWL/Pearson Ed was published in July 2004.

What makes a good forensic examiner?

Curiosity and a willingness to learn. No one of us knows everything. As a community, we know much more than any one person. A willingness to put aside your assumptions and fears, and a willingness to share your thoughts and knowledge with others, I believe, is paramount.

What should an organization look for when hiring a consultant or organization to provide forensic services?

I think that the primary focus should be on the focus of the consultant or organization. How important is it to the consultant or organization to understand and meet your business needs? Do they come in and tell you that they’re going to have to take your production systems down during prime operating hours, or do they look for other ways to meet or even augment your business needs? Are they simply interested in meeting your needs with purely technical solutions that don’t take your business needs into consideration? Also, how realistic and honest with you are they when it comes to taskings?

It’s also a good idea to see if you can talk to other clients and see what kind of track record the consultant or organizations brings with them.

Do you recommend any training in particular or additional books?

There are a number of books out there that are very good for learning; I’d start with “The Cuckoo’s Egg”, by Clifford Stoll. Eoghan Casey’s “Crime Scene Investigation” books, particularly the second edition, are very good, as is Chris Brown’s “Computer Evidence: Collection and Preservation”, Brian Carrier’s “File System Forensic Analysis”, and “Forensic Discovery” by Farmer and Venema. But it’s not about one book, or one set of books. A broad base of knowledge and information is extremely useful…so including “TCP/IP Illustrated” by Stevens, “Google Hacking” by Johnny Long and even “Corporate Espionage” by Ira Winkler will all have their benefits.

I see many people in the security field with past military experience on their resumes. If someone were looking to get into security, by way of military service, what branch and trade would you nudge him or her towards?

First off, I’d nudge them toward the military, in general. Military experience exposes young people to new things and forces them to deal with things they may never have encountered before. Ultimately, these become “war stories”…we hear them all the time; good bosses, bad bosses, experiences, etc. Having life experience is important, as is encountering a variety of different experiences, as security (in general) puts us in a position where we have to work in the face of potentially adversarial conditions. In such conditions, you have to remain calm and professional.

For specific branches, I’ve always been impressed with the emphasis the Air Force places on off-duty education when it comes to promotions. However, most experiences are what you make of them, so working in computer networking in any capacity, in any branch, is a great place to start. Learning to solve problems and adapt to situations is important in consulting.

What about those who start in civilian life?

Pretty much the same thing, but without the military component, obviously. Working in networking, starting out as a junior helpdesk or sysadmin is a great way to get started.

What is your opinion on technical training and certifications as a method for preparing people for forensic work?

Technical training and certifications have a good basis in getting people prepared, but in my perspective, they are only good if they are actually used and evaluated. In the Marine Corps, every Marine receives training in the care, feeding, use and deployment of their weapon…the M-16 service rifle. Even in peace time, Marines further receive annual refresher training. In the civilian world, I am often called on-site to assist in incidents, and see certificates on the walls of the cubicles…people are sent away for training, receive a certification, and then not only do not use it, but are not evaluated on the use of the knowledge or training by their managers.

If there is one person in our industry whom you would like to meet, who would it be and why?

I don’t like to name drop, but I’ve had the great fortune of meeting many people in the community, however briefly. The one person that comes to mind that I haven’t met, and would love to sit down with him, have a beer, and really pick his brain about incident response is Troy Larson. And not because his name is well-known, but because he’s encountered challenges that many of us may never see. To me, he’s the “yeti” of the security industry…he’s apparently found in the Pacific Northwest, there have been sightings, and we may see footprints or glimpses of him now and again, but I have yet to actually meet anyone who’s talked to him. 😉

What do you think is the biggest obstacle to having management “buy in” to an incident handling process?

After working in the commercial information security arena of over 10 years, and having been involved in physical and communications security in the military, I really don’t know what that obstacle is, or what accounts for a lack of buy-in. A very visceral part of me feels that it’s a fear of loosing control. However, I think that that fear, in a lot of ways, comes from an inherent lack of management or leadership ability in a lot of ways. There are a lot of managers who have a great deal of confidence in their ability to perform their tasks, and to hire and lead others, and manage resources to accomplish those tasks. I guess in some ways, to many managers, security seems like a wild west show, with a lot going on and some of it seeming contradictory, particularly if your only window into the security world is through trade journals and the media.

In a lot of ways, I see security (in general) marginalized, perhaps for any number of reasons. Unfortunately, our society is turning to legislation, fines, and even jail time to change this corporate culture, largely due to the fact that many of those in a position to make decisions about security aren’t making the necessary culture changes (and subsequent decisions) themselves.

In the late ’90s, the concern was, how do I sell security to my manager or my customer? In today’s day and age, we shouldn’t have to be doing this any longer. Online incidents have gone from joy-riding on the Information Superhighway to cybercrime with an economic basis and motive, and we need to keep up accordingly, not only in the sense of incident preparation, prevention and detection, but also in response. Security incidents resulting in an economic gain for one side and a loss for another is no longer a matter of if, but when.

What is the one security oriented product/service/project that is missing today?

I don’t think that there’s really one product or service…it’s more of a mindset or component of our corporate culture that’s missing. There are a number of products out there that can be used to meet the needs of particular environment or infrastructure, but regardless of how many products or services you throw at a problem, until the owners…the sysadmins, IT directors, and most importantly, senior management…change their mindset and culture, none of that is really going to matter.

For example, imagine an infrastructure where systems are put into service with little to no configuration management or control, and even if the systems and/or applications generate logs, no one is monitoring them. If that’s the case, what good is it going to do to purchase log aggregation tools, or other tools or products that generate logs?

Why write a book on Windows Forensics?

Because there wasn’t one, at least not one like the one I wrote. I’d done some research into particular areas of forensics on Windows systems, and started presenting at conferences and writing little snippets into online forums. Over time, questions (and in some cases, misquotes) started coming. I found that others were interested in the same thing. I wanted a way to document the information, and keep it available…there was no way I was going to be able to memorize everything…even I would need a reference to refresh my memory about research I’d done or something I’d found.

Simply posting on the web…on site, forum, or blog…didn’t seem to be enough; people were still asking the questions…so why not write a book? I was fortunate enough to find a publisher (Syngress, now owned by Elsevier) who was not only willing, but excited to publish it.

What programming and/or scripting languages do you recommend to assist in Windows forensic analysis? What about Unix/Linux forensic analysis?

Perl. Sometimes batch files on Windows systems are enough to get the job done, but for a greater level of granularity of control, I tend to go with Perl.

Have you ever purchased hard drives on eBay to see what you could glean from them?

No, but I have purchased them from a small shop near the local university. I’m afraid that if I do that anymore, I’ll regret what I end up finding. After all, when doing research into metadata in MS Word documents, I Googled for those types of documents from .mil and .gov sites…you’d be surprised what’s out there.

What type of environment would you recommend a person construct to help them learn forensic analysis?

Acquiring an image of a system is easy…I firmly believe that anyone who is interested in learning can be taught how to acquire images of systems. The truly hard part to teach is how to analyze a system, because it is a “system” in the truest sense of the word. You need to understand how artifacts on a system are created and modified. Once you do, then the absence of an artifact where you expect to see one is in itself an artifact.

In order to do that, it isn’t really all that difficult to set up an environment to teach or to learn forensic analysis. A couple of systems on a simple network are all that’s really needed. From there, there are a number of freeware tools available that allow just about anyone to set up a forensic analysis learning lab. For example, using tools from MS’s SysInternals site, you can monitor the Registry and file system on live systems for changes in real-time, as they occur, during different events. Keeping Locard’s Exchange Principle in mind, this will show us what artifacts we can expect to see as different events occur.

Acquiring an image of a system is really fairly trivial, but the analysis of that system can take time. Using freeware tools such as FTK Imager and dd.exe, you can acquire images of Windows test systems, even those running in virtualized environments. Then, using ProDiscover Basic from Technology Pathways, you can create a .vmdk file for the image, and then use VDK (or the GUI interface VDKWin) to mount the image on a Windows system as a read-only file system, and then using Perl, you can perform deep analysis of both the Registry and file system. Using ACLs and Jesse Kornblum’s md5deep, you can ensure the integrity of the image file(s) during this process.

Have you ever run into a situation where you’ve been asked to perform forensic analysis or recovery of data on an OS which you are unfamiliar with? How did you make out?

Yes, I have. In such cases, I do as much research as I can before hand, and ask a lot of questions along the way. I may not ask those questions of or in front of the customer, but I will ask someone. One of my goals is to develop professional networks, getting to know people that I can go to, if not for an answer, then a pointer to the right direction. No one person can know everything, and by sharing information, we’re all much smarter.

Any plans for a book on a new topic?

I’ve been approached with a couple of ideas. Some of those ideas have included things that are already out there. Others are new. What I’m most interested in, however, is meeting the needs of the community.

What is the one Windows forensic tool that you can’t live without, and why? What about Unix/Linux?

Perl. Perl, as a scripting language, gives me the power and flexibility to extract, correlate, and display data so that it can be more easily understood and analyzed.

Do you have to have an intimate knowledge of the operating system you are performing analysis on or are there some specific areas that people can focus on to obtain the maximum information?

IMHO, specialization is becoming more and more of a requirement, due to the sophistication of the incidents that are being detected.

In general, I feel that a good broad base of knowledge is required for any investigator, but more and more, the age of “Nintendo forensics” is drawing to a close. Incident investigations are becoming less about finding a couple of images or movies on a system and then closing the case. Questions are being asked that cannot be answered using the DOS-era standard of response; ie, power off the system, remove and acquire the hard drive. Keyword searches do not find Registry entries that are binary data types, or ROT-13 “encrypted”.

For example, one of the best sources of information regarding autostart locations within the Registry is anti-virus sites, where the vendors provide write-ups of the malware that they’ve analyzed. This is due to the fact that malware authors are finding these autostart locations and using them in their malware, and the vendor may not even be aware of them!

Now, I do not believe that everyone needs to have intimate knowledge of an operating system and/or applications. However, I do believe that there needs to be a community-based approach to the research; otherwise, if the research is supported solely by an individual or an organization, the results of that research are going to remain close-hold and out of the hands of folks who need it.

Andrew Hay