Tuesday, December 20, 2011

AFoD Blog With Andrew Hoog on Mobile Device Security and Forensics

So this interview is a bit of an experiment in that it’s the first vendor interview that I have conducted for the blog. I don’t plan on doing very many of these because I don’t want the interviews (or anything else I do here) to be thinly veiled sales pitches. However, in this case I wanted to try something out because I know there is a considerable amount of concern on the part of security leaders in regards to enabling mobile devices. It’s one of the hot button topics these days along with cloud computing and advanced threat actors. The reason I wanted to do an interview with Andrew Hoog is that he’s a very sharp fellow whose team over at viaForensics has been approaching mobile device security in very comprehensive manner. In addition to their work in mobile device forensics, they have spent a considerable amount of time and effort studying not only the security implications of the various mobile device operating systems, but also the security issues pertaining to mobile device applications.

I also want to make it clear that this interview does not constitute in any way an endorsement of any of viaForensics products and services. I’m not a viaForensics customer and I have not purchased or used any of their products or services. I have, however, read and favorably reviewed Andrew’s recent iOS forensics book which we discuss during the course of the interview.

If you have a few moments to spare, let me know if you found this interview valuable since I will use the feedback to determine if I do any more vendor interviews in the future and how best to conduct them. Feel free to reach out via email if you don’t want to leave a public comment on the blog.

This will be my last blog post for the year and I want to wish you all the best for 2012. I am humbled and grateful that you continue to read and comment on what I write. I’m particularly thankful for all of the people like Andrew who were nice enough to take time out of their busy lives to participate in the blog interviews this year.

Professional Biography of Andrew Hoog

Andrew Hoog – Chief Investigative Officer and co-founder

Andrew Hoog is a computer scientist, certified forensic analyst (GCFA and CCE), computer and mobile forensics researcher, author of two forensic and security books, expert witness and co-founder of viaForensics, an innovative digital forensic and security firm. He divides his energies between investigations, forensic software development, and research in digital forensics and security. He also has two patents pending in the areas of forensics and data recovery.

He lives in Oak Park, IL, where he enjoys spending time with his family, traveling, great wine, science fiction, running and tinkering with geeky gadgets.

What does a Chief Investigative Officer for viaForensics do?

As Chief Investigative Officer, I am responsible for all non-administrative functions at the company including investigations, research, development, writing books and articles, speaking engagements, winning foosball games and making sure the beer fridge is well-stocked.  Basically, I get to do the really fun stuff but don’t have to worry about the accounting, human resources, etc.  I also work with the two other members of the management team (Chee-Young Kim, President, and Ted Eull, VP of Technology Services) to manage the direction and strategy of viaForensics.

Our course, the real question might be how did I ever come up the title Chief Investigative Officer?  Prior to co-founding viaForensics, I was the Chief Information Officer for a medium-sized company (approx. $750 million in sales) and over my career have held senior IT positions in small, medium and large organizations.  The end goal in the corporate IT world is, of course, to be the Chief Information Officer.  When I finally achieved this at my previous job, I rather liked the title and decided I wasn’t ready to part with it.  So I came up with Chief Investigative Officer which seemed to fit quite nicely.  Plus, there’s the added benefit that I always have a title to select (CIO) on the never-ending barrage of pesky web forms I must fill out.

AFoD Blog: How did you obtain the knowledge that enabled you to get to where you are today? Did you study information technology in a university setting?

Andrew Hoog: I attended Saint Louis University and received a Bachelor of Arts in Computer Science (and a minor in Math).  Yes, I know, a Bachelor of Arts…really?  Well, I’m confident the degree was exactly what I needed.  There are many fantastic technical schools and they generally provide a Bachelor of Science for CS which essentially means more physics and such and less softer skills (such as writing, philosophy, etc.).  But it’s the ability to think critically, reason and communicate both written and verbally that have been major accelerants for my career.  So, an important foundation was set for me at SLU.  And let’s face it, programming in assembly language is nothing to sneeze at so I’m pretty comfortable holding my own with more traditional technical degrees.

Beyond formal education, though, I’m mostly self-taught.  Like many of the readers, I’m absolutely hooked when it comes to computers so I find it enjoyable to work in this very technical discipline.  I didn’t get involved with forensics until recently (2008) and my introduction to the topic included reading many books and blogs and then getting my GCFA.  And that’s one thing I love about the forensics community: their willingness to share knowledge.  Whether it’s on the many great blogs I follow, in books or simply talking to people over email, on the phone or at a conference, the knowledge sharing within our community is a tremendous resource.

AFoD: Can you describe the process you used to teach yourself? How does someone go from having a Bachelor of Arts in Computer Science to being one of the leading mobile device forensics researchers in the field today?

HOOG: Well, first, I certainly appreciate your characterization of the research we’ve performed to date.  There’s a tremendous amount of opportunity in the digital forensics discipline for motivated individuals and companies.

My preferred method for learning is to dive in and be very hands-on.  So, if I’m working on Android, then I want an Android device (well, actually, as many devices as I can get) and I start tinkering.  I do a lot of reading, whether from blogs, academic papers, books or simply source code.  And I like to program…not superbly architected systems, but code that tackles the problem directly.  For that, I use Python and if I happen to develop something useful and compelling, we turn the working code over to excellent programmers who do a better job with the overall architecture, abstraction, development, etc.

But I suspect there are a few fundamental drives I possess (beyond being a forensics geek) that many people in our industry do as well.  First, if I encounter something new, I want to understand how it works.  Second, as I learn the system, I want to expand upon existing knowledge base.  In forensics, that often means how can I gain access to a device, forensically acquire the stored data, and ultimately analyze the information to create actionable intelligence (and that’s the really fun part).  Third, once I’ve figured out something new, I want to code it.  As I mentioned, I like Python as it allows me to rapidly prototype a system and attain results.  And finally, I’m highly motivated because I find all of the above steps incredibly satisfying.  Once I get started on a problem, I don’t want to stop until I feel I’ve at least made a good dent.  I also like to share what I’ve learned which has led to HOWTO blog postings, many presentations (which are slowing being put online at our website) and recently several books.

AFoD: This past summer I reviewed your excellent iOS forensics book that you wrote with Katie Strzempka. You also released your Android forensics book around the same time. What can you tell us about both books? What makes them different from what has been done in the past?

HOOG: The approach for both books is to be very technical and provide examples using as much F/OSS software as possible for reader to follow along.  So, I think the iOS book came out very well and provides not only extensive background and acquisition information, but also how to analyze iOS/HFS file systems, an overview of commercial tools, and a number of techniques anyone with an iOS device and a computer can do.  The Android book doesn’t have to differ from the past as, to my knowledge, it’s the first book out on Android Forensics.  At over 100k words, it is also very detailed and provides steps to build an Android forensic virtual machine (Linux) and plenty of examples.  I also cover the open source YAFFS2 file system in detail. 

As with the approach we’ve taken with viaForensics, the books also push into the mobile security space.  So, chapter 5 in both books deals with mobile security but from the view point of a forensics examiner.  Beyond the background info, the chapters target information to specific audiences: mobile device user, mobile app developer, and corporate IT security responsible for securing mobile data.  What we’ve learned over the past few years is that forensics has a much larger role to play in overall security than it has in the past (IMHO).

The books have been well-received and sales are strong.  Several universities are evaluating the Android book as a basis for a mobile forensics and security class and two universities have officially selected it (one semester class just ended).  Katie was an excellent co-author on the iOS book and deserves much credit.  We have a great team at viaForensics and we like to share our knowledge, so the books were a great fit for us.

AFoD: Digital forensics is a tool intensive discipline and there are a dizzying amount of tools being offered for the mobile device examinations.  What do you recommend to people who are starting from nothing, but want to build out a digital forensics tool set to cover a broad range of mobile devices?

HOOG: One of the challenges of mobile forensics is that it’s very difficult and expensive to support a broad range of mobile devices because there are so many and they can vary greatly.   Generally speaking, this is not an issue in computer forensics since you can pull the drive, attach it to a write blocker and image most of them in the same way.

I recently wrote a long post on this topic arguing that the goal for examiners should be to support the phones that they are 1) mostly likely to encounter and 2) most like able to extract meaningful data from.  This is not to say that you can simply ignore other phones but if you try to support every phone, it will be very difficult. 

There are a number of F/OSS solutions examiners should consider.  First, BitPIM has been around for a while and supports many phones.  We (viaForensics) also developed an Android forensics logical tool (AFLogical) free to qualified law enforcement and government agencies.  So, these are great options to start out.  If you see a wide variety of phones and need to attempt to image them all, you’ll have to purchase a commercial solution that provides broad support (two examples with seemingly happy users are Cellebrite and XRY).  Since we have our own commercial forensics software which focuses on Android, we know how difficult supporting even one platform can be so while they phone may be covered in the product literature, the amount of data extracted can vary.  I would encourage examiners to test ahead of time (if possible) or perhaps check out NIST to see if they have tested the software.

Mobile devices are increasingly important pieces of evidence but they are troublemakers.   So, focus on the most important, high-yield devices.  Take advantage of F/OSS software.  Look at resources you can tap to find out if a mobile forensic platform works well, such as NIST reports, blog posts, MFC, mailing lists, conferences, white papers like our iPhone Forensics white paper or simply call other examiners on the phone and just ask them.    And if you have experiences you can share, add your voice to the discussion so we can all tackle this increasing difficult problem.

AFoD: One of the reasons I wanted to do this interview with you is that you are doing more than just talking about the forensic examination aspect of the mobile device security. For example, the team at viaForensics has spent a considerable amount of time addressing the overall security implications with these devices through avenues such as your appWatchdog work. What do you tell a chief information security officer who asks you about the impact these devices will have on a corporation's risk profile and how that organization should be addressing those risks?

HOOG: Mobile devices are quickly changing the risk profile for corporations and CIOs/CISOs are justly concerned.  It’s interesting to look at how these changes happened so quickly.  When Apple released the iPhone, they were not targeting corporate enterprises directly; they were focused on the consumer.  And while there is now some attention to needs of the enterprise, Apple (as well as Google) is still largely focused on the consumer.  But this lead to an interesting development: employees -- many of them senior executives -- began using mobile devices, both personally and for corporate systems, and they were able to do this without getting the approval of IT.  So, the tables have turned and IT departments must accept the reality that these devices are here to stay.

Early on, corporate IT was not aware of the risk to their organizations but this has changed over time.  A growing part of our business is performing testing and analysis for corporations who are trying to mitigate the risks introduced by mobile devices.  And the risks are considerable.  On the obvious side, an enormous amount of corporate data is cached on mobile devices and is outside the control of the IT department.  The data can easily end up on personal computers or even eBay/Craigslist.  Beyond data caching, devices can be used to compromise a company, whether from an insider or an attacker gaining control of a device.

We are often asked what a corporate (or individual) can do to protect themselves from mobile risks and jotted a few suggestions down just after the Epsilon breach.  We’ve posted a number of free (and one paid) resource to answer these questions and I’ve been interviewed extensively on this topic (so perhaps just Google my name). Here are a few examples:

· Tips for both consumers and corporate IT for securing mobile devices (free)

· A series of 10 questions on mobile security (I chose one but you can access all from the free article):

· Our Mobile Security Risk Study, a very detailed report (80+ pages) covering mobile security risks affecting corporations. The report includes detailed analysis of the efficacy of security controls such as passcode protection, and focuses on the security of iOS (iPhone) and Android (paid)

Rather quickly after starting viaForensics, we realized that digital forensics can play a far larger role by expanding beyond a reactive model (investigations and incident response) and into a proactive model.  The proactive implementation of digital forensics is now a primary focus for viaForensics and has led to initiatives such as appWatchdog (free mobile app security testing), appSecure (paid, sophisticated mobile app security testing and certification) and liveForensics (proactive forensic monitoring for key assets).  This is the really exciting stuff.  We have made tremendous strides and impacts in the larger security space by applying the forensic discipline to the many problems the industry is facing. 

AFoD: Can you talk more about what you mean by the proactive implementation of digital forensics?

HOOG: While we are relative newcomers to the digital forensics field, we’ve been at it long enough to see patterns emerge in many investigations.  For example, how many of us have done the “departing employee data theft” case and when you look at it, you realize 80% of the investigation is the same as the previous one?  Once I see a pattern like that, I can’t help but look for a way to improve (i.e. automate) the process.  And we found that there were ways to do that indeed, especially since most of the forensic tools we use are command line.

The next realization was that while we could tell a client the last time someone connected a USB drive to their Windows workstation, we could not tell them much about previous activity.  So we (and other examiners) have become very good at figuring out what happened with only a fraction of the data points we need.  But it seemed far easier to simply capture that data than to try to guess what happened.  When you look at the forensic metadata you would need for this historical information, it’s really not a lot of data (in terms of MBs).

So we began to work on proactively collecting forensic metadata from key systems on a scheduled basis, typically daily (but we can handle any frequency).  We then store that data, analyze it with the techniques we developed above, and then import all of the information in a data warehouse.  This allows us to provide sophisticated reporting, analysis, dashboards and even visualization to our clients.  We no longer have to guess about the other times a USB drive was connected since we have all the data.  It’s a tremendously powerful solution and we’ve been quietly providing it since the end of 2009.  We call the service liveForensics® and we have a growing list of clients that utilize it.

There’s quite a bit more I could say on that topic but instead, I want to provide one other example.  As we were performing investigation on mobile devices, we were consistently uncovering sensitive data on the phones that no one, except the “bad guys”, would benefit from.  For example, we have uncovered full credit card data (16 digit number, CCV, name, etc.) and it really bothered us.  If law enforcement was doing an investigation, they did not need the CC data.  If we were doing a corporate investigation, again, they had no need of the CC info.  The same goes for “domestic cases” and the end user does not need the CC info insecurely stored on the device.  So, the only benefactors of this info would be cybercriminals. 

We again looked to proactive forensics to begin to address the issue.  We created a free service, appWatchdog®, where we examine popular mobile apps on iOS and Android to determine if they store usernames, password or sensitive app data unencrypted on the device.  If so, we note what is stored and provide a rating for the app on our website.  The consumer can then determine if they apps they use put them at risk for financial or identity theft.  We are coming out with an Android app soon (and hopefully an iOS shortly thereafter) which will look at the apps a user has installed and let them know which ones pass and which have security issues.  The info is also posted on our website and we posted a study recently highlighting the first 100 app audits we completed.

So, these are two examples of how we apply forensic techniques proactively to solve security issues.  Why wait around for an incident to occur when you could use the power of forensics to detect and ultimately thwart the attack?  So, we’re kind of hooked on the proactive forensics thing and we’re just getting started.

AFoD: Thank you for taking the time to do this interview, Andrew. Is there anything else that you'd like the readers to know about regarding what we can expect out of viaForensics in the future?

HOOG: Since viaForensics is heavily invested in forensics/security R&D, there’s quite a bit folks can expect in 2012. The most straightforward are a number of key updates to viaExtract, our forensic software. 

In mid-December, we’ll release version 1.1 which will extract considerably more data from Android devices.  As some of your readers know, we also have significant experience in physical extraction and analysis of Android devices so expect some developments on that front.  In fact, we know a bit about that on the iOS side as well and Windows Mobile is finally positioning itself as a mobile OS worth researching.  Finally, we’ve developed a new SQLite recovery technique which extracts far more data and we’ll likely build that into viaExtract soon.

We are also working on some new NAND Flash acquisition techniques.  This is still very much in the R&D phase, however, on some phones, we expect to have a working NAND Flash write blocker (software based) and we are working on solutions for NAND Flash that have embedded controllers.  Of course, acquiring data is only one part of the challenge so we are working on decoding and analysis tools as well.

Another interesting project we are working on is YAFFS2 support The Sleuth Kit.  This should be good news for the community as there is limited support for YAFFS2 today and we will release our code as open source and part of TSK.  Soon, we will also release an open source version of AFLogical, our Android forensics logical component.

Our liveForensics service is in the process of major upgrades as well. The collection agent is maturing rapidly and we are improving the analysis and reporting interfaces. We will also develop a black box version of the service so it can be deployed internally at our larger clients.

On the mobile security front, we will continue to combine our forensic and security expertise to analyze mobile apps.  If any of your readers have responsibilities in this area, they should keep track of our posts.  We’ll have some interesting findings posted soon and some compelling products on the way.  Our mobile security work is also applicable to mobile malware so we have some interesting things in the works there. 

While there’s quite a bit more, I finish off with one final item.  We’ve recently developed some very advanced techniques for securing mobile devices that extend well beyond any commercially available solutions today.  I can’t dive into specifics yet but if organizations require very advanced security on mobile devices, we will release a solution in 2012 to address current shortcomings.  I better stop now or we’ll bore your readers.  Thanks, Eric.

Sunday, December 4, 2011

AFoD Blog Interview With Rob Lee

One of the things that I have enjoyed immensely about my information security career is that I have had the opportunity to meet and work with some amazing people. Rob Lee is one of those people. I have learned a tremendous amount from him over the relatively short time that I have known him. He’s someone that I have come to trust as a friend and as a professional peer. Anyone who has the good fortune to know Rob knows why I frequently use the hash tag #giantpersistentfriend when referring to him. I count getting to know Rob as one of the highlights of my personal and professional life so far.

Professional Biography of Rob Lee

1ede7fcRob is an entrepreneur in the Washington D.C. area specializing in Information Security, Incident Response, and Digital Forensics. Rob is also the curriculum lead for digital forensic training at the SANS Institute. Rob has more than 15 years' experience in computer forensics, vulnerability and exploit discovery, intrusion detection/prevention, and incident response.

Rob graduated from the U.S. Air Force Academy and served in the U.S. Air Force as a founding member of the 609th Information Warfare Squadron, the first U.S. military operational unit focused on information operations. Later, he was a member of the Air Force Office of Special Investigations where he conducted computer crime investigations, incident response, and computer forensics. Prior to starting his own firm, he directly worked with a variety of government agencies in the law enforcement, U.S. Department of Defense, and intelligence communities as the technical lead for a vulnerability discovery and exploit development team, lead for a cyber forensics branch, and lead for a computer forensic and security software development team. Rob Lee also was a Director for MANDIANT for four years prior to starting his own business.

Rob co-authored the bestselling book Know Your Enemy, 2nd Edition. Rob earned his MBA from Georgetown University in Washington DC. He was awarded the Digital Forensic Examiner of the Year from the Forensic 4Cast Awards. Rob is also an ardent blogger about computer forensics and incident response topics at the SANS Computer Forensic Blog. Rob is also co-author of the MANDIANT threat intelligence report M-Trends: The Advanced Persistent Threat.

AFoD Blog: Why did you choose to attend the United States Air Force Academy?

Rob Lee: I grew up in an U.S. Air Force family.  My father, Col (Ret) Robert E. Lee USAF, and my grandfather, BGen Travis M. Hetherington USAF, both served full careers in the Air Force.  My grandfather was a West Point grad ('33) both are originally from Texas as well.  In fact my Grandfather was deputy director of the NSA when it was formed. I was really inspired by both of them.  The  U.S. Air Force is a part of my blood and it has never left it.  I attended USAFA as I felt it would give me the best chance to work in the space operations career field.  I loved anything to do with rockets, space shuttles, and the possibilities in space.  I was one of those "Space Camp" nerds.  Unfortunately, my personality was not geared for that entire "listening to orders" thing. I liked to do things "my own way" and USAFA was the antithesis to that mentality.  I had former teachers and even my own parents sit me down and ask me if I truly understood what I was getting myself into.  In the end, I went because I felt that if I didn't try I would always end up regretting it. I wanted the challenge.

AFoD: You earned your degree in space operations while you were at the academy.  Can you tell us what sort of courses you took as part of that program?

LEE: Every USAFA grad graduates with a Bachelor of Science degree regardless of major, so we have a very intensive core curriculum.  Most cadets end up taking between 18 and 21 credit hours per semester with most in engineering degrees have to take 21 credit hours.  Those in the Astronautical Engineering majors usually had to take at least 2-3 semesters taking 24 credit hours back in the mid-90s.  For the degree program, there was a mix of computer programing with astronautical engineering courses. I also took the first information operations (IO) course offered at USAFA which really had me redirect my thinking for career options when I graduated.   My senior year, I took an extra course, for "fun", from the USAFA Computer Science dept. in Computer Security.  While I had a large affinity toward computer science and I was good at the programming, most of the career advisers told cadets to choose a major with an operations focus and many felt those with computer science degrees wouldn't probably make rank as easily as your peers.  I don't regret my choice as I really was able to get the best of both degrees in my studies.

AFoD: One of the advantages of attending a United States military service academy is that in addition to getting a great education, cadets have a rich academy experience outside of the classroom. These service academies are some of the finest leadership training institutions in the world.  Can you tell us what sort of extracurricular activities you participated in when you weren't in class and how your overall academy experience prepared you for your future career?

LEE: My 1st year, I was on the USAFA Ski Team.  I was on racing ski teams in high school on Team Breckenridge and Copper Mountain teams in Colorado.  However, trying to manage racing with school ended up being too much and I had to pull back.  Every cadet had to go through Survival Training call (SERE) training as well.  I also participated in a program my 3rd summer where I led newly enlisted airmen through basic training at Lackland AFB.  Leadership and management are very different things and the academy provided much of the environment to practice that.  One thing that clearly helped me out in my career is my participation in the cadet acting group called Bluebards.  Being able to stand up in front of your peers and communicate effectively is a skill I feel everyone should master if they plan to lead and not just manage.

AFoD: So how did you go from a snow skiing Air Force space engineer to a career in information warfare and digital forensics?

LEE: Hahaha... So... as I was entering the first semester of my senior year we had to select our chosen Air Force careers where you chose your warrior class.  I had chosen space operations.  However, as I found out MUCH too late, Space Operations had a color vision requirement.  ACK!  Ironically, it was the same reason that eliminated me from becoming a pilot too.   I was torn apart internally as I had studied so hard to be in Space Command. I wanted to be a Star Trek nerd.  It was around this time that I interested in Information Warfare and was taking a course in it.  I decided in that course, that I was going to make a career of this. 

During an Information Warfare (IW) conference held at the academy in Nov 1995, I ended up meeting two members from the 9th Air Force 609th Information Warfare Squadron.  Little did I know that, at that point, there were the only two members that existed.  Lt. Col. "Dusty" Rhoads and Maj. Andrew Weaver.  I expressed my interest in being assigned to their unit and wrote a letter to them expressing my desire.  They told me to apply to be a Communication Officer (33C) career field.  I did.

You have to understand though, back in 1995, not many had even heard of IW or IO at this point.  What I didn't know was that Lt. Col. Rhoads had "by-name selection authority" given to him by the Chief of Staff of the U.S. Air Force.   Early in the spring I was called into the assignments office at USAFA.  Apparently cadets are not supposed to go find their jobs and I was given a "talking to."  Apparently, orders came down from the Pentagon to assign myself and another cadet to the 609th IWS.  I didn't realize how irregular this was until much later, but I found out that it was one of the first assignments generated for my graduating class, but I couldn't tell anyone about it till assignments day.  They had called me to the office to find out how I "gamed" the system. 

I knew that being selected to go to the 609th would be a career changer for me and spent the rest of my senior year studying every programming book, every security book, and reading online as much as I could.  I also gave up part of my summer after graduation to intern at DISA in Washington D.C. to merely absorb as much as I could.  I came to understand that no one really knew that much about internet warfare or defense and that we ended up taking a really good stab at figuring it out.  The 609th was a great experiment that ended up getting tabled due to politics.  But I do consider the two years I spent at the 609th IWS as my graduate degree of sorts.  I read more books, learned more on the job, and the unit engineered more solutions without manuals.  I’m not sure I could have received a better education in information security through any university.  We just didn't receive a paper to hang on the wall.

AFoD: I had a similar experience. I wanted to be a United States Navy Surface Warfare Officer when I was growing up and it turns out the Navy frowns on officers who were legally blind without their glasses commanding warships. Thus, they invited me not to join them and that was how I ended up in law enforcement. What caused you to leave the Air Force and what did you do after you left?

LEE: Leaving the Air Force was not an easy decision.  Both my father and grandfather were career officers.  However, the services simply did not have a career path for officers that both wanted to stay technical and lead troops although they had many examples of that in pilots, space operators, doctors, etc.  In a nutshell, the personnel center told me that if I intended to be promoted I would have to expand my horizons out of the Information Operations (IO) side of things.  I wrote to many Generals and Colonels that I knew and each told me that it was too soon for a specific IO track in the AF.  However, they also told me that there are other ways to serve my country.  I separated with the intent of going to work for one of the intelligence agencies and ended up working the next 7 years between the CIA and the NSA.  

This is also around the time I started teaching at the SANS Institute.  I attended my first SANS event in Orlando in March of 2000.  It was at this time that they were introducing the intrusion analysis (GCIA) exam.  I was decent at examining packets by hand as a result of my time at the 609th IWS so I wanted to simply challenge the exam without taking the course.  I wrote to Stephen Northcutt and asked him for permission.  He hesitated initially, but approved.  When he approved the waiver via email he mentioned that it was a really difficult test and not many who took the class passed it. He kind of implied that I would probably fail since I didn’t take the training first.  Apparently, almost everyone was failing their initial exam because of the difficulty by the people that took the courses first.  I took the test and scored just somewhere 96%.  I think Stephen must have been receiving scores in his email as I received a call no more than 10 minutes after I finished the test.  He said, "Who are you?"  And I explained my background working at the 609th IWS and AFOSI.  He wanted to meet at the SANS conference in DC that summer and asked me to give a 2 hour presentation on IR and Forensics at "Capitol SANS 2000".  I have been teaching ever since.

In my other position, after I separated from the Air Force, I worked in a very specialized group in a group at a government contractor ManTech called Computer Forensics Intrusion Analysis (CFIA) Division.  I was convinced by Travis Reese that joining a government contractor allowed me to work with more projects than becoming a .gov civilian.  I asked to be assigned to the IOD (Intrusion Operations Division).  I was a researcher where I accomplished vulnerability enumeration and discovery.  In a nutshell, we tried to break things.  Although rare and time consuming, we were usually successful at it.  I also led a team of developers working for a variety of projects in the IO world of the intelligence community.  The most wonderful thing about CFIA was the people and my co-workers.  CFIA was a pool that ended up having under its roof some of the most talented individuals you have probably never heard of.  All I can say was that we did more during my years at CFIA to help this country than I did in the 5 years of service while in uniform.  Many from CFIA eventually left to join other companies such as Kyrus-Tech.  I ended up leaving in 2007 to go to business school at Georgetown University.

AFoD: So what would a highly accomplished information security leader such as you need with an MBA?

LEE: I really wanted a master’s degree and I was torn between technical and getting a business degree.  Why a business degree?  2 reasons really: 

1.  Every organization that is compromised is run by business leaders to an extent.  Being able to understand their concerns from the business angle has made it incredibly easier to sit in board rooms and not only tell them I’m a geek, but a geek that understands them.  They have tended to respect that I have gone out of my way to speak their language and understand the business impacts to a compromise. 

2.  I wanted to do something that I wasn't good at.  Heading the business direction allowed me to explore areas that I wasn't familiar with, particularly business finance.  It was important for me to grasp these concepts as I am purely fascinated by how organizations truly operate.  I was growing tired of not being able to truly look at 10K and not know what is good or bad in a business.  To that end, I hope to start my own organization/business soon and I simply wanted to learn more in an area I never really focused on before.

AFoD Blog: I couldn't agree more. One of the key deficiencies that I see in the larger information security community is a failure to understand how the business world works. Some of the best training I've obtained in my own career has been through an executive education program that I went through at Dartmouth's Tuck School of Business. There has been a considerable amount of effort put forth by business schools to develop future generations of business leaders that the information security community should be taking part of and embracing. There is much more to creating and leading effective security organizations than just technical knowledge.

So let's shift gears a bit here and talk about one of the hot topics in the information security world which is advanced persistent threat (APT). What is your definition of APT?

LEE: The APT is a cyber-adversary displaying advanced logistical and operational capability for long-term intrusion campaigns.  Its goal is to maintain access to victim networks and exfiltrate intellectual property data as well as information that is economically and politically advantageous.

The APT is not a bot-net.  It is not a car.  It is the DNA of an adversarial group. 

AFoD: Do you limit your definition of APT to nation-states acting as the cyber adversary or do you allow, for example, organized crime groups acting independently of a nation-state to also be defined as APT?

LEE: Any group able to display the logistical and operational control for a long term intrusion would fit.  Scale wise, nation states tend to have enough of the people employed to pull this off.  Organized crime -- more people needed to pull of long term attacks = more mouths to feed.  It also doesn't align to their goals.  It is much easier to offline a CC database than to remove IP from a network.  APT to an extent has not been as interested in financial and card data theft operations.  Organized crime tends to not be interested in information that the APT tends to focus on.  Therefore, it could be anyone, but it tends to be nation state actors over organized crime as a result.

AFoD: So what is your advice to an organization that finds itself targeted by an APT adversary?

LEE: Don't over react.  Organizations should gather as much information about the situation they are in before they take any action.  Early execution could simply make the problem worse.  This basically means that you might need to "let the adversary" have their way with your network while you assess exactly where they are prior to strategize a plan for their removal.  Long gone are the days of "Pull the plug and reinstall from backup."  Most organizations start blind, but progress to having very good intelligence on exactly where in their enterprise they will find the APT as a result.  

Once an organization has decent intelligence on how to identify compromised systems by the APT, they should begin planning the remediation actions.  The remediation actions should not be gradual, but a sudden and deliberate plan of action designed to cut off communication between the APT malware and their operators. In addition, remediation should focus on the removal of the malware and the addition of new security measures designed to prevent additional beach head systems and degrade the ease lateral movement by the APT.  This is sometimes difficult if remediation occurs too early.  If a single piece of malware survives, that footprint could be used to gain control back over the network and malware that is more difficult to find and identify is usually deploy by the adversary.

Too often, an organization responds blindly and ends up making the problem much worse than it needed to be.  To the adversary, they cannot tell if you have your act together or if you are fire into the darkness.  They respond as if they are about to be removed and as a result dig deeper into your infrastructure.

AFoD Blog: People like you and I are spending a considerable amount of time getting the message out to senior executives and others that controls aren't enough and that they need to get much better at detecting, responding, and remediating. All that said, controls are still very important.  What do you tell people who ask what sort of controls are the most effective when it comes to combating advanced actors like nation-states and organized crime groups?

LEE: That is such a great question.  I get asked this all the time in the form of "What is the simplest and easiest thing I can do to improve my security to defend against the APT?"  

The SIMPLEST answer? I usually answer, plan migrate to Windows 7 and Server 2008 as quickly as possible.  Most enterprise organizations are still running a Win2K or a WinXP workstation with a Sever 2000/2003 base.  These technologies are over 10 years old now and were created prior to Microsoft's secure computing initiative.  While not full proof, the ability to freely move in a Win7/Server 2008 vs. a WinXP/Server2K environment is night and day.  The amount of capability and additional protection pre-baked into the latest releases of those operating environments will slow the advance of adversaries.  I won't get into each specific as there are many, but the simplest (not the easiest) answer is to "upgrade your enterprise."

As a part of this, once you upgrade to a somewhat homogenous IT environment, it is easier to establish decent application and system controls since there is a reduced common baseline.  Instead of managing upwards of 30-40 different system configurations, an organization could reduce their common desktop environment to less than 10 variations.  With the reduction, application and host based controls can easier track and monitor application white and black lists easier.  In addition to this, the host/server auditing found in Win7/Server 2008 can be configured easier and at a depth that is effective.  WinXP didn't allow for the specific tuning that a Win7/Server2008 environment might hold.

So the benefits to upgrading are not only a better security baseline, but trickle down security benefits from the upgrade as well.   There is no silver bullet in APT defense.  But you can make it more difficult for them to create a beach-head and laterally move.  

The EASIEST answer? Hire the smartest and most experienced people you can get your hands on.  Recruit and court a good leader and give that leader the authority and funds to build the best team possible.  Give them a blank check to do so.   True APT defense is not a technological answer, it is a technology and a people one.  Typically organizations that have the best people will end up creating their own solutions on the fly to combat the APT.  Give them leverage, delegate responsibility, and remove the internal roadblocks that will impede their success.  The key is the team lead. Take your time and consider executive compensation for this individual as they are extremely difficult to find if they are good.  I could get into a series on how to create a structured team that promotes creativity, champions hard work, and empowers the IT security operators to get the job done.  The biggest gripe from most IT security professionals?  Very few listen to them.  Block some time in your schedule. Sit down. Listen.  

AFoD Blog: So what can we expect to see from Rob Lee during the next year or so?

LEE: Over the past 3 years, we have been updating the forensic and incident response courses at SANS to include the latest tactics at finding and defeating the APT.  The course where I have focused the majority of my efforts to train forensicators to deal with the threat has been FOR508:  Advanced Computer Forensic Analysis and Incident Response.  One of the biggest things I am working on currently is an update to FOR508 - Advanced Forensics and Incident Response.  I cannot give away too much about what is coming, but it will be the go-to course for investigating APT and advanced incidents.  This update is big and has been introduced in stages starting last year.  For example we added a section on enterprise and remote forensics already. In fact, we hand out F-Response to each student who attends class.

On a professional front, I’m starting my own company early next year.  I kind of have a unique idea that I’m working off of and am excited to see it come to light.

Next summer we will be doing the 2012 SANS DFIR Summit in Austin TX again.  I am very excited about it.  Leading up to it, we are accomplishing the first SANS DFIR lighting talks on Dec 13th 2011 in DC.  The talks are completely open to the public but you must register.   The Washington D.C. area is known for its density of talented professionals in the field of Digital Forensics and Incident Response, and SANS is bringing 10 top industry experts to you for the first SANS360: DFIR Lightning Talk. In one hour these Digital Forensics and Incident Response experts will discuss the coolest techniques and solutions they have discovered in 2011.If you have never been to a lightning talk it is an eye opening experience. Each speaker has 360 seconds (6 minutes) to deliver their message. This format allows SANS to present 10 experts within one hour, instead of the standard one presenter per hour. The compressed format gives you a clear and condensed message eliminating the fluff. If the topic isn't engaging, a new topic is just 6 minutes away.

Friday, November 4, 2011

The Happy Time

One of the benefits of blogging and speaking about topics that I’m passionate about is that I get to meet many fascinating people who are interested in the same things. One such person is law student Joel Kosh who attended a presentation I gave recent recently at Yeshiva’s Cardozo Law School. Joel found this magnificent January 2010 TED talk by Guy-Philippe Goldstein entitled “How cyberattacks threaten real-world peace”. Guy-Philippe made many insightful points during this talk, but the point that really stuck with me was where he spoke about the imbalance of weapons technology contributing to the likelihood of conflict. During the talk Guy-Philippe stated:

Similarly, if we'd had this talk 30 or 40 years ago, we would have seen how the rise of nuclear weapons, and the threat of mutually assured destruction they imply, prevents a direct fight between the two superpowers. However, if we'd had this talk 60 years ago, we would have seen how the emergence of new aircraft and tank technologies, which give the advantage to the attacker, make the Blitzkrieg doctrine very credible and thus create the possibility of war in Europe. So military technologies can influence the course of the world, can make or break world peace -- and there lies the issue with cyber weapons.

Guy-Philippe went on to explain that we are in a time where we have a technology imbalance when it comes to cyber weapons. This imbalance has increased the risk of conflict that could spill over into the physical world.  He explained it this way:

Just last week, in a New York Times article dated January 26, 2010, it was revealed for the first time that officials at the National Security Agency were considering the possibility of preemptive attacks in cases where the U.S. was about to be cyberattacked. And these preemptive attacks might not just remain in cyberspace. In May 2009, General Kevin Chilton, commander of the U.S. nuclear forces, stated that in the event of cyberattacks against the U.S., all options would be on the table.

We’re definitely in a time where the battlefield favors the attacker. Corporate and government networks are built for business purposes rather than defensive military purposes. One of the primary themes that I bring out in my APT presentations is that while basic information technology controls are critical, they will not keep out advanced actors. An information security model that is focused solely on prevention will fail and will likely do so in a catastrophic manner that will result in substantial loss of intellectual property, customer data, and competitive advantage. The model has to be one that embraced prevention, detection, response, and remediation. The weapons imbalance is so great that keeping attackers out of your network with certainty isn’t a viable option. The cyber version of the Maginot Line makes about as much sense in the 2011 cyber world as it did in the 1940 physical world.

I’m an amateur student of history. I find military history to be particularly instructive because so much of human history revolves around conflict. Human conflict is a historical constant and is a frequent catalyst for substantial change in the course of human events. Just look at how the last couple world wars transformed the course of human history. We’re now in an period of history where it’s possible that cyber warfare could result in similar change if it were to spill over into the physical geopolitical world.

I know most analogies don’t work all that well, but I have been curious about analogies in the physical warfare world that could be used as a tool to help people understand the cyber warfare world. There are several that come to mind from the last century that I think provide a reasonable illustration of the “detect and respond” model where preventative controls (such a fortresses, walls, and mutually assured destruction) either aren’t available or would be ineffective.

The first is the Battle of Britain. This was a conflict where fortifications didn’t apply since it was air warfare. Both sides engaged in traditional intelligence gathering methods for their threat intelligence purposes and ultimately it boiled down to the Royal Air Force being able to quickly detect and respond to German air attacks via the Dowding System. I like this analogy even though it doesn’t apply completely to modern cyber warfare since we have more preventative controls available that can help us win than the British did. However, the core of it is still very similar. The British used threat intelligence and real-time detection methods such as radar to pinpoint where they needed to send their expert incident responders (their pilots).  Processes (such as the formations they flew in) and tools (their aircraft) where important, but the core of their victory came from well led pilots and the people who supported them.

SubmarineThere is a second one that I think I like a bit better and that’s the Battle of the Atlantic. The German Navy knew it had a problem with sea power because of the British Navy’s dominance in this area. Sure, the Bismark was impressive, but it lasted about as long as you expected it would given the threat environment it faced where the British dominated the surface of the sea. Submarines, however, provided the German Navy with a substantial weapons technology imbalance that they exploited with fantastic success during “The Happy Time” where the Allied powers were very poor at detecting and responding to undersea weapons. Eventually, the Allies were able to leverage the proper people, processes, and tools to counter the threat, but only after suffering an incredible amount of damage to their war effort. We are in a cyber version of “The Happy Time” where nation-state and other advanced actors on the attack have a dominant position over those of us on defense.  We need to work very hard to quickly develop the proper people, processes, tools, and geopolitical policies to bring things back into a closer balance or we’re in big trouble.

ONCIX Report

The United States Office of Counter Intelligence released a report titled “Foreign Spies Stealing US Economic Secrets in Cyberspace - Report to Congress on Foreign Economic Collection and Industrial Espionage, 2009-2011” to the United States Congress this week. There have been plenty of news stories that have summarized the report such as this one from the Wall Street Journal.  It should not come as a surprise to anyone, but the report spends a considerable amount of time talking about the threat from nation-states such as China and Russia.  I’m glad that it didn’t just focus on China. As I have been explaining in my v APT presentations, cyber espionage isn’t unique to any particular country. It’s natural for state intelligence agencies to use cyberspace as part of their information collection methods. Not all espionage is equal, however, especially when the intelligence collection departs from traditional goals such as determining intentions and capabilities of a national government and its military and moves into wholesale economic espionage.


Speaking of my APT presentations, I will be giving one on November 16th at 7PM at the November NYC4SEC Meeting. The meeting will be held at the John Jay School of Criminal Justice in NYC. I’ll go through my presentation and there will be plenty of time afterwards for questions and discussion.

Registry Decoder

The nice people at Digital Forensics Solutions have released version 1.1 of their Registry Decoder tool. This is a free tool that you can use for your registry forensics investigations. They have been working very hard on the development of this tool so please give them all of the constructive feedback that you can once you have tested this tool for awhile.


I’ve gone back and forth on how I would use my LinkedIn profile. I started out being pretty permissive with who I’d accept invitations from and then took a much more restrictive stance later on when I realized that I didn’t know many of the people in my network. I’ve decided that the former option is the best use of the profile because it’s a very nice way to meet all sorts of people who would like to connect and communicate with me. Thus, if you have a LinkedIn profile and are in the field, feel free to send me an invitation. I make it a policy not to link my actual profile from my blog because I like to keep a degree of separation between the blog and my employer. However, it’s not all that hard to find me if you just search on my name. Thank you to everyone who have sent me invitations to your network. It’s been great fun getting to know the people who read the blog and to see all of your respective backgrounds.

Saturday, October 15, 2011

Emails of Marque and Reprisal

I was recently interviewed by Michael Kassner on the topic of digital forensics. The interview was geared more towards being an introduction to digital forensics for those who might not be familiar with the topic. You can read that interview on TechRepublic. It was a bit of a switch being the one interviewed and I hope you like the results.

I will be the keynote speaker for the SC World Congress eSymposium on Advanced Persistent Threats. The event will be on October 25th starting at 12PM Eastern. I'll give about a 30 minute presentation on APT and then there will be a question and answer period for about 15 minutes.

warshipNot too long ago I saw a Tweet that mentioned privateering in the context of information security. I don't remember the details of the Tweet or the link that it might have pointed to, but it inspired me to think about the convergence of old maritime law, piracy, and cyber security. My dirty little secret is that information security wasn't my first choice for a career path. When I was growing up, I wanted to be a United States Navy Surface Warfare Officer. I had a bit of a complication when it came to that goal because at the time I was legally blind without corrective lenses. It turns out that the US Navy was sub-enthusiastic about the idea of a partially blind person commanding a powerful warship and they invited me not to join them. I ended up in law enforcement as a consolation prize and eventually caught the cyber security bug which brought me into the private sector.

While I wasn't able to join the team, so to speak, I have kept my love for military history and respect for the work that the military people do in their everyday roles protecting the rest of us. I started to think more about the privateering idea and was struck by how some of the themes from 18th century maritime warfare sound similar to today's cyber espionage issues. Privateering essentially was a practice where a nation-state outsourced some of its naval warfare to private actors who would engage and profit from attacking and capturing enemy shipping.  A privateer would be granted a letter of marque authorizing them to attack enemy ships. The privateers would then attack enemy ships and keep what they captured as payment for their services. The nation-state benefited by having enemy shipping disrupted without having to use their own limited naval resources and the privateer profited from the captured property. It was a nice flex and surge model where a country like the United States could ramp up to meet a threat from a more powerful adversary such the British whose navy was much larger and more powerful than the early United States Navy.

We've seen the reemergence of piracy in the Gulf of Aden that has caused problems for modern shipping. We have dispatched modern naval forces to combat these pirates* and there has even been some talk about using letters of marque to combat the problem. Congressman Ron Paul suggested that very thing in response to Somali pirates. Given that actually capturing Somali pirate vessels just results in grabbing some very unhappy pirates in a cheap boat with some side arms doesn't provide much profit motive, the idea appears to be to place a bounty on the pirates.  I don't know how great of an idea that it is, but the United States Constitution provides the United States Congress powers in area. Specifically, Article I, Section 8 authorizes Congress "To define and punish Piracies and Felonies committed on the high Seas, and Offenses against the Law of Nations" and "To declare War, grant Letters of Marque and Reprisal, and make Rules concerning Captures on Land and Water". 

I'm not a constitutional scholar so I'm not sure if Congress could even authorize Letters of Marque for cyberspace (Emails of Marque and Reprisal?) since the text is talking is specifically talking about the high seas. But setting that aside, if they could do that sort of thing, it would seem to be roughly applicable in the cyber security world. There are some striking similarities to the situation now in the cyber security world in regards to nation-state actors engaging in relentless cyber espionage against private industry and what the United States faced several centuries ago. Back in the 18th century, the United States was up against a very advanced adversary in the British Empire who had sea superiority because of their impressive naval service. The Congress wisely decided to use privateering to leverage private actors to help combat a threat that they could not deal with as effectively using their own naval power. In today's environment, I don't think anyone can say that we're winning against threats like Chinese cyber espionage.  There has been quite a bit of discussion about what role governments should take in protecting the their economic health by protecting their private sector from cyber espionage. Should there be a role for private companies to help defend themselves, their governments, and others by engaging in modern day cyber privateering? How would that even work? I can think of several broad models that could provide frameworks for how this could work.

The Active Model: The first would be an active model where a private entity is granted permission to engage in active measures outside of their organizational boundaries to stop attacks against them. This might include measures such as compromising machines and disabling computers that are being used as command and control (C2) platforms for attacks against them. The core of this model is the government granting a private entity to engage in active cyber warfare against an adversary.

The Passive Model: The second model would be limited model where organizations aren't necessarily allowed to engage in a full cyber shooting war against their adversaries, but are allowed to compromise external machines for purposes of gathering threat intelligence and determining attribution. This core of this model is intelligence gathering to improve the defenses of the organization being attacked and to provide that information to the government and other private organizations.

The Task Force Model: The third model would be something borrowed from the law enforcement community which the task force model where many different agencies send investigators to work on task forces focused on particular issues such as violent crime, terrorism, cyber crime, or drug crimes. A variation on this would be one where private entities (rather than just government agencies) donate personnel and resources to a government lead task force whose goal would be to combat cyber espionage targeting the private sector.  The private sector employees would be assigned to the task force for a certain number of years and then they could be called back to their home organizations where they can teach their internal security people what they learned during their time on the task force and new members can be sent to repeat the process.

A big initial problem I have with the idea of modern day privateering is that modern day networks aren't the same as the high seas. For example, there doesn’t seem to be the equivalent of international waters on the Internet. If you are taking an action against a computer that is attacking you, that computer is sitting inside of someone’s national borders. Great. The United States Congress gave you a “get out of jail free” card, but that's null and void in regards to that other nation’s borders. I'm not a lawyer. I have very little idea how the international law works in regards to this, but I suspect this is a show stopping problem with any sort of modern cyber privateering idea that doesn't involve government direction.

The Active Model just strikes me as patently wretched idea that is just begging to be a modern day information security example of the law of unintended consequences. Bringing down a server is serious business when it's not your own and it could very well be that the server that is attacking you could belong to an oblivious and innocent third party. It could very be your server next time that gets brought down by another privateer. It could also be that the computer you are attacking has been compromised by professional government cyber warriors from your country who made decision not to bring it down because the intelligence they are collecting from it is more valuable than stopping the attacks at that point. Now you've blundered into something you didn't know about and ultimately hurt your own cause. Lastly, this sort of privateering raises the stakes between you and the threat actor you've attacked. Maybe that threat actor will decide to return the favor and convince you to back off by not just stealing your data, but damaging your business operations by disrupting your network. We know how easy it is for advanced threat actors to get into business networks. Are these the people you really want to make angry? My guess is that this would look more like Phoenix Jones rather than John Paul Jones more often than not.

The Passive Model doesn't strike me as terrible of an idea as the Active model, but I'm still not fond of it. Yes, active measures to stop attacks aren't being taken and it's a model that encourages passive intelligence gathering. However, it still involves active measures such as compromising someone else’s computer and putting tools on it to collect data. This brings in most of the risks of the Active Model. There is also the issue of what the point is of collecting the data. Sure, maybe you learn gain more threat intelligence that you can use to defend yourself and pass onto others, but is it really worth the risks and expenditure of resources? What if you actually do manage to track an attack back to a particular nation-state with a reasonable degree of confidence? Then what?

I like the Task Force Model, but I don’t think that’s really privateering anymore given that I would envision that task force would be something that the government would lead and direct and that corporations and other entities would provide a substantial amount of people and resources to operate. Article I, Section 8 of the US Constitution authorizes Congress “to provide for calling forth the Militia to execute the Laws of the Union, suppress Insurrections and repel Invasions”. Essentially, this would be the government calling up a modern day cyber militia to repel cyber espionage against the United States.

* Being a modern day United States Navy Captain who is ordered to combat pirates has got to be a great assignment to get.  You're not playing second fiddle protecting an aircraft carrier and you get to experience what some of your early peers did during the age of sail. If I were a commander who received those orders, I've had a hard time not putting on a bicorne hat and ordering my crew to somehow rig a sail on my shiny modern Oliver Hazard Perry class frigate.

Friday, October 7, 2011

Congressional Hearing on Cyber Threats

I’ve spent quite a bit of time on the blog writing about Advanced Persistent Threat (APT) recently and I have also been working on creating some public presentations on the issue. I will be presenting on the topic of APT at the CISO Executive Summit in Chicago in December of this year and at the CISO Executive Summit in New York in 2012.  Additionally, I will be providing the keynote presentation for the SC Magazine eSymposium on APT later this month.

The United States House of Representatives Permanent Select Committee on Intelligence was nice enough give me more material to work with by holding a hearing this week on “Cyber Threats and Ongoing Efforts to Protect the Nation”. You can find the statements and witness testimony here. They make for very compelling reading, but the best executive summary of the proceedings was provided by Chairman Mike Rogers. His statement is on the website, but you can and should watch his statement on YouTube. It takes just under eight and a half minutes to watch and is an excellent summary of the severity and uniqueness of the threat that we’re facing from Chinese cyber espionage in particular. Rogers hit on all sorts of great points in his statement that everyone should listen to and understand.

I liked the fact that while he didn’t completely dismiss the idea of “Cyber Pearl Harbor”, he understood it’s really not the prime issue of the day. I’m of the opinion that this sort of threat is a bit overhyped.  Yes, a nation-state could use a cyber attack to do something like bring down a power grid by going after the computers that control it. The rub is that doing that would constitute an act of war and the victim nation-state could very well respond in kind using traditional warfare. I’m having a hard time thinking of a scenario other than a full shooting war where China would want to disable a power station in the United States using a cyber attack.  If they did, they might have to read about the effectiveness of that power company’s disaster recovery plan by the light of the fire from the smoking ruins that used to be their own power stations. Fine. You bring our power stations down using malware. We’ll bring yours down using cruise missiles from Virginia class nuclear attack submarines. We’ll see who gets the lights back on first.

The main point that Rogers made is the unique nature of this current threat. Espionage is probably the second oldest profession in the world so having a cyber component to isn’t anything particular shocking. Intelligence organizations have adapted to the information technology age by making cyber espionage a component of a proper intelligence gathering program. Rogers explained that the difference is that traditional espionage is oriented towards obtaining information on the “plans, intentions and capabilities” of other governments and militaries and that this current threat is much more expansive in scope. He summarized his view very nicely when he stated:

These espionage activities over the years, however, have largely been focused on collecting intelligence on foreign governments and militaries, not on brazen and wide-scale theft of intellectual property from foreign commercial competitors.

Rogers then went on to make this powerful statement about what makes this threat different:

I don’t believe that there is a precedent in history for such a massive and sustained intelligence effort by a government to blatantly steal commercial data and intellectual property. China’s economic espionage has reached an intolerable level and I believe that the United States and our allies in Europe and Asia have an obligation to confront Beijing and demand that they put a stop to this piracy. Beijing is waging a massive trade war on us all, and we should band together to pressure them to stop. Combined, the United States and our allies in Europe and Asia have significant diplomatic and economic leverage over China, and we should use this to our advantage to put an end to this scourge.

This is why I state that APT is a geopolitical problem. This is a problem that is bigger than any one of us and we have to band together to fight this threat. We need to pressure our respective governments to address it at the diplomatic and economic level. One of the ways we can do this is through the government relations teams for the organizations that have them. For example, it is very common for large corporations to have people devoted to petitioning government officials for changes in public policy. This is an issue that is critical for the financial health of these organizations and should be a priority for their government relations efforts.

Monday, September 5, 2011

Employee Infosec Academy

You get some of the best writing out of information security people when their cheese slips off their cracker. Frank McClain had just such a moment recently which provided me with a nice setup for this blog post. Frank’s cri du coeur included a description of a system administrator whose response to potential security incident was less than helpful. This system administrator was a good example of someone who doesn’t have a solid awareness of the current threat environment or just doesn’t care. Good system administrators are worth their weight in gold and because they are on the front lines of an organization’s information technology function, they need to be plugged into an organization’s information security program. This post will explore some ideas on how information security people can improve their working relationships in their organization.

I’m still amazed how many lessons I picked up as a patrol officer that I have been able to apply to my information security career. For example, when I was a police officer, there was a resurgence in the concept of community policing that emphasized establishing relationships with the community beyond just having police officers in squad cars answering calls for service. There was an understanding that law enforcement lost something when it made the transition from foot patrol based policing to vehicle patrol based policing.  What was lost was the community interaction and relationship building that police achieved when they had officers on foot assigned to dedicated areas where they would get to know the community and work with them on their problems in both a reactive and proactive manner.

Information security organizations should learn from police departments when it comes to community relations. The information technology people who make up your organization are a key part of the community that you are chartered to protect. System administrators, for example, can be your best friend or, as Frank illustrates, a great impediment to your work. If your only interaction with these people is reactive, you probably aren’t building strong relationships will make working with them much easier during an incident. Developing strong relationships with your information technology staff will make it more likely they will reach out to you when they see a potential issue. Early detection of incidents is critical to the security of the modern enterprise and a suspicious system administrator who sees and reports something odd could be the key event that causes you to get a “win” by early detection of an incident.

Information security organizations can build relationships by doing things such as giving periodic presentations to the information technology staff on the current threats that the organization is facing, what the security team is doing about it, and what should be reported to the security organization. Another concept that can be borrowed from the law enforcement community is the concept of a “ride-a-long”. Ride-a-longs generally take the form of a citizen riding with a patrol officer for all or a part of their shift. The same concept can be used where someone like a system administrator can shadow someone from the organization’s security operations center to see what life is like responding to incidents. This should also work the other way around where security people shadow someone like a network administrator to see what their lives are like and how the security people can better work with them. The one-on-one relationship building from this sort of activity will be invaluable to an organization.

An expanded version of the ride-a-long concept is the citizen police academy. This is a very popular program that has been embraced by law enforcement agencies around the country ranging from larger agencies such as the Las Vegas Metropolitan Police Department and the San Antonio Police Department to smaller agencies such as the Pflugerville Police Department. For example, last year I attended the FBI Citizens’ Academy that was put on by the FBI’s Newark office.  It was a fantastic experience that introduced me to many different aspects of the FBI and left me with a positive view of the organization that I retain to this day.

So who would attend an employee infosec academy? Because security should part of an organization’s culture rather than viewed as a separate function, the opportunity to attend the infosec academy should be extended to everyone. A special emphasis should be placed on having information technology employees attend the academy in an effort to build lasting relationships that are critical to protecting the organization’s infrastructure. However, the program would also be ideal for senior executive management so that they can better understand the role of the security organization and the challenges they face. Building a relationship with a chief financial officer and their staff, for example, could increase their confidence in the security function and lead to a better chance of successful budget requests for new personnel, tool, and training.

What would an employee infosec academy look like? The police have already done the heavy lifting for us. The model of having the academy class come together for several hours each week to learn about a different aspect of security is a great way to structure these classes. Since this is being done in an employment context, organizations could also run full-time academy classes that last a day or more depending on the amount of material to be covered. Specific modules that could be included in the academy would be things like architecture, policy, incident response, digital forensics, and risk assessment. The academy staff should take great effort to present their material in an manner that is informative, but also entertaining. Great pains should be taken to avoid death by PowerPoint style presentations. For example, the digital forensics team could conduct a demonstration where they use a free tool like FTK Imager to recover deleted pictures off a digital camera’s SD card to show how files can be recovered after deletion. The incident response staff could explain their methods for detecting and responding to incidents, show demonstrations of their tools, and walk the class through a real life incident that occurred inside of the organization. The risk assessment team could talk about how they conduct their assessments and walk the class through a completed assessment report.

For a security organization to meet the challenges of the current threat environment, it must be innovative and have the support of the organization that it protects. The law enforcement community recognized this same fact  long ago and created many innovative community programs in response. Information security organizations should learn from their peers in the law enforcement community by adopting and adapting these programs to their organizational needs.

Friday, August 26, 2011

APT: A Geopolitical Problem

An important thing to understand when thinking about advanced persistent threat (APT) is that it’s a much bigger problem than any one of us individually as organizations can handle because it's ultimately a geopolitical issue. We're talking about nation-states who are engaging in attacks against the confidentiality of sensitive data that belongs to other nation-states, their industrial base, academic institutions, and non-profit organizations. In other words, China isn't going to stop using cyber attacks as an active tool for its national security and economic development efforts until someone forces them to do so or their government changes radically.

Being targeted by a nation-state actor is a daunting thing to consider. Matt Olney, who is still the reigning champion of the pithy APT definition, wrote, "APT: There are people smarter than you, they have more resources than you, and they are coming for you. Good luck with that." Matt wasn't kidding when he said they have more resources that you. A nation-state has the ability to levee taxes and print money. I don't care what your organization’s profit margins and revenues were last year, they can't compete when it comes to outspending these people. Nation-states can have tremendous resources when it comes to personnel, intelligence gathering, education, and research and development capabilities. Jonathan Abolins made a fine point in response to my last blog post when he stated that if your organization is targeted by a nation-state for cyber attacks, it's almost certainly being targeted by more traditional physical data collection methods. Nation-states have comprehensive intelligence collection strategies where information warfare is just one piece of their strategy.

So we're cooked, right? Absolutely not. There are things that we can and should be doing to protect our individual organizations from these nation-state actors such as developing robust threat-based security teams. One of the best things we can do to combat this threat is to work hard to raise awareness so that other organizations will wake up and start fighting back also. Nation-states can have immense resources, but they aren't unlimited resources. They have to make resource allocation decisions just like anyone else. The more we collectively fight back against them, the more of their resources they have to expend to keep up with us.  Either they have to allocate more resources to keeping up their current level of overall activity or they have to start making tough choices on who to target and how much to spend on that particular target. Let’s make this really expensive for them.

When you fight back intelligently against this threat, you help everyone else out also. The business case for having your organization properly defend against this threat is the long term health and profitability of the organization. The altruistic case is that your efforts will likely help make others safer also by making hostile nation-states use more of their limited resources.  Maybe that resource drain means that some United States Navy commander at VFA-123 doesn't have to write condolence letters home to a pair of military families because that officer lost two naval aviators in an F/A-18 to an anti-aircraft defense system that was made better by stolen technology. 

This is a lot of vendor noise out there on the topic of APT, but I don't agree with those who say that we should abandon the term APT because of gross misuse by others. We have to fight misuse of the terminology just as we have to fight the misinformation about the subject itself. If we come up with a new term, the marketing people will just abuse it like APT so this a linguistic battle that I'm willing to fight.

So what can you do? The first thing you should do is to educate yourself about the nature of the threat so that you can cut through the noise and properly educate your organizational leadership. The people who I look up to and who are very influential in how I approach this issue are Richard Bejtlich, Rob Lee, and Mike Cloppert. I recommend starting by absorbing anything you can from them such as books, blogs, conference presentations, podcasts, random scribblings on cocktail napkins, articles, and Twitter feeds. There are excellent conferences such as the DoD Cybercrime and SANS Digital Forensics and Incident Response Summit (full disclosure: I teach digital forensics for SANS) that are held each year and include fantastic presentations on nation-state threats along with many other great topics.

You should also maintain at least a working knowledge of the business and geopolitical world around you. Since advanced persistent threat is a nation-state issue, it's important to understand what is happening in the world and how it connects to your daily life as an information security professional. There are resources such as The Wall Street Journal, The Economist, Brookings, Council on Foreign Relations, and Foreign Policy that all have robust and convenient online presences complete with mobile applications.

Even though I'm beating up on some vendors because of their misuse of terminology and sometimes FUD driven marketing, there are great vendors out there who provide a wide variety of tools, services, and educational efforts that are very helpful your efforts. I’ll try and highlight as many as I can in future blog posts. One example is Mandiant who does a fantastic job of educating the community about the nature of advanced persistent threats as well as threats from other actors. They are very open with what they know and I highly recommend their frequent webcasts.

Sunday, August 21, 2011

Advanced Persistent Threat

Like most people who have a strong interest in incident response and information warfare, I follow Richard Bejtlich’s blog and Twitter feed closely. Richard fired off a series of excellent Tweets this week talking about the “advanced” aspect of advanced persistent threat (APT). This inspired me to write this APT post that I’ve had in my head for some time now, but was hesitant to write because there is so much written these days about APT that is quacking and barking from vendors and others who don’t understand what they are talking about. I just didn’t want to contribute to all of the noise about the issue when you already have established experts like Richard, Rob Lee, and Michael Cloppert who are wonderful with this topic.

We’ve gone from a situation where the term APT was used by government cyber warfare people along with some of their partners in the private sector to something that is grossly misunderstood and misused by people who pretend to be experts, but have no earthly clue what they are talking about. Ever since the Operation Aurora information hit the media and information security tool vendor’s marketing departments consciousness, we’ve been inundated with all sorts of shrieking and wailing about APT from vendors and self-appointed experts. Most of it has been noise from people who don’t understand the issue and/or are using the term as a cynical marketing ploy for their products. Yes, of course, tools are important in defending against advance persistent actors as well as other threat actors. However, my eyes pretty much glaze over when I see the words advanced persistent threat as part of vendor tool marketing campaigns. I’ve lost count of the number of times I’ve read marketing information that wants me to think that the vendor has created some amazing unicorn blood fueled tool that will solve all of my problems and not require me to do much else other than to write them a big check each year.

Richard’s Twitter feed is always excellent and the Tweets that he crafted this week were simply brilliant and inspired me to write this post. Some of them where:

“Bruce Lee fought using sticks, nunchakus, or his bare hands. He must not have been that advanced or powerful. Sort of like APT, eh? #sarcasm”

“The Army uses mules to move cargo across mountains in Afghanistan? Those guys must be as advanced as the Spartans! Like APT. #moresarcasm”

“My point is when you only judge an adversary by the TOOLS that YOU see him using (2 errors there) you're making a big mistake. That's #APT.”

I don’t know if a particular event caused Richard to write this series of Tweets, but a personal hot button of mine are people who say that because a particular tool or technique was not advanced, it means that an APT actor was not involved. This is nonsense on stilts. As I pointed out on my own Twitter account, just because the tools and techniques that knocked you over weren’t “advanced” doesn’t mean it was not an advanced actor. It could very well just mean that your defenses were so inadequate that the attacker didn’t have to work very hard to defeat you. It could also mean that there were advanced tools and methods that were part of the campaign against you that escaped your detection or understanding.

As Rob Lee has taught us, APT is a “who” not a “what”. In regards to the who, APT is ultimately nation-state actors like China who are aggressively pursing the theft of a wide range of information information they consider vital to their national interests. It is important to understand that these nation-state actors have broad national interests that extend well beyond military technology. That is why we’ve seen so much APT activity targeting organizations that aren’t in the defense space. Remember the event that made many people aware of this threat was the Operation Aurora incident where organizations like Google, Yahoo, Symantec, Adobe, and Dow Chemical were reported to be targeted along with human rights organizations and think tanks. Some people in the field will also extend the definition of APT to sophisticated organized crime groups who target organizations to steal data such as credit card information. Reasonable people can and do disagree on the definition of advanced persistent threat, but there are definitions that are just silly. For example, you should be very suspicious of a definition that requires the use advanced tools for the activity in question to be classified as being advanced persistent threat.

“Advanced” doesn’t mean the attacker uses sophisticated malware in each attack. Even advanced attackers have limited resources. They aren’t going to send their top people with their best tools after you if it’s not necessary. They have to make decisions on resource allocation just like you do. If you get knocked over by a low tech attack, it might still be an advanced actor, but it could very well mean that you aren’t good enough for them to deploy their best operators and weapons systems. As I heard someone say awhile back, if an organization has its administrator credentials compromised and the attacker is using them to compromise additional computers, we don’t call that hacking anymore, we just call that logging in.

All that said, advanced actors can and will deploy some very sophisticated tools when necessary to achieve their goals. The anti-virus vendors can’t keep up with these attackers which is why anti-virus technology, while necessary, isn’t a comprehensive solution to countering their tools. This is why malware analysis is a key aspect of defending against APT operators. I see it as one of the few areas where the defenders have an advantage against attackers compared to traditional warfare. In traditional kinetic warfare, an attacker can successfully use a sophisticated weapons system such as a stealth fighter and the defender will not have the opportunity to examine that weapon unless they capture it. Additionally, that capture is likely to come after the weapon system has been significantly damaged which will make a full exam more difficult. With cyber warfare, the attackers are commonly leaving their weapons systems behind on the defender’s networks. Many times these weapons are in perfect operating condition. Malware analysis is a vital part of an effective defense strategy against advanced persistent actors. It’s a critical part of incident response because gaining a fuller understanding of malware being used against you can provide the team with additional indicators of compromise which can be used to detect the scope of the attack against you. It also important to your threat intelligence function because it can provide valuable intelligence about who is going after you. This is important information that will aid in your defense especially when compared to your existing body of intelligence data. Because malware analysis is so important, it’s also important to make sure that your team has the ability to do malware analysis beyond just the behavioral level. A fully functional malware analysis capability will include malware analysts who can use skills such as knowledge of assembly language to reverse engineer the tools used by the attackers. If you don’t have a proper malware analysis capability, you are ignoring one of the few advantages defenders have against advanced persistent attackers.

The advanced actors aren’t stupid. They understand that this is a problem that they have especially when they go up against advanced defenders. If their weapon system falls in the hands of a sophisticated defender, it could be reserve engineered and the defender will use that knowledge to defend themselves. Even worse, the defender might share what they have learned with others which can lead to the weapon system not being as effective against other targets. So if they don’t have to use advanced malware against a defender, why would they want to use it? It’s better to use something simple to complete a successful attack and save the more advanced tools and methods for when the basic tools and methods won’t get the job done.

If the eye of an advanced persistent actor like China has fallen upon your organization, you’re in for a long term struggle that won’t end anytime soon. Persistent means just that. APT isn’t something you spray for once and forget about, it’s something that you have to continuously fight for control over your network. That’s hard news to have to break to your organizational leadership, but the sooner they accept this, the better off your organization will be in the long run as it works to defend its intellectual property, business processes, and sensitive internal communications. Yes, of course, it requires good tools, processes, and proper funding to accomplish an effective defense, but your success against APT will live and die by the quality of your people and the leadership that you provide them. It’s imperative that whoever is leading your effort against the advanced persistent actors have a strong understanding of the nature of the threat and the leadership skills to build and lead a highly effective team.