Saturday, November 13, 2010

Certification, Licensing, and Accreditation in Digital Forensics

Considering the subject matter that I’m going to be wading into with this blog post, I want to start off by doing some full disclosure.  I’m a member of the Board of Directors for the Consortium of Digital Forensics Specialists (CDFS) and I’m also in the orbit of the SANS Institute. I’ve done both volunteer and paid work for SANS and the Global Information Assurance Certification (GIAC). I’m hoping to teach my first Community SANS class for them sometime in 2012 which would be a paid engagement.  As always, I speak only for myself on this blog and what I write does not necessarily reflect the views of any organizations that I’m associated with such as CDFS or SANS.

Some of the hottest topics of discussion in the digital forensics community are the issues of certification, accreditation, and  licensing.  In fact, one of the most common errors that I see in these discussions is confusing the terms and their goals.  In the digital forensics community, these terms have specific meanings that I would like to try and define up front.

Certification takes the form of an outside entity who certifies that an individual has met some sort of minimum standard of competency in an area of digital forensics.  The entities that do this inside of the digital forensics community are legion and include organizations such as the International Society of Forensic Computer Examiners (ISFCE), the International Association of Computer Investigative Specialists (IACIS) and GIAC.

Accreditation, for the purposes of this discussion, is an outside entity such as the Forensic Science Accreditation Board (FSAB) or American National Standards Institute (ANSI) who through an accreditation process validates that  a digital forensics certification or organization meets its minimum standards.  For example, GIAC has several of it’s certifications accredited by ANSI including the GIAC Certified Forensic Analyst (GCFA) certification.  There are several entities such as the Digital Forensics Certification Board (DFCB) and IACIS who are interested in pursing FSAB accreditation.

Licensing is a government entity regulating a particular profession in such manner where it becomes unlawful to engage in certain professional activities without a license. There are a whole host of professions that are regulated in this manner to the extent that a person needs government permission to engage in activities such as private investigation, practicing medicine, cutting hair, giving therapeutic massages, and a long list of other activities.

Two out of the three of these things are good ideas for the digital forensics community.  Certification of practitioners and the accreditation of the bodies that certify them are vital to professionalizing the industry and helping us progress as a community.  The licensing of digital forensics practitioners is a bad idea regardless if digital forensics practitioners are required to be licensed as private investigators or specifically as digital forensics examiners.

I’m not an absolutist when it comes to licensing.  I understand that in certain limited cases pertaining to critical issues such as public health and safety, there is an important role for government to play in regulating certain activities.  However, it’s important that we as community understand that the history of professional regulation has not been a rosy one.  Much of what we see here in the United States relative to professional licensing is just a modern day version of the guild system where professions use licensing  to keep out competition and control the market.

The common case that is made by those who support the licensing of digital forensics is that it will somehow increase professionalization by weeding out those who are unethical or incompetent.  This gets into a common mistake that is made by supporters of licensing which is to assume that licensing is a measure of competency.   While it’s true that, licensing arrangements frequently mandate some sort of training in the professional area, this is not necessarily a measure of professional competence. In the cases when testing is performed as part of the process, it is generally used to validate regulatory knowledge rather than professional competency.  It’s that mandatory training requirement (if one exists) that allegedly insures professional competency. Not coincidentally, it’s also what is used to establish modern day guilds that we see in professions like law, medicine, and even massage therapists. 

Because digital forensics is a convergence of technology and law, we already have measures in place that protect the public from unethical and incompetent examiners and methods.  We have standards like Daubert and an adversarial legal process that has well established methods of vetting those who would act as expert witnesses during legal proceedings.  Licensing of digital forensics people is unnecessary in the face of well known and accepted gatekeeping processes for legal proceedings. 

Not only is it unnecessary, but it’s harmful for both the profession and the public.  This is because licensing will likely result in a digital forensics guild system where the government will decide who can practice digital forensics and who can’t.  It will do this without much serious thought to the issue of professional competency which is the banner in which proponents of digital forensics licensing frequently rally under.

One argument is that a digital forensics licensing system can be established that would provide for competency assurance by requiring that licensees have a certification in digital forensics from an approved entity.  This is unhealthy for the community because it could very well result in the various certification organizations having to put a lot of time and money into lobbying the various government entities to allow their certification to be one of the approved certifications.  It gets worse if a government regulatory body were to decide that they were only going to accept one digital forensics certification as the standard for licensing.  That will put the certification bodies in direct adversarial competition with each other to make themselves the standard for that regulatory body.

There also is the issue of law not keeping up with technology which is a frequent occurrence in the digital age.  Even if I were to allow myself to be swayed by some siren song of licensing, how does state specific licensing work here in the United States?  Licensing systems are generally done at the state level.  Digital forensics is very much an interstate and international issue.   What if you have a case that requires you to engage in regulated activities in many states where a license is required for each one?  What if each of those states not only requires a license, but they also require different digital forensics certifications as part of that licensing process?

We don’t need a modern day digital forensics guild system.  We are capable as a community to regulate ourselves through collaborative efforts like the CDFS, the various well established and respected organizations like ISFCE and IACIS, and through the legal system’s standards in vetting people who provide testimony in legal proceedings.

Just say no to digital forensics licensing.

Certification and accreditation are something that we should embrace as a community in part to help ward off any licensing efforts by the government.  This should be an area of common ground between those who support licensing and those who support industry self-regulation.  For example, if one supports licensing of digital forensics professionals as a way to ensure basic competency, there has to be some sort of competency testing component to that process. That component can be achieved by professional certification through the various digital forensics certification bodies.

If we are going to be taken seriously as a profession, we ourselves have to take our profession seriously.  That means coming together as a community to establish minimum standards of competency for digital forensics examiners and providing methods in which examiners can show that they have met these standards.  We have many respected organizations who have spent a lot of time and effort doing that very thing and judging by the amount of people I see who hold digital forensics certifications, we have embraced those efforts as a community.

It’s important to understand that certification does not mean mastery.  It just means that an outside organization has validated that an individual has met the minimum standards as defined by the organization.  In fact, certification doesn’t necessarily even mean professional competency.  Ask any digital forensics hiring manager and they will be able to provide you with stories of certified applicants who failed their hiring process because of a lack of technical competency.  Doing a week of digital forensics training and then obtaining a certification doesn’t mean that someone is necessarily a competent digital forensics examiner, but it’s a start especially someone who is interested in getting into the field.

Accreditation is a key component of certification.  It’s essentially the certification bodies being certified themselves by a trusted outside entity such as the FSAB or ANSI. As a community, we should be pushing the various certification organizations to advance the cause of digital forensics professionalism by pursing accreditation.   We should do this because our professional organizations and their associated certifications will be taken more seriously if these organizations can show that they are following industry standard practices when it comes to the credentialing of digital forensics practitioners.

GIAC went the ANSI route and I think that means that the GCFA certification might be the first digital forensics certification that has achieved accreditation from a well recognized standards organization.

I know IACIS (I’m an associate member) is interested pursing FSAB accreditation.  That’s great to see because IACIS has spent a lot of time and effort into making their CFCE certification into a well respected certification in the digital forensics community.  They recently made the decision to open up that certification process to those who aren’t members of IACIS which is part of what needs to happen for FSAB accreditation.  The FSAB prohibits membership in an organization as a requirement for certification.  I’m not sure when the certification will be available to the public, but IACIS is working on getting that done.

One of the primary premises behind the DFCB is to establish an industry standard digital forensics certification that would achieve FSAB accreditation.  This effort hasn’t gone all that smoothly, unfortunately.  The “Founders” Digital Forensic Certified Practitioners (DFCP)  process that I went through to achieve my DFCP certification was disorganized and understaffed.  Since that time, I haven’t seen much in the way of improvement when it comes to communication and organization on the part of the DFCB.   They haven’t been very good when it comes to communication of what is going on with the organization and what progress is behind made towards their ultimate goals. Transparency hasn’t been a hallmark of the organization.   For example, I would like to know who makes up the various committees.  The website lists who leads their committees, but not who are members, what the committee goals are, and what progress has been made towards those goals.  Early in their history they posted some documents of this nature pertaining to early organizational meetings, but that has not occurred in some time. I’ve yet to find a DFCP certified person who is happy with the organization. They mean very well, but they’ve clearly had some trouble when it comes to communication and execution. I’m hoping things will get better for them as they pick up some momentum because their stated goals are laudable. I would also like to see at least one digital forensics organization achieve FSAB accreditation.

10 comments:

  1. Thanks for this very informative article! You present your view very well.

    I remain skeptical about industry self-regulation, however. Without government involvement, I don't see how we can drive frauds and incompetents out of the profession. It seems to me that computer security in general has passed through its Wild West phase when any hacker can sell services to clueless victims. PCI-DSS shows industry self-regulation, allowing WEP wireless encryption until 2010.

    I don't think the computer security industry is able to police itself well at all. Government licensing is inevitable, because computers are as important and as dangerous as cars or medical devices.

    ReplyDelete
  2. ( I am also a biased party, Chair Board for GIAC )

    That said, I am not hearing anything from CDFS on the street, but maybe I walk the wrong streets?

    The FASB bit is truly interesting, I see your point. Gosh accreditation is a significant pain point. We are on revision 19 of our STI.edu strategic plan to get it accepted by middle states as .0001% of the accreditation and I lecture on strategic planning.

    And I do not disagree with Sam, but I do think we will have higher quality if we can self police.

    ReplyDelete
  3. Sam and Stephen,

    Thanks for stopping by and posting your thoughtful comments. Reasonable people can and will disagree on these issues and I hope my post contributes to the discussion in a constructive manner.

    We already have government involvement. As I pointed out in the post, the judicial branch of government is already very much engaged in detecting incompetency through well established means that apply to digital forensics examiners and other experts. In regards to digital forensics, the judiciary is becoming increasingly knowledgeable and sophisticated because of the eDisco issue.

    If my experience with home contractors is any guide, a licensing hasn't done a great job weeding out those who aren't very good at what they do.

    I'm also concerned that a modern day guild system could stifle innovation in digital forensics. If a state licensing body for digital forensics decides to enforce it's version of best practices, it could result in practitioners hurting their clients via speed, cost, and quality issues because they were concerned that a new tool or technique could result in disciplinary action.

    I learned from Stephen that accreditation is a long hard slog. This is why I think organizations like SANS\GIAC and IACIS have some advantages in obtaining accreditation. They have extensive experience when it comes to establishing standards and then creating testing processes to measure those standards. As I review the FSAB standards for accreditation, that's only one part of the accreditation battle, but it's an important one.

    I'll post more about the AAFS/FSAB aspect in a follow up blog post.

    The CDFS is coming along nicely. I'm not sure what the "go live" date is going to be, but I hope the community likes what we come up with.

    ReplyDelete
  4. Eric,

    Excellent post. I'm still on the fence about this issue, a bit. I want to come down on the side of industry self-regulation. But I see one flaw with your idea that the courts are an adequate protection against frauds and charlatans. How would you address the argument that, by the time a client gets to court (if at all), they've already potentially been defrauded if they happen to contract with an unqualified or under-qualified examiner? In other words, if someone poses as a certified examiner (regardless of specific certification held), what's to protect an organization from that kind of fraud? That's where I see licensure being a possible, though problematic, level of protection. It's then up to the licensing board to vet the licensed individual and ensure that he/she holds all necessary certifications/credentials.

    Creating new laws and licensing boards may arguably be an overreaction to that particular issue. But what's the alternative? Do we insist, instead, that organizations contracting/hiring examiners simply do their due diligence?

    ReplyDelete
  5. Eric,

    I agree with much of your article but I think you missed the boat on the accreditation issue. When dealing with digital forensic, the topic of accreditation generally has to do with the accreditation of labs by organizations like ASCLD/LABS. This is a completely different aspect of accreditation.

    There has been a push in recent years to require mandatory lab accreditation for anyone performing computer forensics. I strongly disagree with this stance. ASCLD/LAB accreditation is designed for crime labs and has been forced to fit the digital evidence community. It is not a good fit.

    I believe each lab should have written protocols and guideline for examiners to follow but considering many of the digital forensic labs in police departments across the United States are one or two person shops, the ASCLD/LAB approach is not realistic. Mandatory accreditation of these labs will mean many local PD labs will have to close.

    I urge you to please articulate in future articles that accrediting certifying bodies (FSAB) is different than accrediting digital forensic labs (ASCLD/LAB).

    Thanks for the article.

    Troy Lawrence

    ReplyDelete
  6. Troy's comments about "missing the boat" on accreditation are correct. Accreditation in our field typically references Forensic Lab accreditation. However, Eric makes a point that is often messed. It is very true that Certifying Bodies also need to be accredited to prove they have met a certain standard in order for their certification to have legitimate merit. This is similar to why most of us attended a College that was accredited by some Accrediting Body (such as the Southern Association of Colleges and Schools) and not just a late night infomercial class. You can learn something from any class you take, but will the credits you earn from the class actually transfer anywhere else? That is one of the benefits of the school being accredited.

    Now for my full disclosure. I am the Lab Director for an ASCLD/LAB accredited lab (in the field of Digital Evidence, Computer Forensics) and the owner of a for profit company. We are not a government agency.

    I respectfully disagree that Troy’s stance that accreditation is not a good fit for the Computer Forensics community. Our company has a total of 3 employees (we did have a 4th person last year) and we were able to earn and we maintain our accreditation and still handle our case load, and in our case make money, because after all that is why you have a for profit business.

    ASCLD accreditation shows our customers and more importantly the justice system that we have verified protocols, procedures, standards, quality assurance, security, insurance, licensing, and a host of other criteria that show that we understand how to process evidence and that we in fact handle evidence properly. It is my belief and one of the reasons I chose to have our lab earn accreditation, that accreditation will be forced upon us in the next few years and it will not be because of anything the computer forensics community has done wrong.

    Congress will be the body that makes this a requirement and it will be because of the mishandling of DNA evidence in yet another Death Row case. Congress has stated that they are feed up with mistakes in Criminal Labs around this country and I feel that they will take the larger step of requiring accreditation for any lab that presents evidence in any court proceeding. I may be wrong, but time will tell. In the meantime, our customers and the attorneys and judges of the cases in which we process evidence understand the value of our accreditation. After all, our small, little private lab has earned the same accreditation as the FBI, Secret Service, DoD, and many other well respected government investigative agencies.

    Thank you,

    Neil Broom, President
    Technical Resource Center, Inc.
    An ASCLD-LAB accredited facility

    ReplyDelete
  7. One important thing to remember about licensing because it is required in certain states is this...

    Licensing in a field in usually not determinative, but in one case, the court held
    that a witness not licensed to investigate fires under a state statute was not
    qualified to testify about the cause of a fire in an arson prosecution. People v.
    West, 264 Ill. App. 3d 176,636 N.E.2d 1239 (1994).

    If licensing is a requirement in your state, and you ignore the requirement, you may not be allowed to testify. If that happens and your client’s case is lost because of your professional negligence, you may be sued. Plus, you may be arrested for breaking a state law.

    It is one thing to argue whether licensing is a good idea or not, but licensing is a reality in many states already and I don’t see that changing anytime soon. In Georgia, you have to be a licensed Private Detective to perform computer forensics investigations and you have to include your company Private Detective number on any advertising to do. The State will issue Cease and Desist Orders on unlicensed investigators.

    ReplyDelete
  8. Gregory,

    Thanks for dropping by. My response is simply that licensing doesn't necessarily prevent your scenario. You can easily have licensed people who are terrible at their jobs and who will harm their clients due to their incompetence. Licensing doesn't, for example, guarantee a good haircut, but it does result in the absurdity of "barbering without a license".

    http://www.orlandosentinel.com/health/os-illegal-barbering-arrests-20101107,0,1491315.story

    ReplyDelete
  9. Troy,

    You caught me. When I was writing up this post, I was concentrating on the issue of accreditation for digital forensics certifications. I wasn’t thinking about the issue as it related to digital forensics labs at all. I became aware of this issue some time ago, but I hadn’t heard much about it so it had dropped off my intellectual radar. I agree that mandatory ASCLD/LAB accreditation for digital forensics labs is a bad idea for the reasons you articulate. That’s an easy sell for me. A voluntary accreditation process that is specifically tailored to digital forensics labs and realistically scales might be a good idea…or it might not. I’ll have to think about it some more, but I’m concerned about the impact of that sort of thing on not only small law enforcement agencies, but private sector actors of all sizes.

    I think we may be making a big mistake as a community to even use the term “labs” to refer to what might be better articulated as something like digital forensics data centers or something similar. We’re not a lab in the traditional sense that we handle easily perishable items of evidence like blood that can easily become tainted or destroyed if not stored or processed properly.

    I’m not sure even a voluntary accreditation system would make sense for someone who is, for example, operating a small scale incident response and forensic analysis consulting firm. Do we really need to have an operation’s “lab” become accredited when it’s just a single person doing incident response, forensics and malware analysis work?

    Thank you so much for your comments. I’ll have to think about this some more and, yes, I’ll bring this issue up in a future blog post. Great idea.

    ReplyDelete
  10. Neil,
    Thank you so much for taking the time to post your articulate commentary on the issue. As we can see, reasonable people can and do disagree on these issues. I have to think about this some more and I can’t claim to have a fully developed opinion on the issue of lab accreditation in its entirety. As I wrote to Troy, being opposed to mandatory ASCLD\LAB accreditation is easy enough for me, but that still leaves an open question of whether, for example, a voluntary process specifically tailored to digital forensics is a good idea.
    Your comments on the mishandling of DNA evidence are well taken. However, I’m wondering if that’s an illustration of how the digital forensics community might be making a mistake by labeling our operations as “labs”. We’re really not operating labs in the same sense that an outfit that is processing biological evidence. Does a home office that includes a couple desktop computers designed to do digital forensic and malware analysis work a lab?

    Your points about licensing are well taken also and are a good illustration of the consequences of licensing. Requiring a private investigators license to perform digital forensics is unreasonable and is something that the community should aggressively fight against.

    I also understand the inevitability argument even though I still chose to resist it. I know at least one person who I deeply respect who has made this argument privately to me and has done it in a compelling manner. However, most things aren’t inevitable until they happen. Bad laws and regulations can be rolled back.

    That said, if we ever do hit the point of inevitability (or even if we’re there now) and we have to accept licensing as community, it should be done in a manner that is specifically tailored to digital forensics rather than for private investigators.

    ReplyDelete