For over twenty years, we have been testing language-based methods to find out if these methods can be trusted. During these twenty years, we have found that some readily-accepted, popular ideas just don’t work when they are actually tested, and we have found some that do work well. We report these findings so that our stakeholders will be wary of methods that aren’t reliable and consider using methods that do work well.
We run our tests outside of any litigation on “ground truth data,” (data that we know the correct answers for, like who really did write which document). Trying to figure out if a method works in the middle of a case is far too late; we work proactively to develop tools from methods that we know work, before a case arises. And testing a method on data that is not “ground truth” (even if the dataset is easy to get or convenient) is futile as no real error rates can be calculated and the researchers never really know how well a method worked to get the correct answers.
When our research –testing, tweaking, re-testing– gets a method to an acceptable level of reliable performance, we transfer the technology over to our sister organization, ALIAS Technology LLC (which you can find out about under Tools).
And during these twenty years, we have met with resistance and other obstacles. We continue our validation testing and empirical research, even if it is not popular in some quarters, because our system of justice often relies on forensic evidence. The strongest system of justice and security cannot be built on shoddy investigation or junk science: only the strongest science can meet the needs of a democratic system of justice and security. Here in the United States, we are so blessed with a justice system which relies on both prosecutorial discretion and a strong defense. Prosecutors I have worked with want to make their case based on strong forensic evidence: they don’t want to put forward junk science. Defense attorneys I have worked with also want to use the strongest forensic science for their clients. Yes, of course, there are experts in the system more interested in money than justice, fame than fairness; every profession has that kind of problem. But at ILE we believe that fundamentally most of us involved in the justice system are all looking for the same thing: good investigation that finds evidence, and strong forensic science to analyze the evidence.
My colleagues and I are working to make sure that forensic linguistics matures into a strong forensic science, able to serve the security, legal and intelligence communities with robust tools for reliable results.