Board of Forensic Document Examiners
BIAS in FORENSICS
2014 AAFS Bias in Forensics Workshop Gets Rave Reviews and Andrew Sulner Receives 2014 Kenneth S. Field Award
2014 AAFS Bias in Forensics Workshop Gets Rave Reviews and Andrew Sulner Receives 2014 Kenneth S. Field Award
The February 17, 2014 multidisciplinary workshop on Bias in Forensics organized by AAFS Jurisprudence Section Chair Andrew Sulner and Innocence Project co-Director Barry Scheck highlighted Day 1 of the weeklong 66th Annual Scientific Meeting of the American Academy of Forensic Sciences (AAFS). Two days later, AAFS President Barry Logan acknowledged Academy VP Betty Layne DesPortes and Andrew Sulner as the recipients of the 2014 Kenneth S. Field Award for Outstanding Service to the Academy.
The February 17, 2014 multidisciplinary workshop on Bias in Forensics organized by AAFS Jurisprudence Section Chair Andrew Sulner and Innocence Project co-Director Barry Scheck highlighted Day 1 of the weeklong 66th Annual Scientific Meeting of the American Academy of Forensic Sciences (AAFS). Two days later, AAFS President Barry Logan acknowledged Academy VP Betty Layne DesPortes and Andrew Sulner as the recipients of the 2014 Kenneth S. Field Award for Outstanding Service to the Academy.
Highlights of the Workshop
William Thompson, PhD, JD began the day with his presentation, “Blind to What? Criteria for Domain Relevance.” Professor Thompson explained the necessity for each forensic discipline to define the criteria that are “domain-relevant” for that discipline and to create strategies to shield forensic examiners from information that is not “domain-relevant.” One of the strategies discussed was “sequential unmasking,” in which information is disclosed (“unmasked”) only after the examiner has completed all the examinations that could be biased by the introduction of that information.
Dan Simon, JD continued this line of thought with his presentation, “Cognitive and Motivational Causes of Investigative Error.” Professor Simon explained how errors can compound upon each other, and the critical need for accuracy in observations, reporting, and testimony. He illustrated various dangers in forensic investigations, such as selective framing (in which an inquiry is framed in terms designed to influence the outcome), selective exposure (in which the information provided is chosen to influence the outcome), and selective stopping (in which the inquiry is ended when the hypothesis appears to be confirmed, but all possibilities may not yet have been considered).
Partially in response to the 2009 NAS Report on Forensic Sciences, the District of Columbia created the “Department of Forensic Sciences” as a crime laboratory independent of any law enforcement agency. Max M. Houck, PhD, its Director, explained the unique aspects of the administrative structure of this laboratory and described some of the challenges and pitfalls in striving to maintain maximum independence (financial and political).
D. Michael Risinger, JD completed the morning presentations by discussing the need to have sequential unmasking protocols in order to ensure that a forensic expert relies only upon domain elevant information in reaching a conclusion from evaluating evidence, the reasons sequential unmasking is resisted by practitioners and whether those reasons make sense, the importance of allowing only the trier-of-fact (judge or jury) to combine information from other sources to reach ultimate conclusions, and finally, methods for cross-examining experts about bias.
After the lunch break, Workshop Chair Andrew Sulner, MSFS, JD presented his program, “Examining Sources of Bias and Illustrating Their Impact on Handwriting Opinions and Expert Testimony of Forensic Document Examiners.” As both an attorney and a 3rd generation forensic document examiner, Mr. Sulner used actual cases and published court decisions to illustrate how even experienced forensic document examiners can allow bias to improperly influence the outcome of their handwriting investigations and the manner in which their opinions are reported or testified to in court. Mr. Sulner also discussed various de-biasing techniques and examination procedures that can be used to enhance the accuracy of signature and handwriting investigations, one of which is the Fischhof Method of “upside-down” comparison that was developed by his grandfather, Professor Julius Fischhof, a pioneer in the field of questioned documents and Eastern Europe’s leading handwriting expert in the 19th century.
Barry C. Scheck, JD, co-Director of the Innocence Project, discussed “Minimizing Bias in Post- Conviction Inquiries Into Possible Miscarriages of Justice: Moving From an Adversarial to an Inquisitional Framework.” Professor Scheck spoke from his experience in working with Conviction Integrity Units in District Attorney’s offices across the country and addressed the problem of bias in “conviction integrity” re-investigations. The use of internal systemic safeguards in the medical industry and business community were discussed as models that could be adapted for use in the legal system to minimize the effect of bias in reinvestigations of potentially wrongful convictions.
Saul M. Kassin, PhD, who was scheduled to present “Forensic Confirmation Bias: How Confessions Corrupt Perceptions and Judgments,” was unable to attend due to an unexpected event. However, Dan Simon did an admirable job filling in, using Dr. Kassin’s slide show presentation to discuss the three classic types of false confessions that have been identified, and to illustrate why a forensic analyst’s awareness that there has been a confession can have a strong biasing influence upon the outcome of a subsequent forensic analysis by creating “corroboration inflation,” whereby the confession itself “produces an illusion of support from other evidence.” This presentation demonstrated why safeguards are needed to identify and prevent false confessions, and to prevent the “ripple effect” of knowledge of
confessions influencing other evidence, including forensic analysis.
In presenting “Forensic Expert Bias: A Judicial Perspective,” Donald E. Shelton, JD, PhD discussed expert bias in terms of the Federal Rules of Evidence and Daubert challenges to expert testimony, the use of “bias experts” to discredit expert testimony, and the use of jury instructions to address issues of potential bias on the part of expert witnesses. Judge Shelton also discussed and illustrated examples of outcome bias on the part of judges.
The final speaker of the day was Steven T. Wax, JD, the criminal defense attorney for Brandon Mayfield, the Oregon attorney who was wrongly accused of the 2004 terrorist train bombings in Madrid, Spain. His riveting description of the facts and circumstances of the Mayfield case, with chilling first hand accounts of the obstacles he faced defending an innocent client faced with what seemed to be incontrovertible proof of guilt, brought all the teachings of the day into sharp reality by clearly demonstrating most of the dangers of bias that had been discussed by the previous speakers.
This workshop made it abundantly clear that awareness of bias is not sufficient, and that preemptive action must be taken to minimize the potential for bias to contaminate forensic investigations and impact the outcome of forensic analyses. Each of the workshop presentations (see Program) was powerful and important, and the beauty of the day was the way each built on the previous presentation(s) to create a whole that was more than the sum of its parts. Attendees had enormous praise for the workshop and interesting questions for the Q&A session that lasted an extra hour at the end of a long day. Attendees took home with them a 600-page compendium of articles, presentations and source materials for future study on this important topic of Bias in Forensics.
This was the second workshop organized by Mr. Sulner and presented at an AAFS Annual Scientific Meeting. The first, entitled "Flawed Forensics: Recognizing and Challenging Misleading Forensic Evidence and Disingenuous Expert Testimony," was presented at the 2012 Atlanta meeting and was also widely acclaimed. For details, click here.
William Thompson, PhD, JD began the day with his presentation, “Blind to What? Criteria for Domain Relevance.” Professor Thompson explained the necessity for each forensic discipline to define the criteria that are “domain-relevant” for that discipline and to create strategies to shield forensic examiners from information that is not “domain-relevant.” One of the strategies discussed was “sequential unmasking,” in which information is disclosed (“unmasked”) only after the examiner has completed all the examinations that could be biased by the introduction of that information.
Dan Simon, JD continued this line of thought with his presentation, “Cognitive and Motivational Causes of Investigative Error.” Professor Simon explained how errors can compound upon each other, and the critical need for accuracy in observations, reporting, and testimony. He illustrated various dangers in forensic investigations, such as selective framing (in which an inquiry is framed in terms designed to influence the outcome), selective exposure (in which the information provided is chosen to influence the outcome), and selective stopping (in which the inquiry is ended when the hypothesis appears to be confirmed, but all possibilities may not yet have been considered).
Partially in response to the 2009 NAS Report on Forensic Sciences, the District of Columbia created the “Department of Forensic Sciences” as a crime laboratory independent of any law enforcement agency. Max M. Houck, PhD, its Director, explained the unique aspects of the administrative structure of this laboratory and described some of the challenges and pitfalls in striving to maintain maximum independence (financial and political).
D. Michael Risinger, JD completed the morning presentations by discussing the need to have sequential unmasking protocols in order to ensure that a forensic expert relies only upon domain elevant information in reaching a conclusion from evaluating evidence, the reasons sequential unmasking is resisted by practitioners and whether those reasons make sense, the importance of allowing only the trier-of-fact (judge or jury) to combine information from other sources to reach ultimate conclusions, and finally, methods for cross-examining experts about bias.
After the lunch break, Workshop Chair Andrew Sulner, MSFS, JD presented his program, “Examining Sources of Bias and Illustrating Their Impact on Handwriting Opinions and Expert Testimony of Forensic Document Examiners.” As both an attorney and a 3rd generation forensic document examiner, Mr. Sulner used actual cases and published court decisions to illustrate how even experienced forensic document examiners can allow bias to improperly influence the outcome of their handwriting investigations and the manner in which their opinions are reported or testified to in court. Mr. Sulner also discussed various de-biasing techniques and examination procedures that can be used to enhance the accuracy of signature and handwriting investigations, one of which is the Fischhof Method of “upside-down” comparison that was developed by his grandfather, Professor Julius Fischhof, a pioneer in the field of questioned documents and Eastern Europe’s leading handwriting expert in the 19th century.
Barry C. Scheck, JD, co-Director of the Innocence Project, discussed “Minimizing Bias in Post- Conviction Inquiries Into Possible Miscarriages of Justice: Moving From an Adversarial to an Inquisitional Framework.” Professor Scheck spoke from his experience in working with Conviction Integrity Units in District Attorney’s offices across the country and addressed the problem of bias in “conviction integrity” re-investigations. The use of internal systemic safeguards in the medical industry and business community were discussed as models that could be adapted for use in the legal system to minimize the effect of bias in reinvestigations of potentially wrongful convictions.
Saul M. Kassin, PhD, who was scheduled to present “Forensic Confirmation Bias: How Confessions Corrupt Perceptions and Judgments,” was unable to attend due to an unexpected event. However, Dan Simon did an admirable job filling in, using Dr. Kassin’s slide show presentation to discuss the three classic types of false confessions that have been identified, and to illustrate why a forensic analyst’s awareness that there has been a confession can have a strong biasing influence upon the outcome of a subsequent forensic analysis by creating “corroboration inflation,” whereby the confession itself “produces an illusion of support from other evidence.” This presentation demonstrated why safeguards are needed to identify and prevent false confessions, and to prevent the “ripple effect” of knowledge of
confessions influencing other evidence, including forensic analysis.
In presenting “Forensic Expert Bias: A Judicial Perspective,” Donald E. Shelton, JD, PhD discussed expert bias in terms of the Federal Rules of Evidence and Daubert challenges to expert testimony, the use of “bias experts” to discredit expert testimony, and the use of jury instructions to address issues of potential bias on the part of expert witnesses. Judge Shelton also discussed and illustrated examples of outcome bias on the part of judges.
The final speaker of the day was Steven T. Wax, JD, the criminal defense attorney for Brandon Mayfield, the Oregon attorney who was wrongly accused of the 2004 terrorist train bombings in Madrid, Spain. His riveting description of the facts and circumstances of the Mayfield case, with chilling first hand accounts of the obstacles he faced defending an innocent client faced with what seemed to be incontrovertible proof of guilt, brought all the teachings of the day into sharp reality by clearly demonstrating most of the dangers of bias that had been discussed by the previous speakers.
This workshop made it abundantly clear that awareness of bias is not sufficient, and that preemptive action must be taken to minimize the potential for bias to contaminate forensic investigations and impact the outcome of forensic analyses. Each of the workshop presentations (see Program) was powerful and important, and the beauty of the day was the way each built on the previous presentation(s) to create a whole that was more than the sum of its parts. Attendees had enormous praise for the workshop and interesting questions for the Q&A session that lasted an extra hour at the end of a long day. Attendees took home with them a 600-page compendium of articles, presentations and source materials for future study on this important topic of Bias in Forensics.
This was the second workshop organized by Mr. Sulner and presented at an AAFS Annual Scientific Meeting. The first, entitled "Flawed Forensics: Recognizing and Challenging Misleading Forensic Evidence and Disingenuous Expert Testimony," was presented at the 2012 Atlanta meeting and was also widely acclaimed. For details, click here.