We provide expert statistical services to attorneys, including both defense attorneys and prosecutors. We contract with bar advocates and public defenders at a rate approved by Massachusetts' Committee on Public Counsel Services (CPCS). Many of these cases have involved the question of police bias, specifically racial profiling.
Poor-Quality Official Data Impedes Justice in Racial-Bias Cases
A version of this post was published as a letter to the editor in Massachusetts Lawyers Weekly,
in print Dec. 5, 2022 and online Dec. 2. For subscribers,
https://masslawyersweekly.com/2022/12/02/letter-poor-quality-official-data-impedes-justice-in-racial-bias-cases/
***
Supreme Judicial Court decisions in Comm. v. Lora, 451 Mass. 425 (2008), and Comm. v. Long, 485 Mass. 711 (2020) may advance racial-justice efforts in Massachusetts. Both affirm that, to demonstrate potential bias in police officers’ treatment of criminal defendants, the accused can present statistical analysis of patterns of such treatment. However, the defense must rely on the state to compile accurate data for analysis.
State-provided data often turns out to be problematic.
As a statistician, I have repeatedly encountered serious deficiencies in traffic-stop data supplied by the Massachusetts State Police (MSP) and Department of Transportation (DOT). Some illustrative examples follow.
1. Wholesale inaccuracies in locations. One recent MSP data file covered 2,500 stops made by troopers assigned to Southeastern MA. Just 3% of the locations listed were in the Southeast. In another file containing wholesale errors, supposedly Truro was on Route 495, and Acushnet was a town in Texas. Last winter the MSP acknowledged “some errors” in their locations; I have seen no improvements since.
2. Missing data. For one set of officers, 40% of 1,000 citation decisions were blank. Those savvy about research know that simply using the data available risks introducing bias into the sample. Missing data also reduce sample size, often weakening analysis.
3. Implausible data.
Citations for certain offenses are omitted from the records. One data file describing 37,000 offenses included zero involving weapons -- even though corresponding police reports showed numerous weapons charges.
For one file, among 1,000 seatbelt citations 47% were the only citations listed for those drivers. Some seatbelt violations supposedly prompted a criminal charge or arrest. None of this is credible considering the secondary seatbelt-enforcement law.
4. Persistent unresponsiveness by the DOT and MSP to information requests. I have made repeated requests that the DOT explain unclear data elements. E.g., does “Search” apply to the vehicle only, or to driver or passenger? Besides “Y” and “N”, how to interpret blanks? No answers to these or any such questions have been received in ten months. The MSP website invites data-related questions via email or web form. Submissions via both have produced no response in four months.
These defects are sure to interfere with prosecutors’ and police departments’ ability to conduct their own analyses, or to demonstrate equitable treatment of different groups. For the accused, far more than an inconvenience, this failure to generate accurate, transparent, usable data denies an important avenue of defense and thus a Constitutional right.
One cannot meaningfully test for over-representation of protected groups in traffic stops without being able to identify the area and thus the population patrolled.
Similarly, one cannot test for disparate treatment of those stopped without valid data on citations issued and for what offenses.
These deficiencies pose barriers to the kinds of defense not only permitted but encouraged by the Court in their Lora and Long decisions.
A version of this post was published as a letter to the editor in Massachusetts Lawyers Weekly,
in print Dec. 5, 2022 and online Dec. 2. For subscribers,
https://masslawyersweekly.com/2022/12/02/letter-poor-quality-official-data-impedes-justice-in-racial-bias-cases/
***
Supreme Judicial Court decisions in Comm. v. Lora, 451 Mass. 425 (2008), and Comm. v. Long, 485 Mass. 711 (2020) may advance racial-justice efforts in Massachusetts. Both affirm that, to demonstrate potential bias in police officers’ treatment of criminal defendants, the accused can present statistical analysis of patterns of such treatment. However, the defense must rely on the state to compile accurate data for analysis.
State-provided data often turns out to be problematic.
As a statistician, I have repeatedly encountered serious deficiencies in traffic-stop data supplied by the Massachusetts State Police (MSP) and Department of Transportation (DOT). Some illustrative examples follow.
1. Wholesale inaccuracies in locations. One recent MSP data file covered 2,500 stops made by troopers assigned to Southeastern MA. Just 3% of the locations listed were in the Southeast. In another file containing wholesale errors, supposedly Truro was on Route 495, and Acushnet was a town in Texas. Last winter the MSP acknowledged “some errors” in their locations; I have seen no improvements since.
2. Missing data. For one set of officers, 40% of 1,000 citation decisions were blank. Those savvy about research know that simply using the data available risks introducing bias into the sample. Missing data also reduce sample size, often weakening analysis.
3. Implausible data.
Citations for certain offenses are omitted from the records. One data file describing 37,000 offenses included zero involving weapons -- even though corresponding police reports showed numerous weapons charges.
For one file, among 1,000 seatbelt citations 47% were the only citations listed for those drivers. Some seatbelt violations supposedly prompted a criminal charge or arrest. None of this is credible considering the secondary seatbelt-enforcement law.
4. Persistent unresponsiveness by the DOT and MSP to information requests. I have made repeated requests that the DOT explain unclear data elements. E.g., does “Search” apply to the vehicle only, or to driver or passenger? Besides “Y” and “N”, how to interpret blanks? No answers to these or any such questions have been received in ten months. The MSP website invites data-related questions via email or web form. Submissions via both have produced no response in four months.
These defects are sure to interfere with prosecutors’ and police departments’ ability to conduct their own analyses, or to demonstrate equitable treatment of different groups. For the accused, far more than an inconvenience, this failure to generate accurate, transparent, usable data denies an important avenue of defense and thus a Constitutional right.
One cannot meaningfully test for over-representation of protected groups in traffic stops without being able to identify the area and thus the population patrolled.
Similarly, one cannot test for disparate treatment of those stopped without valid data on citations issued and for what offenses.
These deficiencies pose barriers to the kinds of defense not only permitted but encouraged by the Court in their Lora and Long decisions.
Critique of Report on Auto Stops and Race in MA
This short .pdf piece explains why statistical errors undermine confidence in the report published by Salem State University and Worcester State University, February 7, 2022. Their anonymous report presumes to show whether racial bias occurs, either in police decisions about which motorists to stop or in which types of citations to issue to the drivers stopped. The report was disseminated by the MA Executive Office of Public Safety and Security (EOPSS), apparently without any vetting as to the soundness of the methods used.
A condensed version of this critique was published as a letter to the editor of Mass. Lawyers Weekly, April 8, 2022. For subscribers, https://masslawyersweekly.com/2022/04/08/letter-statistician-points-to-shortcomings-in-traffic-stop-report/.
This short .pdf piece explains why statistical errors undermine confidence in the report published by Salem State University and Worcester State University, February 7, 2022. Their anonymous report presumes to show whether racial bias occurs, either in police decisions about which motorists to stop or in which types of citations to issue to the drivers stopped. The report was disseminated by the MA Executive Office of Public Safety and Security (EOPSS), apparently without any vetting as to the soundness of the methods used.
A condensed version of this critique was published as a letter to the editor of Mass. Lawyers Weekly, April 8, 2022. For subscribers, https://masslawyersweekly.com/2022/04/08/letter-statistician-points-to-shortcomings-in-traffic-stop-report/.
Analyzing for Bias in Traffic-Stop Cases: Video Presentations
Each of the videos below describes an aspect of analyzing for bias or racial profiling in police traffic stops. These presentations consider issues arising from the decision in Commonwealth v. Long (485 Mass. 711, 2020). (Last updated May 9, 2022; found still applicable Nov. 7, 2022.)
Each video is 3-7 minutes long and assumes no prior statistical background.
Session 1. How might we assess, statistically, whether a pattern of law enforcement is biased?
Session 2. What to expect from official MA data on police stops
Session 3. Statistical Significance: Concepts
Session 4. Statistical Significance: Choosing a Threshold
Session 5. Controlling for Many Factors