« Lots and lots more great new content at great new Inquest website | Main | "The Trump Executions" »

July 30, 2021

Provocative new empirical paper claims to identify the "most discriminatory" federal sentencing judges

Via Twitter, I was altered to this notable new empirical paper by multiple authors titled "The Most Discriminatory Federal Judges Give Black and Hispanic Defendants At Least Double the Sentences of White Defendants."  The paper's title alone likely explains why I describe it as provocative, and this abstract provides more context about the paper's contents:

In the aggregate, racial inequality in criminal sentencing is an empirically well- established social problem. Yet, data limitations have made it impossible for researchers to systematically determine and name the most racially discriminatory federal judges.  The authors use a new, large-scale database to determine and name the observed federal judges who impose the harshest sentence length penalties on Black and Hispanic defendants.  Following the focal concerns framework, the authors (1) replicate previous findings that conditional racial disparities in sentence lengths are large in the aggregate, (2) show that judges vary considerably in their estimated degrees of racial discrimination, and (3) list the federal judges who exhibit the clearest evidence of racial discrimination.  This list shows that several judges give Black and Hispanic defendants double the sentences they give observationally equivalent white defendants.  Accordingly, the results suggest that holding the very most discriminatory judges accountable would yield meaningful improvements in racial equality.

The "new, large-scale database" used for this study is this JUSTFAIR data source published online last year.  I have not previously blogged about the JUSTFAIR data because I have never been sure of its representativeness since the source says it includes nearly 600,000 cases over a recent 18-year period (from 2001 to 2018), but more than twice that number of persons have been sentenced in federal courts during that span.  I fear I lack the empirical chops to know just whether to be reasonably confident or highly uncertain about the JUSTFAIR data, and that broader concern colors my thinking about this provocative new paper's claims that the authors have been able to identify the "most racially discriminatory federal judges."

I would love to hear from readers with strong empirical backgrounds about whether this new paper effectively demonstrates what it claims to identify.  I am initially skeptical because the the two judges labelled "most discriminatory" come from the same federal judicial district and only a few districts are among those that have all the identified "discriminatory" judges.  That reality leads me to wonder if case-selection realities, rather than "discrimination," may at least in part account for any observed racial differences in sentencing outcomes.  Relatedly, when I dig into the local data at the JUSTFAIR data site, the judges identified in this new paper as the "most discriminatory" do not seem to have anywhere close to the most racially disparate rates of above/below guideline sentencing outcomes even within their own districts.

In short, without a much better understanding of the empirics at work here, I am not confident about what this new empirical paper is claiming.  But I am confident that I would like to hear from readers as to what they think about this provocative new paper on an always important topic.

July 30, 2021 at 01:48 PM | Permalink


Professor Berman,

Thanks so much for featuring our research; we certainly appreciate any comments your readers have for us and would welcome feedback!

To highlight and respond to a few points you make:

-With regard to case-selection, we believe that we've controlled for a large deal of that by including the guideline minimum (accounting for enhancements/reductions and any statutory minimum that trumps the guideline minimum) and the offense type in our model; that is, the model is comparing theoretical defendants who are relatively equally situated, legally, no matter what the judge or district's overall caseload looks like. We've also controlled for whether a defendant pleaded guilty or was sentenced after trial, and whether they received any government-sponsored departure — we believe these controls minimize the influence of the decisions made by prosecutors on our judge results.

-The missingness of our data is certainly an important limitation. There's a data paper here (https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0241381) that gives more info on how the data matching pipeline works. In essence, our pipeline could be missing cases due to a few reasons, including: a.) There are instances where we cannot uniquely match cases across the two main federal sentencing databases; this could be due to defendants in the same district who have the same sentence, same offense type, in the same month and year. b.) The docket number does not contain judge initials, contains only one letter, or contains initials that do not correspond to any judge in that district. c.) It is impossible to distinguish between judges who have both the same initials and district. We remove these cases to avoid any incorrect matchings. We don't have any reasons to believe these causes are correlated with defendant characteristics — for example, a defendant's race shouldn't be associated with whether the clerk added judge initials in the docket. (We've also excluded from our analysis immigration-only cases, where a large number of defendants might all receive identical fast track sentences in one district all at once, since these aren't representative of usual sentencing practices. In that instance, the missing data due to duplication of defendants could be related to defendant characteristics.) So, we haven't identified a reason this missing data should affect results for judges for whom we have sufficient data, though we can't exclude that possibility.

-Nonetheless, the missing data means that some judges and districts are missing entirely — that is, we're both "reasonably confident" and "highly uncertain," as you write, about different portions of our results. For judges and districts with adequate data, we would be surprised if the listed judges weren’t engaging in some degree of discrimination, but we definitely acknowledge that there are large gaps of uncertainty where we simply don’t have data.

In any case, we welcome any other thoughts you or your readers have; the paper on SocArXiv has also been updated to reflect answers to some of the questions and feedback we're getting.

Posted by: Nicholas Goldrosen | Aug 3, 2021 11:57:56 AM

Thanks for this substantive explanation, Nicholas, but how can you tell if defendants are "relatively equally situated" if plea negotiations have impacted the guideline calculation --- which they typically do, especially in (c)(1)(C) cases, but also in nearly all others? What if EDPA prosecutors have often given sweet guideline deals to minority defendants, but not to white defendants, so that judges there are much more likely follow the guidelines for only minority defendants? The fact that the top 2 "most discriminatory" judges are senior judges from one district (out of 94 districts) and that a few other districts are heavily over-represented among the "most discriminatory" suggests to me that you may be measuring a case-processing story different than "discriminatory" judges.

Relatedly, I am especially curious and quite unclear as why the judges identified in this new paper as the "most discriminatory" do not seem to have anywhere close to the most racially disparate rates of above/below guideline sentencing outcomes even within their own districts according to the JUSTFAIR data site. I appreciate efforts to match up similar cases while looking for racially disparate sentencing patterns in how seemingly similar cases are being handled. But to label certain judges "most discriminatory" on one set of metrics (this new paper) while then having them not appear to be nearly as racially disparate in above/below guideline sentencing in another metric (the JUSTFAIR data site) leaves me confused as to what part of your own data-run to trust.

Most fundamentally, I am concerned about calling out a group of judges as "most discriminatory" without a much clearer accounting of the cases and context for their sentencing decision-making. Did these judges have a lot of drug cases? Child porn cases? Firearm cases? Fraud cases? Cases that went to trial? Cases that included lots of mandatory minimums? Saying you have "controlled for a large deal of that" without detailing just how, especially on a potent topic like race, just makes me eager to more fully understand how your controls work. Perhaps real empiricists can see you have taken care here, but I am struggling to make my way through the jargon in order to feel confident in your conclusions.

Posted by: Doug Berman | Aug 3, 2021 3:17:58 PM

The Federal Community Defenders Office in the EDPA has now released a statement in response:


So has the U.S. Attorney's Office:


Posted by: Lisa Mathewson | Aug 7, 2021 9:22:11 AM

Post a comment

In the body of your email, please indicate if you are a professor, student, prosecutor, defense attorney, etc. so I can gain a sense of who is reading my blog. Thank you, DAB