Excerpt

Predictive judicial analytics holds the promise of increasing the fairness of law. Much empirical work observes inconsistencies in judicial behavior. By predicting judicial decisions—with more or less accuracy depending on judicial attributes or case characteristics—machine learning offers an approach to detect when judges are most likely to allow extralegal biases to influence their decision-making. In particular, low predictive accuracy may identify cases of judicial “indifference,” where case characteristics (interacting with judicial attributes) do not strongly dispose a judge in favor of one or another outcome. In such cases, biases may hold greater sway, implicating the fairness of the legal system.

Arbitrary Decision-Making

There is ample social scientific evidence documenting arbitrariness, unfairness, and discrimination in the US legal system. To give just a flavor of the relevant research:

• United States federal appeals court judges become more politicized before elections and more unified during war (Berdejo and Chen 2016; Chen 2016b).

• Refugee asylum judges are two percentage points more likely to deny asylum to refugees if their previous decision granted asylum (Chen, Moskowitz, and Shue 2016).

• Politics and race also appear to influence judicial outcomes (Schanzenbach 2005; Bushway and Piehl 2001; Mustard 2001; Steffensmeier and Demuth 2000; Albonetti 1997; Thomson and Zingraff 1981; Abrams, Bertrand, and Mullainathan 2012; Boyd, Epstein, and Martin 2010; Shayo and Zussman 2011), as does masculinity (Chen, Halberstam, and Yu 2016b, 2016a), birthdays (Chen and Philippe 2018), football game outcomes (Chen 2017; Eren and Mocan 2016), time of day (Chen and Eagel 2016; Danziger, Levav, and Avnaim-Pesso 2011b), weather (Barry et al. 2016), name (Chen 2016a), and shared biographies (Chen et al., 2019 (forthcoming)) or dialects (Chen and Yu 2016).

• There are also various papers showing clear judicial biases in laboratory environments, such as the influence of anchoring, framing, hindsight bias, egocentric bias, snap judgments, representative heuristics, and inattention (Guthrie, Rachlinski, and Wistrich 2000, 2007; Rachlinski et al. 2009; Rachlinski, Wistrich, and Guthrie 2013; Simon 2012).

Thus, the primary question is not whether these problematic features of the legal system exist. Rather, the dilemma facing policymakers is what, if anything, can be done. This chapter will argue that predictive judicial analytics in the form of applied statistical/machine learning (from causal inference to deep learning) holds at least some promise on this front.

Bibliography

Abrams, D., M. Bertrand, and S. Mullainathan. 2012. “Do Judges Vary in Their Treatment of Race?” Journal of Legal Studies 41 (2): 347–383.

Albonetti, C. A. 1997. “Sentencing Under the Federal Sentencing Guidelines: Effects of Defendant Characteristics, Guilty Pleas, and Departures on Sentence Outcomes for Drug Offenses, 1991-1992.” Law & Society Review 31 (4): 789–822.

Amaranto, D., E. Ash, D. L. Chen, L. Ren, and C. Roper. 2017. “Algorithms as Prosecutors: Lowering Rearrest Rates Without Disparate Impacts and Identifying Defendant Characteristics ‘Noisy’ to Human Decision-Makers.” Working paper, Social Science Research Network (SSRN). https://ssrn.com/abstract=2993003.

Ash, E., and D. L. Chen. 2017. “Predicting Punitiveness from Judicial Corpora.” Working paper. http://users.nber.org/~dlchen/papers/Predicting_Punitiveness_and_Sentencing_Disparities_from_Judicial_Corpora.pdf.

Ash, E., D. L. Chen, and S. Naidu. 2017. “Ideas have Consequences: The Impact of Law and Economics on American Justice.” Working paper. https://users.nber.org/~dlchen/papers/Ideas_Have_Consequences.pdf.

Barry, N., L. Buchanan, E. Bakhturina, and D. L. Chen. 2016. “Events Unrelated to Crime Predict Criminal Sentence Length.” Working paper. https://users.nber.org/~dlchen/papers/Events_Unrelated_to_Crime_Predict_Criminal_Sentence_Length.pdf.

Berdejo, C., and D. L. Chen. 2016. “Electoral Cycles Among US Courts of Appeals Judges.” Journal of Law and Economics 60 (3): 479–496.

Boyd, C. L., L. Epstein, and A. D. Martin. 2010. “Untangling the Causal Effects of Sex on Judging.” American Journal of Political Science 54 (2): 389–411.

Bushway, S. D., and A. M. Piehl. 2001. “Judging Judicial Discretion: Legal Factors and Racial Discrimination in Sentencing.” Law & Society Review 35 (4): 733–764.

Chen, D. L. 2016a. “Implicit Egoism in Courtrooms: First Initial Name Effects with Randomly Assigned Defendants.” Working paper.

———. 2016b. “Priming Ideology: Why Presidential Elections Affect US Judges.” Working paper, SSRN (Social Science Research Network). https://ssrn.com/abstract=2816245.

Chen, D. L. 2017. “Mood and the Malleability of Moral Reasoning.” TSE Working Paper n. 16-707, Institute for Advanced Study in Toulouse, France. https://www.tse-fr.eu/publications/mood-and-malleability-moral-reasoning.

Chen, D. L., X. Cui, L. Shang, and J. Zhang. 2019 (forthcoming). “What Matters: Agreement Between US Courts of Appeals Judges.” Journal of Machine Learning Research.

Chen, D. L., M. Dunn, L. Sagun, and H. Sirin. 2017. “Early Predictability of Asylum Court Decisions.” In Proceedings of the 16th Edition of the International Conference on Artificial Intelligence and Law, 233–236. New York, NY: Association for Computing Machinery.

Chen, D. L., and J. Eagel. 2016. “Can Machine Learning Help Predict the Outcome of Asylum Adjudications?” In Proceedings of the 16th Edition of the International Conference on Artificial Intelligence and Law, 237–240. New York, NY: Association for Computing Machinery.

Chen, D. L., Y. Halberstam, and A. C. L. Yu. 2016a. “Covering: Mutable Characteristics and Perceptions of (Masculine) Voice in the US Supreme Court.” TSE Working Paper 16-680, Toulouse School of Economics, Toulouse, France. https://www.tse-fr.eu/publications/covering-mutable-characteristics-andperceptions-voice-us-supreme-court.

———. 2016b. “Perceived Masculinity Predicts United States Supreme Court Outcomes.” PLoS ONE 11 (10): e0164324.

Chen, D. L., T. J. Moskowitz, and K. Shue. 2016. “Decision-Making Under the Gambler’s Fallacy: Evidence from Asylum Judges, Loan Officers, and Baseball Umpires.” Quarterly Journal of Economics 131 (3): 1181–1242.

Chen, D. L., and A. Philippe. 2018. “Clash of Norms: Judicial Leniency on Defendant Birthdays.” TSE Working Paper n. 18-76, Institute for Advanced Study in Toulouse, France. http://www.iast.fr/publications/clash-norms-judicial-leniency-defendant-birthdays.

Chen, D. L., and A. Yu. 2016. “Mimicry: Phonetic Accommodation Predicts US Supreme Court Votes.” Working paper. http://users.nber.org/~dlchen/papers/Mimicry.pdf.

Danziger, S., J. Levav, and L. Avnaim-Pesso. 2011. “Reply to Weinshall-Margel and Shapard: Extraneous Factors in Judicial Decisions Persist.” Proceedings of the National Academy of Sciences 108 (42): E834.

Eren, O., and N. Mocan. 2016. “Emotional Judges and Unlucky Juveniles.” American Economic Journal: Applied Economics 10 (3): 171–205.

Guthrie, C., J. J. Rachlinski, and A. J. Wistrich. 2000. “Inside the Judicial Mind.” Cornell Law Review 86 (4): 777–830.

———. 2007. “Blinking on the Bench: How Judges Decide Cases.” Cornell Law Review 93 (1): 1–44.

Mustard, D. B. 2001. “Racial, Ethnic, and Gender Disparities in Sentencing: Evidence from the US Federal Courts.” Journal of Law and Economics 44 (1): 285–314.

Pope, D. G., J. Price, and J. Wolfers. 2013. “Awareness Reduces Racial Bias.” NBER Working Paper No. 19765, National Bureau of Economic Research, Cambridge, MA. https://www.nber.org/papers/w19765.

Rachlinski, J. J., S. L. Johnson, A. J. Wistrich, and C. Guthrie. 2009. “Does Unconscious Racial Bias Affect Trial Judges?” Notre Dame Law Review 84 (3): 1195–1246.

Rachlinski, J. J., A. J. Wistrich, and C. Guthrie. 2013. “Altering Attention in Adjudication.” UCLA Law Review 60:1586–1618.

Schanzenbach, M. 2005. “Racial and Sex Disparities in Prison Sentences: The Effect of District-Level Judicial Demographics.” Journal of Legal Studies 34 (1): 57–92.

Shayo, M., and A. Zussman. 2011. “Judicial Ingroup Bias in the Shadow of Terrorism.” Quarterly Journal of Economics 126 (3): 1447–1484.

Simon, D. 2012. In Doubt: The Psychology of the Criminal Justice Process. Cambridge, MA: Harvard University Press.

Steffensmeier, D., and S. Demuth. 2000. “Ethnicity and Sentencing Outcomes in US Federal Courts: Who is Punished More Harshly?” American Sociological Review 65 (5): 705–729.

Thomson, R. J., and M. T. Zingraff. 1981. “Detecting Sentencing Disparity: Some Problems and Evidence.” American Journal of Sociology 86 (4): 869–880.

BACK TO Law as Data