February 2017 | Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, Sendhil Mullainathan
This paper examines how machine learning can improve and understand human decision-making, focusing on a critical judicial decision: determining whether defendants should be released or detained before trial. Judges are required to base their decisions on a prediction of whether a defendant would commit a new crime if released. The authors use a machine learning algorithm trained on defendant characteristics to predict crime risk, analyzing data from 758,027 defendants arrested in New York City between 2008 and 2013. The algorithm's predictions are compared to judges' decisions, revealing that judges often release defendants deemed high-risk by the algorithm. This suggests that judges may be mis-ranking defendants, leading to suboptimal outcomes.
The authors use various econometric strategies, including quasi-random assignment of cases to judges, to evaluate the algorithm's performance. Their results show that using the algorithm could reduce crime by up to 24.8% without changing jailing rates, or reduce jail populations by 42.0% without increasing crime rates. The algorithm also reduces all categories of crime, including violent crimes, and significantly reduces the percentage of African-Americans and Hispanics in jail.
The study also finds that judges may be mis-predicting risk, responding to 'noise' as if it were a signal. This suggests that integrating machine learning into an economic framework is essential for realizing its value. The authors also find similar results in a national dataset, indicating that the findings are not unique to New York. The paper highlights the importance of considering both the accuracy of predictions and the broader context of decision-making, including the potential for racial equity and the complexity of different types of crimes.
The authors conclude that machine learning can be a valuable tool for improving human decisions, but its effectiveness depends on integrating it into an economic framework that considers the link between predictions and decisions, the scope of payoff functions, and the construction of unbiased decision counterfactuals. The study provides insights into the challenges of evaluating machine learning predictions in the context of human decisions and the potential for significant welfare gains through the use of algorithmic predictions.This paper examines how machine learning can improve and understand human decision-making, focusing on a critical judicial decision: determining whether defendants should be released or detained before trial. Judges are required to base their decisions on a prediction of whether a defendant would commit a new crime if released. The authors use a machine learning algorithm trained on defendant characteristics to predict crime risk, analyzing data from 758,027 defendants arrested in New York City between 2008 and 2013. The algorithm's predictions are compared to judges' decisions, revealing that judges often release defendants deemed high-risk by the algorithm. This suggests that judges may be mis-ranking defendants, leading to suboptimal outcomes.
The authors use various econometric strategies, including quasi-random assignment of cases to judges, to evaluate the algorithm's performance. Their results show that using the algorithm could reduce crime by up to 24.8% without changing jailing rates, or reduce jail populations by 42.0% without increasing crime rates. The algorithm also reduces all categories of crime, including violent crimes, and significantly reduces the percentage of African-Americans and Hispanics in jail.
The study also finds that judges may be mis-predicting risk, responding to 'noise' as if it were a signal. This suggests that integrating machine learning into an economic framework is essential for realizing its value. The authors also find similar results in a national dataset, indicating that the findings are not unique to New York. The paper highlights the importance of considering both the accuracy of predictions and the broader context of decision-making, including the potential for racial equity and the complexity of different types of crimes.
The authors conclude that machine learning can be a valuable tool for improving human decisions, but its effectiveness depends on integrating it into an economic framework that considers the link between predictions and decisions, the scope of payoff functions, and the construction of unbiased decision counterfactuals. The study provides insights into the challenges of evaluating machine learning predictions in the context of human decisions and the potential for significant welfare gains through the use of algorithmic predictions.