Novel Technology to Revolutionize the Process of Jury Selection and Voir Dire in Criminal Cases

I. Introduction

The Middle Ages in England marked the genesis of the jury system. A jury is a sworn body of people summoned to render an impartial verdict or to set a penalty or judgment in criminal cases. A trial jury consists of 6-12 jurors who are selected from the jury pool. Verdicts for cases are settled based on the agreement of three-fourths or five-sixths majority. Hence, juries can make or break a case proving them to be an indispensable tool to criminal cases however the tedious cross- questioning and examination of each potential jury member by the prosecutor and defense can be tedious. ³

The jury selection process begins with the court’s clerk calling 12 people in batches from a jury list of often hundreds of people. The judge gives the jury a preface of the case and inquires if there are any conflicts of interest or reasons a potential juror cannot serve. The prosecutor and defense then alternatively question each prospective juror on knowledge of the case or specific experiences that may cause them to unfair. This process of questioning is known as voir dire. During voir dire, lawyers scrutinize each potential juror to determine if a potential juror harbors any biases that would prevent them from being impartial. If during the voir dire process either lawyer believes that the juror could be prejudiced about a case, they can be dismissed for cause. Each lawyer can make peremptory challenges as well. Peremptory challenges give lawyers the authority to excuse a potential juror without stating a cause. ⁴

Peremptory challenges are often based on intuition and are therefore likely to be influenced by observable characteristics of potential jurors i.e. race, gender, clothing, wealth etc. The very nature of the peremptory challenge allows uncorroborated and unsubstantiated criteria to be used in excusing jurors. This often exacerbates the problem of racial, gender and class discrimination in legal proceedings. The use of peremptory challenges was unrestricted for 200 years until the Supreme Court ruled that the provision intended to facilitate the selection of impartial jurors could also violate other key elements of fairness, such as the Constitutional rights of every citizen to serve as a juror. Considering lawyers have been licensed to use any criteria they deem fit in issuing peremptory challenges and with their motivation to select juries favorable to their clients, it’s highly likely that lawyers will not fully conform with the supreme court’s restrictions against considering race or gender. However, to counter this very problem the U.S. Supreme Court created a process called a "Batson challenge," to determine whether the prosecution has removed a juror due to discriminatory bias. It involves the following steps: First, the defendant would have to present sufficient evidence to suggest that the prosecutor made the challenge because of discrimination. Followed by the prosecutor providing a legitimate, non-discriminatory explanation for the challenge. On the basis of both the cases presented, the judge decides whether the challenge was purposefully discriminatory. ⁵ But the Batson challenge opened a new portal of problems, if the defendant proves a Batson violation during the jury selection process it leads to the dismissal of the entire panel of potential jurors. In case a violation is filed in the middle of a trial, its declared a mistrial and select a new jury is selected through the same long and drawn- out process. If the Batson violation isn't proven until after the defendant has already been convicted, then a court of appeal will typically overturn the conviction and grant a new trial wasting the time, resources and efforts of all parties involved in the case.

Additionally, COVID-19 has also contributed to the problems of the jury selection process. For jury selection, trial courts deploy jurors in smaller groups across separate, larger rooms, and often confine them to the first floor of the courthouse as an attempt to maintain social distancing. Decisions made on the organization of prospective jurors and any information jurors receive from the court system about COVID-19 could influence jurors’ trial reactions; calming some and scaring others. ⁶ Many of them do not wish to participate in the voir dire process and develop resentment for the legal system. This has inflated the problem and distaste for jury selection among lawyers.

Bearing in mind all the shortcomings associated with the manual selection of jurors, the main goal of this paper is to create an automated jury selection system called the PereiraJuryNet which identifies key traits commonly considered during voir dire such as political preference and awareness, conflicts of interest, demographic traits, experience and knowledge of the population eligible for jury duty as per conditions set by different nations and cross- references them with the essential facts and nature of a case to select the best fit jurors. For the purposes of this study data of 800 potential jurors were collected and cross- referenced with 5 true and diverse cases that have taken place in UK courts.

Literature Overview

Theoretical and practical aspects of the jury trial outcomes have been studied by J. M. Abruzzo ⁷ who presents a machine learning approach to feature impact analysis in the context of American criminal jury trial. Additionally, M. I. Norton, S. R. Sommers and S. Brauner ⁸ have studied biases in the jury selection paradigm. However, there’s a virtual deficit of systematic research done in using machine learning for jury selection and voir dire. Little attention is paid to scientific research on the development of automated jury selection systems using machine learning specifically the Random Forest algorithm. The lack of a comprehensive study on using a machine learning model to select the best fit jurors for any case and the need to study this issue in light of growing inefficiencies in the manual selection of jurors served as a basis for researching and developing the pioneering PereiraJuryNet.

II. Theoretical background

A. Package requirements

The majority of the work was done using NumPy, Scikit-learn and Pandas library.

NumPy - NumPy is an open- source, fast computational array and matrices package which allows for high- level mathematics and computations on multi-dimensional array objects.

Pandas - Pandas is an open- source, highly dynamic package which complements python by allowing for the creation of table objects. The available data structures that come inbuilt with the Pandas library are series object, data frames and panel objects.

Scikit-learn- Scikit-learn is an open- source machine learning library that allows for supervised and unsupervised learning. Within it, comes various machine learning models called estimators which can be fitted to our NumPy array data and then can be utilized to predict values without the need for retraining every time.

B. Data Science Pipeline

Developing the problem statement - It is the preliminary task that incorporates the understanding of the problem at hand, the methodology expected to be applied and the results desired. Without thorough pacing of this step, the results obtained, despite their accuracy and value, could mean nothing to the result.

Data Collection - After the problem statement has been set in stone, the next step involves scouting and acquiring the perfect dataset to obtain the desired goals. The sources of the data could be open- source data lakes and data houses like Kaggle or manually generated datasets like the one used in this project

Data Wrangling and Preprocessing - Data wrangling, in a nutshell, is the process of converting data from one format to one which is ideal for analysis. This process is done by wrangling the data ie the set of processes which include identifying missing values, count the missing values in each column, using an optimum solution to deal with missing values, correcting data formats of all columns, data standardization, data normalization, label encoding, binning and even adding indicator variables if required. This process is also called data cleaning as it essentially is the cleaning of data for further analysis.

Model Selection and Building - Model Selection involves the essential step of selecting the right model for the task. The data is split into a training and testing set on which the selected model then undergoes training and testing. After the preliminary results are obtained, further hyperparameter tuning and optimization steps are incorporated and the steps of training and testing are done again. Based on the obtained results, an ideal model with the best results is selected and passed for deployment which involves testing and usage in the real world.

C. Machine learning pipeline

The proposed model used the following Machine Learning models:

1. Random Forest

2. SVM

3. Naive-Bayes

4. Adaboost

Random Forest - It essentially is a model built on the underlying idea of a decision tree which is a supervised learning model which can be used in regression and classification models. The decision tree works on the basis of selecting a feature as the most important and then generating results based on it. Random forest builds upon decision trees by randomly selecting features and creating multiple trees to generate outputs, which are then combined to provide a final output. It gets its name due to its usage of various trees.

SVM - SVM is a supervised learning model which ideally finds its usage with small datasets and classification problems. It works on the principle of generating a perfectly fit hyperplane in an N- dimensional feature space between clusters of similar data points. In this model, based on the output which is in the range of [-1,1], the classification is done.

Naive-Bayes - It is built upon Bayes theorem which is the probability of one output occurring given that another output has occurred. This works on the core principle that each feature of a dataset is classified exclusively, independent of the other feature sets and that all the features sets have the same weight on the outcome. This is more of a rigid algorithm and hence often times than not, does not produce the best results but does give us a strong understanding of the interdependence between the feature sets.

Adaboost - This classifier is more of a combination of weaker models. It acts as a boosting algorithm which essentially means that it takes weaker algorithms, tweaks them and combines them to generate a stronger algorithm. The weak algorithm usually used is decision trees with a single split, named stumps. It takes decision trees and adds higher weights on features that are hard to understand, and leaves the features which are easily understood. This algorithm finds use in both classification and regression tasks.

D. Confusion Matrix

Confusion Matrix - A Confusion Matrix is a testing metric used to determine the efficacy of a developed classification model to real- world testing. A typical 2x2 confusion matrix is generated when the classification model generates 2 labels. The table provides us with the True positives and negatives of the dataset and Predicted positives and negatives. This matrix in turn can help us determine other characteristics of the model, which include F1 Scores, Accuracy and Precision.

Research Design

There is an urgent need for a system of jury selection free from human discrimination and biases, a less monotonous system and a system free from the burden of following COVID-19 protocols. The pioneering automated jury selection model called PereiraJuryNet proposed in this paper serves as the primal solution to all of the problems associated with manual jury selection. It was developed following the tried-and-true data science pipeline of developing the problem, data collection (or acquisition), data wrangling and preprocessing, model selection and building.

A. Data set

The data set for this project was collected in successive steps. First, we consulted national courts’ websites (such as uscourts.gov and judiciary.uk) and researched legal websites (such as inbrief.co.uk and americanbar.org). The individual data on the eligibility and criteria of prospective jurors from a pool of registered voters were aggregated from these websites. The data set so produced of 800 jurors was then derived from the extrapolation of the aforementioned research. This data set involved the segregation of prospective jurors on the basis of features, namely- sex, age, race, occupation, education level, them being the victim of any crime, political preference, religion, petty crimes committed, social media posts on socio-economic and political topics such as homicide, micro and macroeconomic indices, terrorism, social justice movements, Islam, minority crimes and rights and liberal reform. This data was cross-referenced with the keywords, facts and nature of 5 cases that have been tried in UK supreme courts, namely- 1. A rape case: X (Appellant) v Kuoni Travel Ltd (Respondent) ⁹ (Keywords: Serial rape, assault, sexual assault, married woman, imposter and fraud); 2. A Minority hate crime case: Gang attack on Muslim students at City University [2009] ¹⁰ (Keywords: Gang attack, stabbing, violent intimidation, religious hate crime, verbal abuse and false terrorism accusations); 3. A Tax evasion case: Hayes (FC) (Respondent) v Willoughby (Appellant) [2013] UKSC 17 ¹¹ (Keywords: Fraud, tax evasion, embezzlement, financial crime and intrusion on privacy); 4. An Islam terrorism case: Her Majesty’s Treasury (Respondent) v Mohammed Jabar Ahmed and others (FC) (Appellants); Her Majesty’s Treasury (Respondent) v Mohammed al-Ghabra (FC) (Appellant); R (on the application of Hani El Sayed Sabaei Youssef) (Respondent) v Her Majesty’s Treasury (Appellant) [2010] UKSC 2 ¹² (Keywords: Islamic terrorism, international terrorism, weapon violence, mass shooting, extremism, national security threat and militant); and 5. A Homicide case: The towpath murders [1953] ¹³ (Keywords: Missing teens, raped, murdered, mutilated, homicide) to determine the best fit jurors for diverse cases. The data regarding the facts and nature of the cases were procured from the UK’s supreme court website (supremecourt.uk).

B. Data science and machine learning pipeline

After the generation of the custom dataset, the imported data went through the data wrangling pipeline which included identification and elimination of missing values and NaN values, data type conversions and label encoding which is essentially the conversion of categorical data points into its numerical counterparts which makes training easier. This resulted in a data frame in which each feature was numerical. The features, namely- Sex, Race, Occupation, Education Level, Victim of any crime, Political Preference and Religion were encoded to numerical values and the features S- Homicide, S- Macro/Microeconomics indices, S- Islam, S- Terrorism, S- Social justice movements, S- Minority crime and rights and S- Liberal reform were encoded to binary values of 0 and 1 (where S stands for social media posts on the specified topic). The dataset was then split into a 70% training and 30% testing set.

After deciding on the problem being of the classification type, based on past research, Random Forest, SVM, Naive-Bayes and Adaboost were considered based on their accuracies, ability to handle large feature sets and ability to strike to balance between bias and variance. Using the Scikit learn libraries, the following modules were imported, tuned and then trained on the dataset, tested and the accuracy of all the models was promising (Refer to Table 1).

C. Confusion Matrix

A confusion matrix for all the developed models was generated using the sklearn metrics library to understand how our model performed, by comparing the true outcomes for jury selection and the predicted jury selection outcome of our model. The X and Y axis of our 5x5 Confusion Matrix contains the following outcomes – Rape, Homicide, Islam Terrorism, Tax Evasion and Minority hate crime. See Matrix 1, Matrix 2, Matrix 3 and Matrix 4 for reference.

Matrix 1: Random Forest

Matrix 2: SVM

Matrix 3: Naive- Bayes

Matrix 4: Adaboost

III. Results

On training and testing of the various machine learning algorithms on the custom dataset, it was observed that Random Forest generated the best accuracy, with an accuracy score of 82.91% followed by SVM at 73.34%, Naive-Bayes at 71.25% and Adaboost at 70.41%. The best results obtained by Random Forest can be attested to the fact that they find their basis on the core concept of decision trees which makes understanding and scaling across various features

relatively easy and random feature selection ability which prevents it from being highly dependent on only a few features. Thus, the Random Forest algorithm assures an 82.91% guarantee of choosing the best fit juror from a data set of registered voters for jury selection. This translates to an accuracy of roughly 9 out of 12 jurors being the most appropriate for any given case.

IV. Conclusion and future work

The utilization of Random Forest to build this novel Jury selection model PereiraJuryNet, with its ability to remove the commonly associated biases and flaws with human decision making, we believe, can find its place in courtrooms all over the world. The accuracy of 82.91% on a novel model definitely opens the door for future improvements. Some of the suggested ways would include the trial of better machine learning models like XGBoost, appending the dataset with more data, better hyperparameter tuning, better optimization techniques and tweaking the bias-variance trade-off.

*****

Acknowledgments

We would like to praise and thank God, the almighty who has us granted countless blessings in the course of our lives. Further, we wish to thank Manipal Institute of Technology, Manipal and Our Own English High School, Sharjah for offering us knowledge, teaching us the significance of resilience and providing us with myriad opportunities. A very special word of thanks goes to our parents, Roosewalt and Janet, who have been constant pillars of strength.

*****

Footnotes

1. Author is a student at Manipal Institute of Technology, Manipal, India.

2. Author is a student at Our Own English High School, India.

3. How Courts Work, Americanbar.org, (September 09, 2019), https://www.americanbar.org/groups/public_educ ation/resources/law_related_education_network/how_courts_work/juryselect/ ↩

4. Jennifer K. Robbennolt and Matthew Taksin, Jury selection, peremptory challenges and discrimination, American Psychological Association, (January, 2009), https://www.apa.org/monitor/2009/01/jn ↩

5. Discrimination in Jury Selection, Nolo.com, https://www.nolo.com/legal-encyclopedia/discrimination-jury-selection.html ↩

6. Covid-19 fears and the juror experience during this time, Law.com, (February 16, 2021), https://www.law. com/thelegalintelligencer/2021/02/16/covid-19-fears-and-the-juror-experience-during-this-time/ ↩

7. Joseph M. Abruzzo, Sensitivity of Jury Trial Outcomes to Trial Factor, Stanford.edu, http://cs229.stanford. edu/proj2016/report/AbruzzoSensitivityOfJuryTrialOutcomesToTrialFactors-report.pdf ↩

8. M. I. Norton, S. R. Sommers et. al., Bias in jury selection: justifying prohibited peremptory challenges, Onlinelibrary. wiley, (December 07, 2007), https://onlinelibrary.wiley.com/doi/10.1002/bdm.571 ↩

9. X (Appellant) v Kuoni Travel Ltd (Respondent), Supremecourt.uk, https://www.supremecourt.uk/cases/uksc-2018-0102.html ↩

10. Police investigate gang attack on Muslim students in London, The Guardian, (November 09, 2009), https://www.theguardian.com/uk/2009/nov/09/racist-attacks-students-city-university ↩

11. Hayes (FC) (Respondent) v Willoughby (Appellant), Supremecourt.uk, https://www.supremecourt.uk/cases/docs/uksc-2012-0010-judgment.pdf ↩

12. Her Majesty’s Treasury (Respondent) v Mohammed Jabar Ahmed and others (FC) (Appellants); Her Majesty’s Treasury (Respondent) v Mohammed al-Ghabra (FC) (Appellant); R (on the application of Hani El Sayed Sabaei Youssef) (Respondent) v Her Majesty’s Treasury (Appellant) [2010] UKSC 2, Supremecourt.uk, https://www.supremecourt.uk/cases/docs/uksc-2009-0015-press-summary-02.pdf ↩

13. Murder most foul, The Law Society Gazette, (June 03, 2005), https://www.lawgazette.co.uk/analysis/murder-most-foul/1775.article ↩