Prevention of miscarriages of justice using machine learning

We are a small independent family team who share a deep frustration with injustices in criminal cases that result in innocent people going to jail. Our backgrounds are in university and industry-based scientific research including medicine, mathematics and business. We are seeking ways to mathematically determine guilt based on evidence and so prevent people from being wrongfully convicted for crimes they did not commit, and you may be able to help.

So far we have found that machine learning may be able to assist in reducing the number of such miscarriages of justice (https://academic.oup.com/lpr/article/19/1/43/5807887). We now need to validate our findings and we are appealing for volunteers to help with that work.

Volunteers are needed to review on-line recordings of real trials and assess the evidence as if they were a member of the jury. The first trial we have chosen is Minnesota vs Derek Chauvin in the US. We chose this trial as it is a recent trial and it was televised live from start to verdict and on-line recordings are widely available on YouTube and elsewhere.

​ What’s required

The Derek Chauvin trial provides a rare opportunity to assess the prosecution and defence evidence in a high profile murder trial and to see the extent to which individual interpretations of the evidence impact the results from our machine learning model. For those of you already familiar with the Derek Chauvin trial the work should take very little time (a few hours). Volunteers not yet familiar with the trial will need to spend more time as they familiarise themselves with the on-line coverage. The work does not need to be completed in one go, but can be spread over as many days as required.

We would like you to do the following:

1. If you did not follow the trial in real time, then please find and view recordings of the actual trial and closing arguments. So you are sure you’re watching the trial first-hand, please use courtroom videos from a reliable source such as Court TV (MN v. CHAUVIN (2021) https://www.courttv.com/trials/mn-v-chauvin-2021/) or Washington Post. See ‘Links to trial’ below.

2. After or while viewing the trial, identify what you consider to be strands of evidence presented by the prosecution and defence – you can choose as many strands as you think appropriate. You will be asked to describe your evidence strands in a few words to give the substance of the evidence. There are no right or wrong ways of identifying evidence strands – it is simply your interpretation we are after, much like a jury member.

3. We would now like you to enter your strands of prosecution and defence evidence into our on-line model, where you will be asked a series of questions about your evidence strands. The following instructions are for using the on-line model, but if you prefer to work with an off-line WORD document, please contact us at enquiries@justicebrd.com and we can send you a WORD template and instructions.

4.  If you have not yet registered on our web site please do so, using your email address, here: https://www.justicebrd.com/showSignUp. These details will not be visible to others and you will not be identified in any publication, but referred to as a number (volunteer no. 1, say). Once registered you will be able to view the cases we used to train and test the model, and download the case narratives to get a feel for the level of detail needed to capture trial evidence.

5. Once registered you can sign in to the web site and create your Derek Chauvin case – from the top menu select Cases > Create new case, read the instructions at the top, enter a name for your case, and click ‘Save new case’.

6. You will see a list of all the cases used to train and test our machine learning model. Find your new case in the list and click the ‘Evidence’ icon (magnifying glass) in the rightmost column to start entering your evidence.

7. First, enter your prosecution evidence, strand by strand. For each strand of your prosecution evidence click the ‘Create new strand’ button at the top right, enter a short description for the strand, make sure ‘Prosecution’ is selected, and answer the questions about quality and relevance, giving a brief justification for your answers. Click ‘Save new strand’ at the bottom left to save each strand. If you make a mistake, or want to change anything, don’t worry because you can go back and make any necessary changes at any time.

8. Second, enter your defence evidence, strand by strand. For each strand of your defence evidence click the ‘Create new strand’ button at the top right, enter a short description for the strand, make sure ‘Defence’ is selected,  answer the questions about quality and give a brief justification for your answer.

9. After you answer the quality questions for a defence strand, you will be asked to identify which strand(s), if any, of prosecution evidence you consider it challenges, and assess the strength of that challenge. You will be presented with a list of all the prosecution strands you have created. For each one select how strongly the defence strand challenges this prosecution strand, leaving the drop down as ‘0 - none‘ if there is no challenge. You will be asked to give a brief justification for your assessment of challenge strength. A defence strand may challenge none, some, or all of the prosecution strands, so make sure you consider each prosecution strand in turn. Click ‘Save new strand’ at the bottom left to save each defence strand.

10. You will choose from a scale (drop-down list) when you assess a strand of prosecution evidence for relevance (from not relevant to highly relevant) and a strand of defence evidence for strength (from none to great strength) with which it challenges the prosecution evidence. Your choice is best done by following your instincts and not thinking in terms of a right or wrong absolute choice. Volunteers could easily differ in their choice on the relevance and strength scale for the same strand of evidence and that would not be a problem for the model as it works in terms of the relative relationship between the prosecution and defence strands. 

11. Don’t worry if you need to break off at any point, just make sure you have saved the strand you are working on and you will be able to pick up where you left off next time you sign in.

If you need help at any time, please just contact us at enquiries@justicebrd.com.

​ The results

As soon as you have finished entering all your evidence strands you will be able to see the guilt classification for your case as determined by the machine learning algorithm.

Go to Cases > Show cases from the top menu, find your case in the list, and the Model Output column will show the guilt classification (Guilty or Not guilty) and the Probability of Guilt column will show the probability with which the model has classified the outcome.

You can see this graphically by going to Cases > View results. To download a summary report of your case (in Word format) go to Cases > Show cases, find your case in the list, click the ‘Evidence’ icon on the right, and then click the Print Case button from the top right of your case summary screen.

​ Next steps and contact

Please let us know at enquiries@justicebrd.com when you have finished entering your case, and please don’t hesitate to get in touch with any questions or if we can help at any stage.

LINKS TO TRIAL VIDEO

COURT TV (MN v. CHAUVIN (2021)
https://www.courttv.com/trials/mn-v-chauvin-2021/

WASHINGTON POST
https://www.youtube.com/watch?v=IWbA2brcEbg
Derek Chauvin trial begins with opening statements and first witnesses - 3/29 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=sZ92-geeYBo
Derek Chauvin trial continues with witness testimony for second day - 3/30 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=GZMdt1RDGVE
Derek Chauvin trial continues with witness testimony for third day - 3/31 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=qz_BwDvd6Cw
Age restricted video: Derek Chauvin trial continues with witness testimony for fourth day - 4/1 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=E4XNcAZaeSQ
Derek Chauvin trial continues with witness testimony for fifth day - 4/2 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=JP5OpOEW2_g
Derek Chauvin continues with witness testimony for sixth day - 4/5 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=MoVMQ0HcLKQ
Derek Chauvin trial continues with witness testimony - 4/6 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=CPS9kYWckIQ
Derek Chauvin trial continues with witness testimony - 4/7 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=uLVXFFqZPiE
Derek Chauvin trial continues with witness testimony - 4/8 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=OQ9MspUIxVA
Derek Chauvin trial continues with witness testimony - 4/9 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=czEgldD05cI
Derek Chauvin trial continues with witness testimony - 4/12 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=aVHgckacpHE
Derek Chauvin trial continues with witness testimony - 4/13 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=ZgWVH_KF8Dk
Derek Chauvin trial continues with defense testimony - 4/14 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=S0GVL8omNB8
Derek Chauvin trial continues with defense testimony - 4/15 (FULL LIVE STREAM)
https://www.youtube.com/watch?v=OVq9SFwSmao
Closing arguments in the Derek Chauvin trial - 4/19 (FULL LIVE STREAM)