Actuarial sentencing and Bias Behind Bars
bit.ly/3BP0AZu
You can also find the GitHub repository for this presentation at:
About me
Tom Cardoso, investigative reporter
at The Globe and Mail
First, a primer on prisons
- A sentence of two years or longer lands you in federal prison.
- Roughly 12,000 to 14,000 people in custody each year.
- Most prisoners don’t serve until their sentence is up — instead, they get paroled or stat released
- Various institutions across the country for men and women. They’re divided into minimum-, medium- and maximum-security facilities.
- We have a massive overincarceration problem for Indigenous and Black people.
Why did we focus on risk scores?
- Within prison, risk assessments are a constant. Prisoner are repeatedly assessed and reassessed, and their life inside depends in large part on how they score
- In essence, these tools are designed to measure an person’s risk to the public and their odds of rehabilitation
- Work, housing, treatment programs, privileges, parole – scores affect it all
- As a federal judge put it in 2015, risk scores are “like a branding – hard to overcome”
- Canada is something of a world leader in risk assessments
- The risk-need-responsivity model, used around the world, was developed in Canada
- Most tools will output a level: low, medium, high, etc.
- Others will also give you a numeric score, as well as a person’s likelihood of reoffending within a certain period of time
- Criminal risk tools are the grandparents of modern machine learning models. They’re pen-and-paper algorithms.
While CSC’s tools don’t explicitly ask about a person’s race, they can certainly capture it by asking questions about family structure, substance use, levels of education, where a person grew up, etc.
In fact, race has previously been a direct component of risk assessments.
In 1928, Ernest Burgess, a Canadian-born University of Chicago sociologist, built a now-famous tool for the Illinois Board of Paroles that predicted a person’s odds of reoffending. One of the 22 variables used was the race of the person’s father.
So: What are the Correctional Service of Canada’s risk assessments like?
CSC uses two separate risk scoring approaches:
Actuarial tools, which require officers to fill out a questionnaire and tally up the resulting score
Structured professional judgment tools, which rely on an officer’s experience and opinion
Though there are many risk tools and types of scores in federal prison, two are most important: security classification and the reintegration potential score.
Custody Rating Scale
Static Factors
Dynamic Factors
But…
But: Evaluating risk is tricky. Tons of room for bias.
- Highly subjective to the assessor (could lead to low inter-rater reliability)
- Results are tough to interpret. What does it mean to have a “medium” security risk?
- Experts disagree over how risk scores should be developed and administered
- Current federal risk tools are old. Over time, they become less effective
- Most were built off a sample of mostly white, male prisoners from the 1980s
- Danger of cultural bias
Another major problem with risk scores is that they often rely on criminal history, which becomes a proxy for race. As Prof. Bernard Harcourt at Columbia University put it to me:
“You’re not predicting the actual essence of the person and their dangerousness – you’re predicting how likely it is that, because of policing disparities, they would get arrested again. … You’re not predicting crime. You’re predicting policing.”
In other words, being Black or Indigenous becomes a risk factor in itself.
What we found
After using a model to control for variables like age, gender, the severity of people’s offences and past contact with the criminal justice system…
Black men are roughly 24% more likely than white men to end up in maximum security at admission.
Indigenous men are roughly 30% more likely to have the worst reintegration score at any point.
Both are less likely to reoffend after controlling for reintegration scores.
Even worse for Indigenous women
They’re roughly 64% more likely than white women to end up in maximum security at admission.
Also roughly 40% more likely to receive the worst reintegration score.
And, as an internal report we obtained made clear: The government had warned the Correctional Service in 2004 of serious bias in its risk scores.
To this day, the risk tools used to produce those scores remain unchanged.
Earlier this year, also explored the impact of race and risk scores on parole.
Parole decisions are their own kind of risk assessment.
Before you’re released, you need to go before the Parole Board of Canada and make your case for release — a process which includes questions about your risk to reoffend, and which considers your risk scores.
“The law is saying that at the time of release, we’re supposed to be looking at what they’re doing to society in the future. … When you say that, you’re saying we’re going to punish people for what they haven’t yet done.” — Prof. Tony Doob, U of T
- This is Renford Farrier, who received a life-10 sentence after pleading guilty to second-degree murder in the early 90’s.
- But after 30 years, Renford is still inside. He says it’s because of his race, and the impacts that has on his risk scores and experience of incarceration.
- Worth listening to Renford tell his own story. He spoke on The Globe’s podcast, The Decibel. Listen here.
- He’s not alone. Our analysis found that Indigenous, Black and other racialized men are respectively 26, 24 and 20 per cent less likely than their white peers to be paroled in the first year they’re eligible.
Response and feedback
- First story published on a Saturday. By Monday, the House of Commons’ public safety committee had announced a study of systemic racism in prison risk assessments with all-party support
- Prime Minister acknowledged findings a few days later
- Lawyers have used our reporting at parole hearings
- Two class-action lawsuits filed against the government on behalf of tens of thousands of prisoners
- Findings reiterated by Auditor-General of Canada earlier this year
- Series has won journalism awards both in Canada and abroad
What surprised us about what we uncovered?
- The (quantifiable!) impact of race on people’s scores was very large
- In fact, it was so large we were initially convinced we’d made a grave methodological error
- Furthermore, the Correctional Service had been aware of these issues, both through a Public Safety Canada report and numerous reports from other agencies and experts, for almost two decades — but did nothing
- Even parole officers we spoke with had concerns about these tools, and they use them every day!
Where does the government (and academia) go from here?
- Given the current class-action suit, it seems unlikely CSC will voluntarily pull its risk tools any time soon
- The series seems to have kicked off a conversation in some places about what a better custodial risk assessment might look like
- (Do we even need one at all?)
- It’s worth asking: What would a culturally-appropriate risk tool look like?
- How can we mitigate and counteract the avalanche of biases faced by racialized prisoners?
Stories
- Bias behind bars: A Globe investigation finds a prison system stacked against Black and Indigenous inmates (Oct. 24, 2020)
- How we did it: How The Globe uncovered systemic bias in prisoners’ risk assessments (Oct. 24, 2020)
- A shot in the dark and 185 megabytes of data: How I investigated a system of bias in Canada’s prison system (Oct. 24, 2020)
- ‘This needs action’: NDP, experts call for solutions to racial bias in federal prison risk assessments (Oct. 26, 2020)
- Fight against systemic racism in prison wins all-party support (Oct. 26, 2020)
- More needs to be done to fight systemic racism in federal prisons, Justin Trudeau says (Oct. 27, 2020)
- MPs ask Bill Blair for timeline to address systemic racism in prisons (Nov. 2, 2020)
- For Indigenous women, systemic racial bias in prison leaves many worse off than men (Dec. 31, 2020)
- Inmate risk assessment tool still in use 16 years after report raises concerns about bias against women (Jan. 3, 2021)
- Proposed class-action suit against Ottawa suggests inmates face systemic bias in risk assessments (Jan. 12, 2021)
- Injunction request aims to end prison risk-assessment tool biased against Indigenous people (June 2, 2021)
- No way out: Once behind bars, racialized people are far less likely to get paroled when they are eligible. A Globe analysis examines why (February 22, 2022)
- Auditor-General says corrections authorities not preventing systemic racism in federal prisons (May 31, 2022)
Questions?
Perhaps we can start with a discussion: What would a culturally-responsive risk tool look like?