Skip to content

Advertisement

Open Access

Evaluating concurrent validity of criminal justice and clinical assessments among women on probation

  • Camila D. Odio1,
  • Megan Carroll2, 3,
  • Susan Glass4,
  • Ashley Bauman5,
  • Faye S. Taxman6 and
  • Jaimie P. Meyer7Email author
Health & Justice20186:7

https://doi.org/10.1186/s40352-018-0065-6

Received: 1 December 2017

Accepted: 22 March 2018

Published: 7 April 2018

Abstract

Background

Women in the criminal justice (CJ) system experience complex and comorbid medical, psychiatric, and substance use disorders, which often contribute to CJ involvement. To identify intersections between CJ and health needs, we calculated Spearman r correlations between concurrent CJ and clinical assessments from women on probation in Connecticut who were enrolled in a clinical trial. We examined longitudinal trends in CJ risk scores over 9 years of observation (2005–2014), modeling time to probation recidivism with shared gamma frailty models and comparing contiguous time points by Wilcoxon matched-pairs signed rank tests.

Results

Women (N = 31) were predominantly white (67.7%) with at least some high school education (58.1%) and mostly unemployed (77.4%) and unstably housed (83.9%). Most met clinical criteria for severe substance use and/or psychiatric disorders. Concurrent measures of substance use, mental health, social support, partnerships, and risk by the Level of Service Inventory-Revised (LSI-R) and clinical assessments were not significantly correlated. The LSI-R personal/emotional sub-score, however, positively correlated with the Addiction Severity Index psychiatric composite score (r = 0.40, 95% CI 0.03–0.68, p = 0.03). After adjusting for age, race and number of previous events, having some high school education versus none marginally decreased the hazard for probation recidivism and having > 5 inpatient psychiatric admissions versus none increased the hazard of probation recidivism 7-fold (HR 7.49, 95% CI 1.33–42.12, p = 0.022). Women with 0–1 recurrent probation terms (n = 16) had a significantly lower mean LSI-R score than those with 2–4 recurrent probation terms (35.9 [SD 6.4] versus 39.2 [SD 3.0], p = 0.019), but repeated LSI-R scores did not change over time, nor vary significantly beyond the group mean.

Conclusions

In this small, quantitative study of women on probation, widely used CJ assessment tools poorly reflected women’s comorbid medical, psychiatric, and substance use needs and varied minimally over time. Findings illustrate the limitations of contemporary CJ assessment tools for women with complex needs. The field requires more comprehensive assessments of women’s social and health needs to develop individualized targeted case plans that simultaneously improve health and CJ outcomes.

Keywords

WomenProbationCriminal justiceHIVRisk needs assessment

Background

The US incarcerates more of its citizens, and more women, than any other nation worldwide (Pew Chartitable Trusts, 2009). At year-end 2015, 2.7% of all US adults were under some form of correctional supervision, including prison, jail, probation and parole, and 1.9% were on probation or parole (Kaeble, 2016). Compared to the general adult population, people involved in criminal justice (CJ) systems disproportionately experience co-occurring medical, psychiatric, and substance use disorders (Bronson et al., 2017), and report four times more adverse childhood events (Reavis et al., 2013). More than half of all inmates meet criteria for substance abuse, and two-thirds of female and one-third of male prisoners have a diagnosed psychiatric disorder (Bronson & Berzofsky, 2017). Compared to CJ-involved men, CJ-involved women have a higher burden of complex comorbidities, including HIV (Binswanger et al., 2010). Women involved in CJ systems thus experience a set of unique vulnerabilities.

Incarceration is destabilizing and has detrimental effects on health (Binswanger et al., 2012). CJ involvement often disrupts continuity of care for chronic health conditions, including HIV, and contributes to poor health outcomes when resources are not provided during or following imprisonment (Meyer et al., 2014; Rich et al., 2015). Women are particularly vulnerable to disrupted care because they are more likely than their male counterparts to have experienced violence victimization and have untreated psychiatric and substance use disorders, including depression and post-traumatic stress (Women in the Criminal Justice System Briefing Sheets, 2007). Some women report substance use as a coping mechanism for traumatic life events (Morash, 2010), and fewer than 20% receive substance abuse treatment and 25% receive mental health services while involved with the criminal justice system (McCampbell, 2005).

Unrecognized or untreated psychiatric and substance use disorders are associated with recidivism and poor CJ outcomes (Fu et al., 2013; Baillargeon et al., 2010). Conversely, recognition and treatment of health needs, particularly substance use disorders, have been linked to decreased recidivism (Wooditch et al., 2014; Lamberti, 2016). In addition, lower rates of recidivism are reported among those who are employed (Holzer & Raphael, 2018), but poor health is a risk factor for unemployment (van Rijn et al., 2014). Women’s health has even broader reaching effects, as more than half of CJ-involved women are mothers (Glaze & Maruschak, 2010), and poor maternal health is associated with behavioral problems, poor health outcomes, and delayed verbal development in their children (Hardie & Turney, 2017).

As women interface with health and CJ systems, issues can potentially be identified so that they may be linked to care and services. Yet these systems are often difficult to navigate, particularly for women with limited health literacy or self-efficacy, and highly siloed. The seeming alignment between public health and public safety risks and outcomes place a burden on assessment protocols to reflect both CJ risk and need factors, including health issues that complicate outcomes. This raises issues related to the adequacy of the contemporary CJ risk and need assessments (RNA) to identify complex health issues. The field has developed an array of standardized instruments to measure static risk factors and dynamic needs and thereby efficiently triage large numbers of people in an era of mass incarceration (Monahan & Skeem, 2013). Criminal justice RNA are distinct from clinical assessments because they are focused on actuarial, rather than individual, factors that contribute to recidivism and they include a set of dynamic factors that are potentially modifiable (Taxman et al., 2006). In some states, RNA are used in criminal sanctioning; in others, scores are applied in post-judicial settings to develop and monitor case plans and to manage supervising officers’ caseloads (Kehl & Kessler, 2017). The most commonly used RNA instrument is the Level of Service Inventory-Revised (Lowenkamp et al., 2009), which has overall and sub-scores that are reliable and valid predictors of recidivism (Flores et al., 2006). Although there is limited data on how measured CJ risk factors and health risks interact, one study reported decreased longitudinal survival for individuals with high, compared to low, CJ risk (Folsom & Atkinson, 2007). Another cohort study of nearly 1000 adolescents demonstrated that co-occurring substance use and psychiatric disorders moderated the relationship between CJ risk markers and re-arrest (Schubert et al., 2011).

Because health and social factors are dynamic, additional concerns have been raised regarding the sensitivity of RNA instruments to adequately detect changes in individual-level factors over time (Wooditch et al., 2014; Schlager & Pacheco, 2011). Many of the dynamic needs are single item questions while others are questionnaires that include lifetime, last year and/or last month behavior. The dynamic need factors are often poorly conceived, lack content validity, and vary in measurement of concepts, limiting the utility of the RNA instrument to measure needs or to change over time (Via & Dezember, 2016).

Our small, quantitative study explores the intersections of health and CJ risk factors to better understand the dynamics of RNA measurements over time. We were particularly interested in how measurements of CJ risk reflect health risk for women. This is a secondary data analysis involving 31 women on probation, who underwent concurrent measurements of health and CJ risks as part of a clinical trial and probation supervision, allowing us to examine concurrent validity along several key domains. By reevaluating how CJ risk is measured, we hoped to identify modifiable factors that could affect women’s health and CJ outcomes, thus streamlining services for women and promoting women’s overall health.

Methods

Study setting

The study took place in Connecticut, wherein the state’s Judicial Branch, Court Support Services Division (CSSD), oversees adult probation for approximately 9000 women. For the approximately 750 women state-wide who are classified as “high risk” for re-offending, probation services are provided by the Women Offender’s Case Management program (WOCM), which includes gender-responsive case managers, specially trained probation officers, and women-specific “alternatives in the community” programming. HIV-related or other health-focused programming is not currently routinely incorporated into probation case plans.

Study participants and procedures

The parent study, known as +Pink (registered as NCT #03175094 at clinicaltrials.gov) and described elsewhere (Marcus et al., 2017), was designed to inform, adapt, and pilot test a behavioral HIV risk reduction intervention for CJ-involved women. Women were referred to +Pink by trained study staff on-site at probation offices, and HIV and primary care clinics in New Haven and Hartford, Connecticut. They could also be referred to the study by probation officers, case managers, or community healthcare providers, or could self-refer using a Qualtrics link or leaving a message on a secured dedicated voicemail. Referred participants were screened by a trained research assistant for the following inclusion criteria: 1) age ≥ 18 years old; 2) identify as female (cis- or trans-gendered); 3) CJ-involved, meaning they were sentenced to probation or receiving intensive pretrial supervision by a probation officer, sentenced to parole, or released from jail or prison within 60 days; and 4) living with diagnosed HIV or at-risk for HIV. HIV risk was defined as ever having injected drugs, ever exchanging sex for money or goods, being diagnosed with a sexually transmitted infection (STI) within 90 days of enrollment, having unprotected sex with a partner whose HIV status is unknown or HIV+ in the 90 days prior to enrollment, or being incarcerated in prison or jail during the two years prior to enrollment. Probation status was preliminarily confirmed by searching the Judicial Branch website for publicly available records. Women were excluded if they were unable or unwilling to provide written informed consent, unable to comfortably converse in English, or were threatening to study staff. After providing written informed consent and officially enrolling in the study, participants were asked to sign a Release of Information, which included data from CSSD and Adult Probation.

Data sources

For each study participant who allowed release of information from CSSD and Adult Probation, the unique probation identification number was confirmed using the individual’s first and last name, date of birth, and social security number. Date of study enrollment was matched to current probation term start and end dates to confirm probation status (i.e., currently sentenced to probation or receiving intensive pretrial supervision by a probation officer.) All women who self-reported being on probation had this status confirmed with a 100% match rate. Using the unique probation identification number, CSSD data managers then extracted all available demographic and CJ risk information collected between 2005 and 2016. As part of routine supervision procedures, within six months of entrance and ideally again during, and at the conclusion of each probation term, probation officers administer the Level of Service Inventory-Revised (LSI-R) in paper form (Bonta, 1995). Interviews are conducted in English or Spanish (with a phone-based translator), on a one-to-one basis with the probation officer and the participant. The instrument is divided into 10 subscales that include both dynamic and historical factors, each with a specific focus area of self-report, including: criminal history, education/employment, financial status, family/marital, accommodation (housing), leisure/recreation, companions, alcohol/drug problems, emotional/personal, and attitudes/orientation. Within each subscale, 2–10 yes or no questions are asked, for a total of 54 questions in the entire survey. Scores are totaled and categorized into low (0–21), medium (22–28), and high (> 29) risk groups (Ostermann & Herrschaft, 2013). The LSI-R is examined by CSSD on an individual case basis but has never been examined longitudinally in aggregate nor compared to health indicators. Probation officers also administer the Adult Substance Use Survey-Revised (ASUS-R) on intake (Alcohol and Drug Abuse Assessments, Surveys, and Software Programs, 2017). It is possible that probation officers independently collected additional information on women’s motivations for their behaviors, but this data is not routinely recorded in the central database. Extracted data was uploaded into password protected files and securely transferred to the research team for further analysis.

Clinical assessments were collected for each participant after enrollment in the clinical trial, as part of a baseline interview. Trained study staff conducted study interviews in English in a private setting using a structured protocol and validated instruments to measure substance use and substance use disorder severity, mental health, social support, partnerships, and HIV-related risk behaviors, as described further below. There was no qualitative data collected as part of the baseline study assessment. All data were entered into RedCAP. Clinical assessments were then merged with CSSD data using probation and study unique identifications numbers and securely shared among approved study team members for further analysis.

As shown in Fig. 1, 92 women on probation were contacted and screened for eligibility for enrollment in the clinical trial, of whom 31 were either ineligible based on the pre-set inclusion/exclusion criteria (as described above) or were not reachable within the catchment area. Of the 61 women who were enrolled in the clinical trial, 28 were not on probation (i.e. they were on parole or were recently released from prison or jail) and 2 were on probation but did not consent to CSSD data release. Ultimately, 31 women released CSSD data (representing 108 assessments over 9 years of observation) and were included in the present analysis.
Fig. 1

Flow of participants in +Pink Clinical Trial

Measures

Demographic data were obtained from CSSD records and included all available probation term start dates, age, race (categorized as black, white, or other), highest level of completed education (categorized as no, some, or completed high school), employment status (categorized as student, employed, unemployed, or disabled), and satisfaction with current housing accommodation.

Substance use was measured in the clinical trial by the Addiction Severity Index (ASI), 5th Edition alcohol and drug composite scores (McLellan et al., 1985). The ASI measures both past 30-day and lifetime substance use. Alcohol composite scores ≥0.17 indicate severe alcohol use disorders by DSM-IV criteria and drug composite scores ≥0.12 indicate severe substance use disorders by DSM-IV criteria (Rikoon et al., 2006). Substance use was measured by CSSD using the Level of Service Inventory-Revised (LSI-R (Bonta, 1995) alcohol/drug sub-scale and the ASUS-R, which asks about the number of times each of 9 substances have been used in one’s lifetime and in the past 6 months. The ASUS-R provides categorical levels of use for each substance (1 = one to 10 times; 2 = 11 to 25 times; 3 = 26 to 50 times; 4= > 50 times), giving a maximum score of 36 (Alcohol and Drug Abuse Assessments, Surveys, and Software Programs, 2017).

Mental health was measured in the clinical trial by the ASI psychiatric composite score, in which scores ≥0.22 are consistent with DSM-IV criteria for a severe psychiatric disorder (Calsyn et al., 2004), and the Quick Inventory of Depressive Symptomatology score (QIDS) (Rush et al., 2006). The QIDS is measured continuously with scores ranging 0–27 and higher scores indicate a more severe mood disorder; scores > 11 indicate major depression (Rush et al., 2003). Mental health was measured by CSSD using the LSI-R emotional/personal sub-scale.

Social support was measured in the clinical trial using the Social Support scale (Huba & Melchior, 1996), which is scored continuously from 0 to 100 with higher scores indicating greater social support; and in CSSD using the LSI-R family/marital and companions sub-scales.

Partnerships and relationships were measured in the clinical trial using the Revised Conflicts Tactics Scale-2 (STRAUS et al., 1996; STRAUS et al., 2003), which measures physical, sexual, emotional, and verbal intimate partner violence exposure and perpetration; and in CSSD with the LSI-R family/marital sub-scale.

Risk was measured in terms of HIV-related risk in the clinical trial using the NIDA Risk Behavior Assessment (Dowling-Guyer et al., 1994); and in CSSD in terms of criminogenic risk using the total LSI-R score. LSI-R results approaching the maximum score of 54 suggest higher criminogenic risk of “reoffending.”

Statistical analysis

The concurrent validity of CSSD and clinical assessments within each domain described above (substance use, mental health, social support, relationships/partnerships, and risk) were estimated by Spearman r correlations, with p < 0.05 indicating statistical significance. For participants with multiple available CSSD assessments over the observation period, the most recent assessment was used, which was contemporaneous with clinical assessments. Trends of all LSI-R scores over time were visually examined and quantified by Wilcoxon matched-pairs signed rank tests comparing initial, reassessment and discharge LSI-R scores. These analyses were performed with PRISM version 7 for Mac OS X. Risk factors for time to recidivism to probation, defined by any new probation intake after completion of an initial term, were examined using a shared gamma frailty model adjusted for age, race and number of previous events. The shared gamma frailty model includes a random effect to account for the intra-subject correlation observed in repeated measures data. The probability of not having a recurrent probation intake over time was assessed by examining Kaplan-Meier curves. These analyses were performed using R version 3.3.2 and the “survival” package version 2.38.

Results

As shown in Table 1, of the 31 women included in this secondary data analysis, the majority (58.1%) had some high school education, were unemployed (77.4%) and white (67.7%), and nearly all (83.9%) reported that their housing situation was inadequate, which essentially reflects the demographic profile of women on probation in the state. Participants’ mean LSI-R total score was 30.2 (SD 5.3), representing the highest CJ risk category. On further examination of potential contributing factors to CJ risk, many women identified current problems with alcohol (90.3%) or drugs (54.8%), and were receiving treatment for mental health issues, including multiple prior inpatient psychiatric admissions. Sixty-five percent of the women had a prior record of assault or violence. Three of the 31 participants were diagnosed and living with HIV, though many experienced significant HIV risk in terms of prior injection drug use (41.9%), transactional sex (32.3%), and sex with a partner with HIV or unknown sero-status (45.2%).
Table 1

Characteristics of Study Participants (N = 31)

Characteristics

N (%) or mean (SD)

Age, mean (SD)

33.7 (10.8)

Race, n (%)

 Black or Other

10 (32.3)

 White

21 (67.7)

Education, n (%)

 No High School

13 (41.9)

 Some High School

4 (12.9)

 High School Graduate

14 (45.2)

Employment Status, n (%)

 Student/Employed

5 (12.1)

 Unemployed

24 (77.4)

 Disabled

2 (6.5)

Unsatisfactory Housing Accommodation, n (%)

26 (83.9)

Total LSI* Score, mean (SD)

30.23 (5.33)

Record of Assault/Violence, n (%)

20 (64.5)

Current Alcohol Problem, n (%)

28 (90.3)

Current Drug Problem, n (%)

17 (54.8)

Current Mental Health Problem, n (%)

13 (41.9)

History of Inpatient Treatment, n (%)

 Never

14 (45.2)

 1–2 Times

11 (35.5)

 3–5 Times

4 (12.9)

 More than 5 times

2 (6.5)

HIV and Risk Factors, n (%)

 Diagnosed with HIV

3 (9.7)

 Ever used injection drugs

13 (41.9)

 Ever traded sex for drugs or money

10 (32.3)

 Had unprotected sex with HIV+ or HIV status unknown partner in the past 90 days

14 (45.2)

*LSI level of service inventory

Of the 8 individuals with non-zero ASI alcohol composite scores, the mean was 0.17 (SD 0.16), which meets DSM-IV criteria for a severe alcohol use disorder. Of the 23 individuals with non-zero ASI drug composite scores, the mean was 0.11 (SD 0.10), which reflects severe substance use disorders. Of the 22 individuals with completed ASI psychiatric composite scores, the mean was 0.43 (SD 0.24), which is nearly double the cut-off criteria for severe psychiatric disorders; 8 women met clinical criteria for major depression by the QIDS.

Concurrent validity analyses revealed very few correlations between the CSSD surveys and the clinical assessments performed in four domains (Table 2). In the substance use domain, lifetime ASUS-R alcohol and drug scores were positively associated with the ASI, when only ASI items that quantified years of prior alcohol or drug use were included. Of the six LSI-R sub-score associations examined, only the LSI-R personal/emotional sub-score was positively correlated with a clinical assessment, the ASI psychiatric composite score. There was also no significant difference in median LSI-R family/marital sub-scores between women who did (n = 10) and did not (n = 14) report experiencing intimate partner violence in the past year on the Revised Conflicts Tactics Scale. When comparing overall CJ risk to HIV-related sexual risk behaviors, the correlation approached statistical significance when one high outlier was removed (r = 0.34, 95% CI: -0.06 to 0.64, p = 0.08).
Table 2

Correlations between criminal and health assessments, by domain (N = 31)

Longitudinal examination of the 108 RNA assessments (including 80 intake assessments) revealed that six women had no further probation intakes after completing their first term, ten had one additional term and two had four additional terms (Fig. 2). Median time to recidivism for those with no prior probation terms was four years, but for those with one or two prior probation terms, the median time to recidivism decreased to three and two years respectively. The number of prior probation terms was inversely correlated with the median time to recidivism.
Fig. 2

Individual trajectories over time with Risk and Need Assessments (N = 31)

Table 3 shows the shared gamma frailty model of time to recidivism to probation. After adjusting for age, race and number of previous events, those with some high school education had a 60% lower hazard of reentry as compared to those with no high school education (HR: 0.40, 95% CI: 0.16 to 0.99, p = 0.048). Those with more than 5 inpatient psychiatric admissions had a 7.5 times greater hazard of recidivism as compared to those with less than five inpatient admissions (HR 7.49, 94% CI: 1.33 to 42.12, p = 0.02), though there were only two individuals who had > 5 inpatient admissions for psychiatric issues. Although the adjusted model did not reveal an association between total LSI scores and time to probation recidivism, women with 0–1 probation reentries (n = 16) did have significantly lower mean LSI-R scores than those with 2–4 reentries (35.9 [SD 6.4] versus 39.2 [SD 3.0], p = 0.019). Data was not available on whether participants were incarcerated in prison or jail during the observation period.
Table 3

Time to Recidivism by Shared Gamma Frailty Model (N = 31 women, 80 observations)

 

HR (95% CI)a

p-value

Education

 No High School

1.00 (referent)

 Some High School

0.404 (0.16, 0.99)

0.048

 High School Graduate

0.752 (0.35, 1.61)

0.460

Unsatisfactory housing

1.577 (0.91, 4.08)

0.350

Total LSI score

0.951 (0.89, 1.02)

0.130

Record of assault/violence

0.262 (0.09, 0.75)

0.013

Current alcohol problem

0.871 (0.31, 2.42)

0.790

Current drug problem

1.059 (0.58, 1.94)

0.850

Current mental health problem

0.957 (0.53, 1.73)

0.890

History of inpatient psychiatric treatment

 Never Treated

1.00 (referent)

 1–2 Times

1.429 (0.51, 4.01)

0.500

 3–4 Times

2.09 (0.62, 7.10)

0.230

 5 or more times

7.49 (1.33, 42.12)

0.022

HR hazard ratio, LSI level of service inventory

aadjusted for age, race, and number of previous events

Analysis of the LSI-R scores over time did not reveal significant trends (Fig. 3). Twenty-seven women completed the LSI-R more than once (N = 121 assessments), and trends did not vary significantly beyond the mean. There were no significant correlations in LSI-R scores between contiguous time points, likely related to a lack of variation. Mean intake LSI-R scores did not differ significantly from reassessment (38.4 [SD 5.0] versus 38.9 [SD 9.2], p = 0.71, n = 22) nor discharge LSI-R scores (36.7 [SD 3.2] versus 37.9 [SD 8.0], p = 0.56, n = 10). Time on probation was not associated with significant changes in LSI-R scores over time in our study sample population. Italicized values represent statistically significant difference.
Fig. 3

Boxplot of Level of Service Inventory Scores over Time

Discussion

In this cross-sectional study of 31 women on probation and living with or at-risk for HIV, we found few significant associations between clinical and CJ measurements of substance use, mental health, social support, relationships/partnerships, or risk. To our knowledge, this is the first evaluation of concurrent validity between the LSI-R and established clinical assessments of depression, substance abuse, and social support in women on probation. That very few correlations were observed suggests that the LSI-R insufficiently reflects women’s overall physical, psychiatric, and emotional health or needs.

Because criminal justice RNA are typically based on actuarial tables, they may misestimate women’s risk and needs with tangible impact on women’s criminal sanctions and supervision case plans (Ostermann & Herrschaft, 2013; Smith et al., 2009). Poor measurement qualities of LSI-R subscales additionally inflate risk by combining fixed and dynamic factors (Via & Dezember, 2016) and do not meet criteria for psychometrically sound scales. The LSI-R is inherently limited by reporting and social desirability biases; clients may perceive far greater repercussions to reporting ongoing substance use, for example, to a probation officer than to a clinician or researcher, even if actual substance use was corroborated by urine toxicology. Unrecognized and untreated substance use and psychiatric disorders are major contributors to poor health and CJ outcomes, and therefore CJ RNA are designed to screen for these substance use and psychiatric disorders. While clinical/research institutions and CJ systems would ideally collaborate to improve individual and public health and safety (Rich et al., 2011), potential cross-talk is limited if goals and measures are not aligned. As a result, women interfacing with both health and CJ systems may have to manage competing institutional priorities, instead of the systems themselves shifting to be more women-centered.

The use of RNA in CJ settings is recommended to identify risk levels as well as service needs. This study illustrates that while RNA are important, there is more work to be done to ensure that the tools serve their multi-dimensional purposes: 1) to identify risk level to ensure that women are not exposed to over-supervision in probation/parole settings; 2) to identify need factors that can be modified to reduce involvement in the CJ system and improve overall functioning; and 3) to tailor services to the unique needs of the individual. Many of the existing RNA tools, like the LSI-R, were developed over 30 years ago and therefore do not reflect this focus on the unique needs of individuals. In contrast, the Women’s Risk Needs Assessment (WRNA), for example, is a more recent tool designed to detect gender-specific needs (e.g., in-depth measures of psychiatric disorders; abuse and trauma histories) and strengths such as self-efficacy (Voorhis et al., 2010). These domains on the WRNA more closely parallel clinical assessments. More individualized and health-focused tools have the power to simultaneously improve people’s health and CJ outcomes, especially if periodically quality-assured, delivered by well-trained staff, and when used (as intended) to inform case plans (Viglione et al., 2015). But this study has shown that more sophisticated RNA tools are additionally needed to ensure that clients receive appropriate services.

Our evaluation of 108 CJ RNA available over 9 years of observation revealed minimal variation in risk scores beyond the mean and no significant trend in scores over time. We observed no difference between initial and reassessment LSI-R scores or initial and discharge scores, contrary to expected declining LSI-R scores over time as a result of targeted case plans (Schlager & Pacheco, 2011; Raynor, 2007). This finding may be due to the quality of the LSI-R administration or it could be due to misidentification and under-treatment of behavioral health needs which generally exacerbated the women’s conditions.

To date, little research has focused on probation recidivism among women with or at-risk for HIV; this work begins to address that gap. Women with 2–4 recurrent probation terms had significantly shorter time to recidivism and higher LSI-R scores than those with fewer probation terms, which is consistent with observations of larger, more diverse cohorts (Vose et al., 2013). Our model adjusted for age, race and number of previous probation periods because these are established risk factors for recidivism in people living with and at-risk for HIV (Baillargeon et al., 2010; White et al., 2008). Older age and more previous probation terms served accounted for a significant increase in the hazard of probation recidivism. Our model indicated that some high school education and no previous inpatient admissions for psychiatric issues were protective against probation recidivism, which aligns with previous studies (Fu et al., 2013; White et al., 2008). Perhaps unexpectedly, we found that a history of prior assault was protective against probation recidivism. Since violent assault is associated with prison/jail recidivism (Fu et al., 2013), this offense may be more likely to result in an incarceration rather than a probation sentence. Further qualitative data is needed to better understand why women perpetrate violence and how this impacts their lived experiences of interacting with CJ systems, applying theory-driven approaches to fully understand people’s behavior.

Despite the novelty of our analytic approach that explored the concurrent validity of criminal justice RNA and clinical assessments using standardized instruments and our inclusion of all RNA assessments over nearly a decade of observation, this study was limited primarily by small sample size. This may have limited our power to detect a significant difference between scores over time, though our findings are consistent with those of larger studies of women. Decreased concurrent validity between the CSSD surveys and clinical assessments could represent social desirability bias since participants may provide more honest responses to clinical researchers as opposed to probation officers. Future studies could examine whether participant responses vary with the position of the interviewer. Generalizability may also be limited by the small sample size. While the demographics of our sample reflects that of women on probation in the state (the population from which the sample was derived), non-Hispanic white and highly education women may be over-represented here compared to other CJ settings. As a proof of concept, however, our work was designed primarily to generate further discussion about the utility of RNA for women with complex health needs and to direct areas for future research. As CJ reform is implemented to decarcerate and keep women in the community longer, it will become ever more important to accurately measure and intervene to address women’s needs.

Conclusions

In this cross-sectional study of women on probation, widely used RNA tools poorly reflected women’s complex medical, psychiatric, and substance use needs and varied minimally over time, which potentially undermines efforts to direct case plans and reduce recidivism. Future studies should evaluate the ways in which a more nuanced, individualistic approach to evaluating and modifying women’s complex needs can simultaneously improve health and CJ outcomes.

Declarations

Acknowledgements

We thank Connecticut Court Support Services Division for their collaboration on this project, particularly officers and regional managers from the Women Offenders Case Management program. We appreciate the time, energy, and dedication of our study team, especially Barbara Valdes. We thank study participants, without whom this work would not have been possible, and for whom this work is dedicated.

Funding

Research and career development support provided by the National Institutes on Drug Abuse (K23 DA033858 to JPM). Funding sources played no role in data collection, analysis, drafting of the manuscript, or decision to submit the manuscript for publication.

Availability of data and materials

We are unable to share the data because the Memorandum of Understanding between CSSD and Yale specifies that the data will not be made publicly available.

Authors’ contributions

This study was conceptualized by CO, MC, SG, and JPM with input from AB and FT. CO and MC conducted all analyses and were responsible for primary interpretation of findings. CO drafted the manuscript, with guidance from JPM. All authors reviewed and revised the manuscript and approved the final version of the manuscript prior to submission for publication.

Ethics approval and consent to participate

The study protocol was reviewed and approved by the Human Investigations Committee at Yale University and the Institutional Review Board at CSSD. Yale University and CSSD entered into a Memorandum of Understanding to enable secured data sharing. A Certificate of Confidentiality from the National Institutes of Health provided additional assurances of confidentiality. All study participants provided written informed consent in order to enroll in this study. Additionally, they signed a Release of Information, which included data from CSSD and Adult Probation.

Consent for publication

Not applicable

Competing interests

The authors have no competing interests to report.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Department of Internal Medicine, Yale New Haven Health, New Haven, USA
(2)
Department of Biostatistics, Yale School of Public Health, New Haven, USA
(3)
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, USA
(4)
Connecticut Judicial Branch, Court Support Services Division, Wethersfield, USA
(5)
Bauman Consulting Group, LLC, Loveland, USA
(6)
Criminology, Law & Society, George Mason University, Fairfax, USA
(7)
AIDS Program, Yale School of Medicine, New Haven, USA

References

  1. Alcohol and Drug Abuse Assessments, Surveys, and Software Programs. Available at: http://aodassess.com/. Accessed 22 Jan 2018.
  2. Andrews, D, Bonta, J. (1995). The LSI-R: The Level of Service Inventory–Revised. Multi-Health System. Toronto, Canada.Google Scholar
  3. Baillargeon, J, Giordano, TP, Harzke, AJ, et al. (2010). Predictors of reincarceration and disease progression among released HIV-infected inmates. AIDS Patient Care and STDs, 24, 389–394.View ArticleGoogle Scholar
  4. Binswanger, IA, Merrill, JO, Krueger, PM, White, MC, Booth, RE, Elmore, JG. (2010). Gender differences in chronic medical, psychiatric, and substance-dependence disorders among jail inmates. American Journal of Public Health 100, 476–482.View ArticleGoogle Scholar
  5. Binswanger, IA, Nowels, C, Corsi, KF, et al. (2012). Return to drug use and overdose after release from prison: A qualitative study of risk and protective factors. Addiction Science & Clinical Practice, 7, 3.View ArticleGoogle Scholar
  6. Bronson, J, Berzofsky, M. Indicators of Mental Health Problems Reported by Prisoners and Jail Inmates, 2011–12. 2017; Available at: https://www.bjs.gov/content/pub/pdf/imhprpji1112.pdf.
  7. Bronson, J, Stroop, J, Zimmer, S, Berzofsky, M. Drug use, dependence, and abuse among state prisoners and jail inmates, 2007–2009. 2017; Available at: https://www.bjs.gov/content/pub/pdf/dudaspji0709.pdf.
  8. Calsyn, DA, Saxon, AJ, Bush, KR, et al. (2004). The addiction severity index medical and psychiatric composite scores measure similar domains as the SF-36 in substance-dependent veterans: Concurrent and discriminant validity. Drug and Alcohol Dependence, 76, 165–171.View ArticleGoogle Scholar
  9. Dowling-Guyer, S, Johnson, ME, Fisher, DG, et al. (1994). Reliability of drug users’ self-reported HIV risk behaviors and validity of self-reported recent drug use. Assessment, 1, 383–392.View ArticleGoogle Scholar
  10. Flores, AW, Lowenkamp, CT, Holsinger, AM, Latessa, EJ. (2006). Predicting outcome with the level of service inventory-revised: The importance of implementation integrity. J. Crim. Justice, 34, 523–529.View ArticleGoogle Scholar
  11. Folsom, J, & Atkinson, JL. (2007). The generalizability of the LSI-R and the cat to the prediction of recidivism in female offenders. Criminal Justice and Behavior, 34, 1044–1056.View ArticleGoogle Scholar
  12. Fu, JJ, Herme, M, Wickersham, JA, et al. Understanding the revolving door: Individual and structural-level predictors of recidivism among individuals with HIV leaving jail. AIDS Behav. 2013; 17 Suppl 2:S145–S155.Google Scholar
  13. Glaze, L, Maruschak, L. (2008). Bureau of Justice Statistics Speical Report: Parents in prison and their minor children. NCJ 222984. Available at: https://www.bjs.gov/content/pub/pdf/pptmc.pdf.
  14. Hardie, JH, & Turney, K. (2017). The intergenerational consequences of parental health limitations. Journal of Marriage and the Family, 79, 801–815.View ArticleGoogle Scholar
  15. Holzer, H, & Raphael, S, Stoll M. Employment Barriers Facing Ex-Offenders: Understanding the Nexus between Prisoner Reentry and Work: Employment Dimensions of Reentry. Urban Institute Reentry Roundtable. May19-20, 2003. New York University Law School. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.508.2285&rep=rep1&type=pdf.
  16. Huba, G, Melchior, L, Staff of the Measurement Group, HRSA/HAB’s SPNS Cooperative Agreement Steering Committee. (1996). Module 46: Social Support Form. The Measurements Group. Culver City.Google Scholar
  17. Kaeble, D, & Glaze, L. (2016). Correctional Populations in the United States, 2015. NCJ 250374. Available at: https://www.bjs.gov/content/pub/pdf/cpus15.pdf.
  18. Kehl, DL, Kessler, SA. Algorithms in the criminal justice system: Assessing the use of risk assessments in sentencing. 2017; Available at: https://dash.harvard.edu/handle/1/33746041. Accessed 21 Jan 2018.
  19. Lamberti, JS. (2016). Preventing criminal recidivism through mental health and criminal justice collaboration. Psychiatric Services, 67, 1206–1212.View ArticleGoogle Scholar
  20. Lowenkamp, CT, Lovins, B, Latessa, EJ. (2009). Validating the level of service inventory—Revised and the level of service inventory: Screening version with a sample of probationers. Prison Journal, 89, 192–204.View ArticleGoogle Scholar
  21. Marcus, R, Copenhaver, M, Altice, FL, Meyer, JP. (2017). Where rubbers meet the road: HIV risk reduction for women on probation. 2017 Annual Meeting of the Society for Applied Anthropology. Santa Fe, New Mexico.Google Scholar
  22. McCampbell, S. (2005). The gender-responsive strategies project: Jail applications. Gender-Responsive Strategies for Women Offenders. U.S. Department of Justice, National Institute of Corrections, Accession Number 020417.Google Scholar
  23. McLellan, AT, Luborsky, L, Cacciola, J, et al. (1985). New data from the addiction severity index. Reliability and validity in three centers. The Journal of Nervous and Mental Disease, 173, 412–423.View ArticleGoogle Scholar
  24. Meyer, JP, Cepeda, J, Springer, SA, Wu, J, Trestman, RL, Altice, FL. (2014). HIV in people reincarcerated in Connecticut prisons and jails: An observational cohort study. Lancet HIV, 1, e77–e84.View ArticleGoogle Scholar
  25. Monahan, J, Skeem, JL. Risk Redux: the resurgence of risk assessment in criminal sanctioning. Rochester, NY: Social science research Network, 2013. Available at: https://papers.ssrn.com/abstract=2332165. Accessed 21 Jan 2018.
  26. Morash, M. (2010). Women on Probation and Parole: A Feminist Critique of Community Programs and Services (Northeastern Series on Gender, Crime, and Law). Northeastern University Press.Google Scholar
  27. Ostermann, M, & Herrschaft, BA. (2013). Validating the level of service inventory-revised: A gendered perspective. Prison Journal, 93, 291–312.View ArticleGoogle Scholar
  28. Pew Chartitable Trusts. Pew center on the states, One in 31: The long reach of American corrections. 2009. Available at: http://pew.org/220dydz. Accessed 21 Jan 2018.
  29. Raynor, P. (2007). Risk and need assessment in British probation: The contribution of LSI-R. Psychology, Crime & Law, 13, 125–138.View ArticleGoogle Scholar
  30. Reavis, JA, Looman, J, Franco, KA, Rojas, B. (2013). Adverse childhood experiences and adult criminality: How long must we live before we possess our own lives? The Permanente Journal, 17, 44–48.View ArticleGoogle Scholar
  31. Rich, JD, McKenzie, M, Larney, S, et al. (2015). Methadone continuation versus forced withdrawal on incarceration in a combined US prison and jail: A randomised, open-label trial. The Lancet, 386, 350–359.View ArticleGoogle Scholar
  32. Rich, JD, Wohl, DA, Beckwith, CG, et al. (2011). HIV-related research in correctional populations: Now is the time. Current HIV/AIDS Reports, 8, 288–296.View ArticleGoogle Scholar
  33. Rikoon, SH, Cacciola, JS, Carise, D, Alterman, AI, McLellan, AT. (2006). Predicting DSM-IV dependence diagnoses from addiction severity index composite scores. Journal of Substance Abuse Treatment, 31, 17–24.View ArticleGoogle Scholar
  34. Rush, AJ, Trivedi, MH, Ibrahim, HM, et al. (2003). The 16-item quick inventory of depressive symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): A psychometric evaluation in patients with chronic major depression. Biological Psychiatry, 54, 573–583.View ArticleGoogle Scholar
  35. Rush, AJ, Trivedi, MH, Wisniewski, SR, et al. (2006). Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: A STAR*D report. The American Journal of Psychiatry, 163, 1905–1917.View ArticleGoogle Scholar
  36. Schlager, MD, & Pacheco, D. (2011). An examination of changes in LSI-R scores over time: Making the case for needs-based case management. Criminal Justice and Behavior, 38, 541.View ArticleGoogle Scholar
  37. Schubert, CA, Mulvey, EP, Glasheen, C. (2011). Influence of mental health and substance use problems and criminogenic risk on outcomes in serious juvenile offenders. Journal of the American Academy of Child and Adolescent Psychiatry, 50, 925–937.View ArticleGoogle Scholar
  38. Smith, P, Cullen, FT, Latessa, EJ. (2009). Can 14,737 women be wrong? A meta-analysis of the LSI-R and recidivism for female offenders. Criminol. Public Policy, 8, 183–208.Google Scholar
  39. Straus, Ma, Hamby, Sl, Sue, Boney-Mcoy, Sugarman, Db. (1996). The revised conflict tactics scales (CTS2): Development and preliminary psychometric data. Journal of Family Issues, 17, 283–316.Google Scholar
  40. Straus, Ma, Sl, Hamby, Warren, W. (2003). The Conflict Tactics Scales Handbook. Los Angeles: Western Psychological Services.Google Scholar
  41. Taxman, FS, Thanner, M, Weisburd, D. (2006). Risk, need, and responsivity (RNR): It all depends. Crime Delinquency, 52, 28–51.View ArticleGoogle Scholar
  42. The Sentencing Project. (2007). Women in the Criminal Justice System Briefing Sheets. Available at: http://www.sentencingproject.org/wp-content/uploads/2016/01/Women-in-the-Criminal-Justice-System-Briefing-Sheets.pdf.
  43. van Rijn, RM, Robroek, SJW, Brouwer, S, Burdorf, A. (2014). Influence of poor health on exit from paid employment: A systematic review. Occup Env. Med, 71, 295–301.View ArticleGoogle Scholar
  44. Via, B, & Dezember, A, Taxman, F. (2016). Exploring How to Measure Criminogenic Needs. In Handbook on Risk and Need Assessment (Ed.), Fave S, Taxman and Amy Dezember (Abingdon: Routledge), Routledge Handbooks Online. Available at: https://www.routledgehandbooks.com/doi/10.4324/9781315682327.ch14. Accessed 28 Mar 2018.
  45. Viglione, J, Rudes, DS, Taxman, FS. (2015). Misalignment in supervision: Implementing risk/needs assessment instruments in probation. Criminal Justice and Behavior, 42, 263–285.View ArticleGoogle Scholar
  46. Voorhis, PV, Wright, EM, Salisbury, E, Bauman, A. (2010). Women’s risk factors and their contributions to existing risk/needs assessment: The current status of a gender-responsive supplement. Criminal Justice and Behavior, 37, 261–288.View ArticleGoogle Scholar
  47. Vose, B, Smith, P, Cullen, FT. (2013). Predictive validity and the impact of change in Total LSI-R score on recidivism. Criminal Justice and Behavior, 40, 1383–1396.View ArticleGoogle Scholar
  48. White, MC, Marlow, E, Tulsky, JP, Estes, M, Menendez, E. (2008). Recidivism in HIV-infected incarcerated adults: Influence of the lack of a high school education. J. Urban Health Bull. N. Y. Acad. Med., 85, 585–595.View ArticleGoogle Scholar
  49. Wooditch, A, Tang, LL, Taxman, FS. (2014). Which criminogenic need changes are most important in promoting desistance from crime and substance use? Criminal Justice and Behavior, 41, 276–299.View ArticleGoogle Scholar

Copyright

© The Author(s). 2018