Index Calculation Methodology
The indicators used by the Justice Index take two basic forms. For the Self-Representation Index, Language Access Index, and Disability Access Index, each indicator asks whether the jurisdiction has adopted certain practices “through a statewide statute, rule, regulation, appropriation, or other written guidance.”
For the Attorney Access Index, the Justice Index relies on a single indicator, the “civil legal aid attorney ratio,” for producing the Index score. To calculate this ratio, we divide the number of full-time-equivalent civil legal aid attorneys employed in the state by the number of people in the state with incomes at or below 200% of the federal poverty level.
Justice Index 2016 presents additional indicators and data sets that, while not used to calculate the Index scores, help provide a more complete picture of access to justice in the 52 jurisdictions. Such indicators and data sets, which can be found on the Findings pages and in the downloadable documents, include, for example, Comparison Charts analyzing the Index scores in relation to median income, region, population, and specific demographics; ratios of civil legal aid attorneys to all attorneys; and the number of civil legal aid attorneys and legal aid organizations in each jurisdiction.
Composite Index Calculation
To produce the Composite Index, each of the four subject matter index scores are added together and then divided by four, according an equal weight of 25% to each category’s set of indicators.
Weights of Indicators
Two weighting systems are used in the Justice Index. For the Self-Representation Index, Language Access Index, and Disability Access Index, NCAJ assigned a weight of 1, 5, or 10 to each of the indicators. This scale, with its three basic values signaling contrasting levels of importance, was selected for ease of understanding and usability. For the Attorney Access Index, NCAJ did not assign a weight, but rather accorded a value to the single indicator based on the arithmetically obtained score, adjusted to a scale of 100, reflecting the civil legal aid attorney ratio (i.e., a higher ratio translated into a higher score).
In setting weights at 1, 5, or 10, NCAJ relied on its staff’s expertise and on consultations with national subject matter specialists. NCAJ took into account (i) the importance of the law, rule, or practice for access to justice; (ii) the relative importance of the law, rule, or practice as compared to other indicators in the same subject matter index; and (iii) the cumulative weight of multiple indicators that track aspects of the same issue (for example, where a court’s website is evaluated based on consideration of multiple indicators).
To determine a score for each jurisdiction with respect to each indicator, NCAJ made a finding of either “Yes” or “No.” (As stated above, it is important to note that a “No” finding means only that a “Yes” finding could not be established; it does not represent an affirmative finding that the practice has not been adopted.) Points were awarded for “Yes” findings in the values 1, 5, or 10, reflecting the weights assigned to each indicator. No partial points were awarded for any indicators.
Standardization of All Index Scores to a 100-Point Scale
The Composite Index and the four subject matter indexes rely on a point scale of 0 to 100.
For the Self-Representation, Language Access, and Disability Access Indexes, each jurisdiction’s scores for all indicators were added together to determine a total raw score for the category index. The raw score was then divided by the maximum number of points that could be earned in the category, and multiplied by 100 to yield the scaled index final score.
For example, consider this hypothetical category with three indicators/questions and the following assigned weights:
Assuming a “Yes” for each indicator, the maximum raw score possible for a state in this category is 16 (10 + 5 + 1). If a state received a finding of “Yes” for Q1 but not for Q2 or Q3, it would have a raw score of 10, which is the weight assigned to Q1. To determine the scaled index score, the raw score, 10, would be divided by the maximum number of points available, 16, resulting in a quotient of 0.625, which would be multiplied by 100 for a scaled index final score of 62.5.
To calculate a score in the Attorney Access category for each state, NCAJ began by treating each state’s civil legal aid ratio as the state’s raw score.
The raw score was then converted to a 100-point scale suitable for inclusion in the Composite Index, by comparing it to a civil legal aid ratio used by NCAJ as a benchmark goal (the “benchmark civil legal aid ratio”). NCAJ set this goal at 10 civil legal aid lawyers per 10,000 people with incomes at or below 200% of the federal poverty level. Thus, a state with a raw score of 10 or higher (matching or exceeding the benchmark goal) would earn the maximum Attorney Access scaled index score (for inclusion in the Composite Index) of 100.
To calculate each state’s actual scaled index score, we divided the state’s raw score by 10 (the benchmark civil legal aid ratio figure), and then multiplied by 100. Thus, the scaled index scores are expressed as a percentage of the benchmark figure. (See “Attorney Access Methodology” for more information).
Computation of the Composite Index
The four category index scores were given an equal weight of 25% and averaged together to produce each state’s score for inclusion in the Composite Index, which also relies on a scale of 0 to 100.
For example: If a state has the following category index scores:
Then the composite index score for this state is = (22 + 39 + 65 + 22) / 4 = 37.
An exception to this formula was made for Puerto Rico, for which the Composite Index score excludes the category index for Language Access.
Visualizations Relying on Additional External Data
For three categories of data, the Justice Index also presents data visualizations that draw comparisons between the Justice Index’s findings and certain external data sources. The Justice Index categories for which these visualizations have been prepared, and the external data sources, are the following:
American Bar Ass’n, ABA National Lawyer Population Survey (2015), http://www.americanbar.org/content/dam/aba/administrative/market_research/national-lawyer-population-by-state-2015.authcheckdam.pdf (Providing the number of active lawyers for the 50 states, D.C., and Puerto Rico. Most recent link here.).
Legal Services Corporation, Number of Attorneys Per LSC-funded Organization (on file with NCAJ)
U.S. Census Bureau, “Annual Estimates of the Resident Population: April 1, 2010 to July 1, 2014,” 2014 Population Estimates, http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=PEP_2015_PEPANNRES&src=pt
U.S. Census Bureau, “Poverty Status in the Last 12 Months,” 2014 American Community Survey 1-Year Estimates, http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=ACS_14_1YR_S1701&prodType=table (providing data by state for “[a]ll individuals below 200% of poverty level”).
U.S. Census Bureau, “Annual Estimates of the Resident Population: April 1, 2010 to July 1, 2014”, http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=PEP_2015_PEPANNRES&src=pt
U.S. Census Bureau, Median Household Income by State – Single-Year Estimates: 2014,
U.S. Census Bureau, “Selected Economic Characteristics,” 2014 Puerto Rico Community Survey, http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=ACS_14_5YR_DP03&prodType=table (providing median household income for Puerto Rico).
Camille Ryan, U.S. Census Bureau, Language Use in the United States: 2011, at 11 (Table 4), available at http://www.census.gov/prod/2013pubs/acs-22.pdf (figures used in the Justice Index for state percentages of people with limited English proficiency were determined by adding together the percentages in the columns “Spoke English ‘not well’” and “Spoke English ‘not at all’”).
U.S. Census Bureau, “Percentage of People with a Disability – States and Puerto Rico,” 2014 American Community Survey 1-Year Estimates, http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=ACS_14_1YR_GCT1810.US01PR&prodType=table