ESSA Accountability Technical Manual

The Every Student Succeeds Act (ESSA) challenges state education agencies (SEAs) and local education agencies (LEAs) to improve student outcomes by addressing the student-, teacher-, and school-level factors that drive achievement gains.

The purpose of the technical manual is to document key details of critical components of New Mexico’s Every Student Succeeds Act (ESSA) accountability system. Each section includes descriptions of relevant 1) methodology; 2) business rules; 3) definitions; and 4) internal decisions that impact the component.

The information is used to create an ESSA (or index) score which helps to identify schools that are doing well overall and schools that may need extra support to improve.

Long-term Goals and Measures of Interim Progress

Language Arts

Methodology:

Academic long-term goals were originally established in 2016-17 for the lowest performing student group to have a proficiency rate of at least 50% (with statewide averages for all students of 64.9% in language arts and 61.2% in math) by 2022, with simultaneous gains for all groups on near-parallel tracks.¹ NMPED’s approved 2021-22 addendum for the consolidated state plan established that long-term goals would shift forward by two years for academic achievement, graduation rate, and English language proficiency. As such, the long-term goals applied for 2022-23 accountability are the same as the original 2022 goals.

2022-23 language arts long-term goals

These goals were originally established based on baseline data from the PARCC assessment. New Mexico has since transitioned from the PARCC assessment and now requires the New Mexico Measures of Student Success & Achievement (NM‑MSSA) in grades 3-8 and the SAT School Day in high school. NMPED will establish new long-term goals and measures of interim progress in an amended CSP to be submitted to the U.S. Department of Education for 2023-24 accountability.

Mathematics

Methodology:

Academic long-term goals were originally established in 2016-17 for the lowest performing student group to have a proficiency rate of at least 50% (with statewide averages for all students of 64.9% in language arts and 61.2% in math) by 2022, with simultaneous gains for all groups on near-parallel tracks. NMPED’s approved 2021-22 addendum for the consolidated state plan established that long-term goals would shift forward by two years for academic achievement, graduation rate, and English language proficiency. As such, the long-term goals applied for 2022-23 accountability are the same as the original 2022 goals.

2022-23 Math long-term goals

These goals were originally established in the 2019 CSP based on baseline data from the PARCC assessment. New Mexico has since transitioned from the PARCC assessment and now requires the New Mexico Measures of Student Success & Achievement (NM‑MSSA) in grades 3-8 and the SAT School Day in high school. NMPED will establish new long-term goals and measures of interim progress in an amended CSP to be submitted to the U.S. Department of Education for 2023-24 accountability.

Progress in English Language Proficiency

Methodology:

The original long-term goals and measures of interim progress for English language proficiency were originally established with a target year of 2022. NMPED’s approved 2021-22 COVID addendum for the consolidated state plan established that long-term goals would shift forward by two years for academic achievement, graduation rate, and English language proficiency. As such, the long-term goals applied for 2022-23 accountability are the same as the original 2022 goals.

English language proficiency long-term goals for 2022-23

Key Decisions & Rationale

Topic: PED Policy Change in EL Exit Criteria

Description of decision: In a March 2023 memorandum to schools, PED announced a change to the EL exit criteria based on partnership work conducted with Regional Educational Laboratory (REL) Southwest Report: Effects of Reclassifying English Learner Students on Student Achievement in New Mexico, which replaced the State Policy EL Identification, ELP Placement, and Exit Criteria Memorandum dated April 24, 2017. The NMPED collaborated with the REL Southwest who analyzed student-level data for SYs 2017-2018 and 2018-2019 for ACCESS for ELLs, English language arts, and mathematics assessments. On average, students in the study sample that scored 4.7 or higher performed above the statewide grade-level average in English language arts and mathematics.

Date of decision: March 16, 2023

Other information: EL policy is managed by the PED Language & Culture Division. This updated EL exit criteria will be reflected in NM’s 2024 Consolidated State Plan Addendum.

Graduation Rate

Methodology:

The original long-term goals and measures of interim progress for graduation rates were established with a target year of 2022. NMPED’s approved 2021-22 COVID addendum² for the consolidated state plan established that long-term goals would shift forward by two years (due to the pandemic) for academic achievement, graduation rate, and English language proficiency. As such, the long-term goals applied for 2022-23 accountability are the same as the original 2022 goals.

In addition to the required 4-year adjusted cohort graduation rate goals, NMPED set long-term goals (terminating in 2021-22 and thus shifted forward to 2022-23) and measures of interim progress for the 5-year and 6-year adjusted cohort graduation rates.

4-year adjusted cohort 2022-23 graduation rate target by student group

5-year adjusted cohort 2022-23 graduation rate target by student group

6-year adjusted cohort 2022-23 graduation rate target by student group

Accountability System Indicators

Accountability Framework

The framework for New Mexico’s ESSA accountability system recognizes that school performance should be assessed within five overarching categories aligned to ESSA requirements for meaningfully differentiating schools: 1) academic achievement, 2) academic progress, 3) English language proficiency, 4) graduation rate, and 5) indicators of school quality and success. These categories are fulfilled across twelve indicators, listed in the table below, which also illustrates that all twelve indicators apply to high schools and seven indicators apply to elementary and middle schools.

Each indicator is scored based on student outcomes - for all students and each major student group - in that area for all accountable schools. The points for each indicator reflect the weights in the table below (again, separate for elementary/middle schools and high schools) and are aggregated into an overall ESSA score. The weights were established based on stakeholder input and ESSA requirements.

Both elementary/middle schools and high schools can earn up to 100 points. Schools serving grades K-2 can earn up to 55 points. The same weights apply when calculating overall ESSA scores for all students in the school and for each student group (subgroup) that meets minimum n-size requirements to receive points and designations.

A student group - either all students or any subgroup - must have at least 20 students that meet enrollment criteria for inclusion in a given indicator calculation (i.e., for points to be assigned to that indicator for that group). If a school does not have data for a minimum of 20 students in a given indicator, the points for that indicator are removed from the calculation for the overall ESSA score, and the school or student group’s score is calculated using a rescaled number of possible points.

Overall ESSA scores are used to assign school designations as described in more detail below.

The indicators above are included in a school’s ESSA overall score calculations based on the school’s grade configuration, with three options: K2 only schools (i.e., no tested grades); elementary/middle schools; and 3) high schools. The table below shows which indicators apply for these grade configurations.

For example, an elementary school might receive 20 points for math proficiency, 25 for reading proficiency, 6 for math growth, 4 for reading growth, 2 for English learner progress, 8 for science proficiency, and 10 for regular attendance to receive a total score of 75 points out of 100. A high school with the same rates for each indicator would receive 12 points for math proficiency, 15 for reading proficiency, 3 for math growth, 2 for reading growth, 1 for English learner progress, 4 for science proficiency, 10 for regular attendance, and could receive up to 35 additional points for 4 year, 5 year, and 6 year graduation, graduation growth, and college and career readiness.

Indicators included in total point calculations, by school type

*Including students in 9th or 10th grades with Assessment Bureau-approved uptest waivers.

In order to receive an overall score, a group must have scores (i.e., must meet minimum n-size of 20) for at least two indicators, including at least one of these: ELA achievement, math achievement, 4-year graduation rate, 5-year graduation rate, or 6-year graduation rate.

Student groups

Each measure is scored and reported when minimum n-size is met for the following student groups:

• All Students
• Race/Ethnicity (Asian/Pacific Islander, Black, Hispanic, Native American, White)
• Economically Disadvantaged (eligible for Free/Reduced Priced Lunch Program)
• English Learners
• Students with Disabilities

Description of decision: Student Group Labels AREA staff and consultants reviewed terminology used to describe student groups for consistency and alignment with state perspectives and priorities. As a result, some group labels are being updated as of 2022-23 accountability reporting.

Date of decision: February 22, 2024

Other information: Note that cross-agency discussion and action planning are necessary to ensure that terminology is used consistently across teams, webpages, documents, and reports/dashboards.

Description of decision: Multiracial student group

A multiracial student group was not included in New Mexico’s 2019 CSP. NMPED is considering including this group in the 2024 CSP amendment, but the group is not referenced in this technical manual.

Date of decision: March 2024

Minimum n-size for Calculating Points and Designations

New Mexico uses a minimum n-size of 20 throughout its accountability system. This includes calculations of indicator outcomes and overall ESSA points, and resulting designations of Comprehensive Support and Improvement, Additional Targeted Support and Improvement, and Targeted Support and Improvement.

New Mexico’s minimum n-size of 20 strikes a balance between the need for statistical power and stability, ensuring the reliability of accountability identifications and setting a threshold that is low enough to detect smaller subgroups in many of the state’s smaller districts and schools. This enables us to include as many schools as possible in subgroup and identification decisions. The compromise between the competing goals of including as many student groups as possible in the accountability system and greater statistical reliability is to maintain the minimum number of students at 20.

School Designations

New Mexico assigns designations for support in categories aligned with ESSA requirements:

• Targeted Support and Improvement
• Additional Targeted Support and Improvement
• Comprehensive Support and Improvement escalated from Additional Targeted Support and Improvement
• Comprehensive Support and Improvement for low graduation rate
• Comprehensive Support and Improvement for overall low performance
• More Rigorous Interventions escalated from Comprehensive Support and Improvement

Additionally, New Mexico has two state-established designations: Spotlight Schools and Designations of Excellence.   The designations and their timeline in relation to the 2022-23 tech manual are presented in the figure below (linked from here):

Designations of Excellence

Methodology:

The designation of Excellence is assigned to schools scoring above the 90th percentile on any indicator.   Business Rules:.

What schools are included in the calculations for this designation?

Since the calculation is per indicator, all schools with ESSA indicator scores are included in ranking for this designation.

What if a school’s ESSA overall score qualifies for Designations of Excellence, but the school also qualifies for CSI or MRI, or for TSI or ATSI based on student group outcomes?

ESSA designations of CSI, MRI, TSI, and ATSI are not mutually exclusive with Designations of Excellence; a school with any of these designations may receive a Designation of Excellence if it has an indicator score above the 90th percentile for that indicator.

Spotlight Schools

Methodology:

Spotlight Schools are those with ESSA overall scores above 75% of schools statewide.

Business Rules:

What schools are included in the calculations for this designation?

All schools with ESSA overall scores are included in ranking for this designation.

What if a school with an ESSA overall score qualifies as a Spotlight School, but the school also qualifies for CSI or MRI based on school wide outcomes, or for TSI or ATSI based on student group outcomes?

ESSA designations of CSI, MRI, TSI, and ATSI are mutually exclusive with the Spotlight designation. A school with a CSI, MRI, TSI, or ATSI designation may not be identified as a Spotlight School.

Comprehensive Support and Improvement - low performance

Title I schools are designated on a three year cycle as CSI for performance if their ESSA overall score is in the bottom 5% of Title I schools statewide. The 5th percentile threshold is set separately for high schools and non-high schools and involves ranking the overall scores for Title I schools in each category.

Business Rules:

What schools are included in the calculations for this designation?

Only Title I schools are identified as CSI for performance. For the 2022-23 designations, schools already identified as CSI for Graduation or as More Rigorous Interventions (MRI) were removed from the ranking when determining the 5th percentile threshold for CSI designations.

What was the ESSA Overall threshold score for CSI designation in 2022-23 (reports released in 2023-24)?

There are separate CSI thresholds for elementary/middle and high school.

Comprehensive Support and Improvement - low graduation rate

Methodology:

Schools are identified for CSI-low graduation rate if the average of their 4-year cohort graduation rate across three consecutive years (i.e., 2019-20, 2020-21, and 2021-22) is below 66.67%.

Business Rules:

What schools are included in calculations to designate CSI-low graduation rate?

Any school with a 4-year adjusted cohort graduation rate for the most recent three years is included in this determination. Note that, for the 2022-23 reporting year, the minimum n-size of 20 students applies to the most recent year’s 4-year adjusted cohort graduation rate (i.e., the 2021-22 rate), but not prior year’s rates.

What years of data are included in calculations for this determination?

The CSI-low grad rate determination is based on three years of data. However, graduation rate data is lagged by one year, so if the current accountability year = T, the graduation rate years included are T-1, T-2, and T-3. For 2022-23, this means that the relevant graduation rate years are 2021-22, 2020-21, and 2019-20.

Comprehensive Support and Improvement - more rigorous interventions

Methodology:

New Mexico first identified schools that qualified for more rigorous interventions beginning in 2018-2019 based on the historical underperformance of all students. (These were previously Priority Schools.) The next cohort eligible for more rigorous interventions was the group identified in 2022-23, based on 2021-22 assessment data.

Business Rules:

What schools are eligible for MRI designation?

The 2018-2019 CSI-MRI cohort included those schools that were previously identified as Priority Schools under ESEA flexibility prior to the passage of ESSA.

Only schools previously identified for CSI for low performance or low graduation rate in 2018-19 that did not meet exit criteria in 2021-22 are eligible for MRI, which is not based on a statewide ranking of ESSA overall scores.

Comprehensive Support and Improvement - ATSI conversion

Methodology:

Schools identified as ATSI in 2018-19 are not eligible for escalation of support into CSI until 2023-2024 (using 2022-2023 data) if they do not meet exit criteria in 2022-23.

Schools identified as ATSI in 2022-23 (using 2021-22 data) are eligible for escalation of support into CSI in 2023-24 (using 2022-23 data) if they do not meet exit criteria in 2022-23.

This conversion applies only to Title I ATSI schools.

Business Rules:

What schools are eligible for ATSI conversion to CSI?

A school must be both an ATSI school identified in the prior cohort identification year and a Title I school to be eligible for ATSI conversion to CSI. Schools satisfying these criteria that also did not qualify to exit in 2022-23 are designated as CSI - ATSI conversion.

Targeted Support and Improvement

Methodology:

NMPED defines “consistently underperforming” subgroups of students as those groups with ESSA overall scores at or below the bottom 5th percentile of Title I schools when compared to the referent group of that specific subgroup (e.g., schools serving the lowest 5th percentile of economically disadvantaged students among all economically disadvantaged students in the state) for the three most recent years. For 2022-23 TSI designations, two years of data were used, i.e., a school was designated as TSI if any student group’s performance was at or below the 5th percentile threshold in both 2021-22 and 2022-23 when compared to that referent group’s overall performance.

For methodology, the following steps were applied in 2022-23.

1. Rank the 2021-22 ESSA overall scores - by subgroup - separately for all high schools and non-high schools.
2. Determine the 2021-22 5th percentile cut for each statewide ranking of subgroup ESSA overall scores. This is the TSI threshold for each group.
3. Rank 2022-23 ESSA overall scores - by subgroup - separately for all high schools and non-high schools.
4. Determine the 5th percentile cut for each statewide ranking of subgroup ESSA overall scores. This is the TSI threshold score for each group.
5. Any school with at least one student group that had an ESSA overall score below the statewide 5th percentile threshold score for that group in both 2021-22 and 2022-23 was designated as TSI.

As with CSI schools, separate identification thresholds were applied for high schools and non-high schools.

Business Rules:

What groups are included in calculations to determine TSI designations?

Only student groups with ESSA overall scores are included in TSI calculations. As a reminder, in order to receive an overall score, a group must have scores (i.e., must meet minimum n-size of 20) for at least two indicators, including at least one of these: ELA achievement, math achievement, 4-year graduation rate, 5-year graduation rate, or 6-year graduation rate.

What schools are included in calculations to determine TSI designations?

Any school - Title I or non-Title I - may be identified for TSI. As such, all student group overall ESSA scores are compared to their within-group performance (e.g., students with disabilities groups below the 5th percentile of performance when compared to the performance of all groups of students with disabilities in the state), regardless of whether the group is in a Title I or non-Title I school.

What were the 2022-23 overall ESSA score thresholds for TSI identifications?

TSI thresholds are established for each student group, separately for elementary or middle schools and high schools. A group needs to fall below the 5th percentile of its referent group for two consecutive years: 2021-22 and 2022-23.

*There were too few high schools with Overall ESSA Scores for the Asian and Black student group to produce a 5th percentile threshold score.

*There were nine schools that received zero points for the black student group.

Why don’t I see exit criteria for TSI designations in the exit criteria section below?

TSI schools are identified annually. As such, there are no exit criteria for the TSI Designation. The way to “exit” is not to be identified in the next round of designations.

Additional Targeted Support and Improvement

Methodology:

Schools that satisfy both of the following criteria are designated as ATSI for 2022-23 designations:

1. The school qualifies for TSI designation based on within-group performance, e.g., schools with SWDs below the 5th percentile of performance when compared to all schools with SWDs in the state for the two most recent years.
2. The ESSA overall score for the student group(s) that qualified for TSI is also in the bottom 5% statewide compared to the CSI threshold, i.e., all ESSA scores for schools statewide.

For methodology, the following steps were applied for 2022-23 designations.

1. Start with the list of schools that qualify for TSI designation in 2022-23.
2. From that list, determine which student group scores that qualified for TSI are also below the 5th percentile statewide CSI threshold score for 2022-23.
3. If the TSI student group’s score is also below the statewide CSI threshold score, the school is designated at ATSI. 

Re-identifying early exit ATSI schools

Some schools identified as TSI in 2018-19 were exited from that status prematurely in 2021-22. The following criteria were used to determine if an early-exited ATSI school should remain exited, be re-identified, or escalated into CSI. Designations for 2022-23 (released in 2023-24) were adjusted based on whether a school meets these criteria:

1. Increase in ESSA overall score ranking when comparing the 2018-19 subgroup rank with the 2022-23 subgroup rank.
2. Demonstrate positive change in language arts and math proficiency growth from 2021-22 to 2022-23.

Business Rules:

What years of data are included in ATSI calculations?

For 2022-23 ATSI designations, because ATSI is contingent upon TSI designation, two years of data were used. A school was designated as TSI if its performance was at or below the 5th percentile of within-group performance in both 2021-22 and 2022-23.

What groups are included in calculations to determine ATSI designations?

Only student groups with ESSA overall scores in both 2021-22 and 2022-23 are included in ATSI calculations. As a reminder, in order to receive an overall score, a group must have scores (i.e., must meet minimum n-size of 20) for at least two indicators, including at least one of these: ELA achievement, math achievement, 4-year graduation rate, 5-year graduation rate, or 6-year graduation rate.

What was the ESSA Overall threshold score for CSI designation in 2022-23 (reports released in 2023-24)?

There are separate CSI thresholds for elementary/middle and high school.

What schools are included in calculations to determine ATSI designations?

Any schools - Title I or non-Title I - may be identified for ATSI. As such, all schools with at least one student group that has an ESSA overall score are included in the rankings that may determine ATSI designation.

Exit Criteria for School Designations

NM exit criteria in 2022-23 generally require a school to demonstrate improvement in performance relative to other schools in the state and compared to past performance.

Exit Criteria: Comprehensive Support and Improvement - low performance

Methodology:

Schools identified for Comprehensive Support and Improvement were eligible to exit if they satisfied the following criteria:

1. The relative ranking of the school’s overall ESSA score between school years 2018-19 and 2022-23 improved to exceed the 5th percentile.
2. The proficiency rate for the school between school years 2021-22 and 2022-23 improved.

Business Rules:

What CSI - low performance identification cohort (ID year) is eligible to exit in 2023-24 (based on 2022-23 data)?

CSI Low Performance 2018-2019 Cohort: Schools that received services beginning in SY 2018-2019 are eligible to exit using the average of the three most recent years of available accountability data (i.e., SY 2022-2023, 2021-2022, and 2020-2021).

CSI Low Performance 2022-2023 Cohort: For schools that received services beginning in SY 2022-2023, schools are eligible to exit using the average of the three most recent years of available accountability data.

Exit Criteria: Comprehensive Support and Improvement - low graduation rate

Methodology:

Schools identified for CSI based on low graduation rate are eligible to exit if their three-year average 4-year adjusted cohort graduation rate is above 66.67%.

Business Rules:

What graduation years are used to determine the three-year average graduation rate?

Graduation rates used for accountability purposes are lagged by one year. For the 2022-23 reporting year, the average of the 2022-21, 2020-21, and 2019-20 graduation years is used (i.e., the most recent prior three years).

What CSI - graduation rate identification cohort (ID year) is eligible to exit in 2022-23?

CSI Graduation Rate 2018-2019 Cohort: For schools that received services beginning in SY 2018-2019, schools are eligible to exit using the average of the three most recent years of available graduation rate (i.e., SY 2022-2023, 2021-2022, and 2020-2021).

Exit Criteria: Comprehensive Support and Improvement - more rigorous interventions

Methodology:

Schools identified for more rigorous interventions (MRI) have the same exit criteria as schools identified for CSI - low performance.

Business Rules:

What CSI - more rigorous interventions identification cohort (ID year) is eligible to exit in 2023-24 based on 2022-23 data (SY 2018-19 cohort)?

While there were initially schools identified for more rigorous interventions in the first cohort of ESSA identifications, this status was reverted to CSI-low performance in 2018-19. As such, there are no CSI - more rigorous interventions to consider for exit in 2023-24.

Exit Criteria: Additional Targeted Support and Improvement

Methodology:

Schools identified as ATSI may exit by satisfying the following two criteria:

1. The relative ranking of the school’s assessment proficiency rate between school years 2018-19 and 2022-23 increased.
2. The proficiency rate for the school between school years 2021-22 and 2022-23 improved.

Business Rules:

What ATSI identification cohort(s) are eligible to exit in 2023-24 based on 2022-23 data (SY 2018-19 cohort)?

The 2018-2019 ATSI Cohort received services beginning in SY 2018-2019, and schools in this cohort are eligible to exit in 2023-24 based on 2022-23 data.