We asked state representative Kevin Verville to provide an in-depth analysis of New Hampshire’s latest statewide assessment results. He is a two-term representative serving the communities of Candia, Deerfield, and Nottingham. He has served on multiple committees in his town, including the municipal budget committee, trustee of trust funds, open space committee, and safety services committee of Deerfield. He is a former high-school science teacher, with two children in his local public school.
Each fall there is a data dump of the previous school year’s assessment results. Most people see the headlines, and are surprised or shocked to see results like those from the 2018-2019 New Hampshire Statewide Assessment System (NHSAS) as shown in Figures 1, 2, & 3.
And why shouldn’t we be?! Only 60% in English language arts (ELA), and fewer than half of New Hampshire public school students are proficient mathematics and science.1 The data and analysis in this report will be for grades four through eight. High school data exists, but will not be considered here.
We should consider the word “proficient.” Merriam-Webster defines proficient as “well advanced in art, occupation, or branch of knowledge.” Lexico, powered by Oxford, defines proficient as “competent or skilled in doing or using something.” Lexico defines competent as “Having the necessary ability, knowledge, or skill to do something successfully.”
It is also worth noting that all the data cited, and used in this report is from government sources. I have merely put the data into a uniform graphical format for ease of comparison. While many will rightfully criticize the over reliance on standardized tests as a complete measure of academic performance, it is a reliable metric to measure and evaluate performance over time of groups, and cohorts. Let us be clear about what these assessments are attempting to measure. These assessments are attempting to measure reading, writing, mathematics, and science in grade school students. The basics are the basics. Standards, curriculum, and methods may, and do change, and there needs to be some metric to determine the effectiveness of those changes. To posit that we cannot measure academic performance of the basics by assessment does not pass the smell test. These are the assessments that both the federal government, and the New Hampshire Department of Education has deemed the metrics to be used to measure academic performance of New Hampshire public school students.
Only a fraction of the data available from these sources is considered here, and those with interests in particular groups, or categories of students are encouraged to seek out that data for consideration and analysis. This report is an extension of data analysis I have conducted using my local pre-k through 8th grade public school. Further, there are many other sources of similar types of data not included in this report. Often the data is difficult to locate on websites, and then must be manually entered into a spreadsheet for analysis and comparison. This report is but one fruit of countless hours I have spent toiling in the data mines in an effort to create products that allow more people to understand what is going on in our public schools.
The National Assessment of Educational Progress (NAEP)2, aka “The Nation’s Report Card” recently released the 2019 results, which are also for the 2018-2019 school year. NAEP assessments are administered across the country every two years on odd years in grades four, eight, and twelve in a variety of subjects. NAEP scores in both Reading, and Mathematics are continuing their recent downward trend at both the national, and New Hampshire levels.
Figures 4 & 5 shows the trend in raw scores for both the nation and New Hampshire since 2003 for 4th and 8th grade reading, respectively. The Common Core State Standards© were recommended to New Hampshire public school districts by the New Hampshire Department of Education in 2010. This was a departure from the previous set of recommended standards known as “Grade Level Equivalents,” or GLE’s. The good news is that New Hampshire public school students outperform the nation as a whole. The bad news is that the average score for both the nation and NH does not even meet the level of proficient as defined by NAEP. Further, the average NH score is declining at a faster rate than the national average. These are very concerning facts.
This average score decline over the past four years does not tell the entire story. Unfortunately, I could only find national level data for Figures 6 & 7. Figures 6 & 7 shows the 10th, 25th, 50th, 75th, and 100th average percentile score since 2003. What is most telling about this data, that is not told by the previous data that the majority of decline in the average score is a result of dramatic declines in average score for the 10th & 25th, and to a lesser degree the 50th percentile students. That is to say our lowest performing public school students are doing far worse than our higher performing students, who have declined in performance to a much lesser degree.
Now let us consider the same data for mathematics. Figures 8 & 9 shows the trend in raw scores for both the nation and New Hampshire since 2003 for 4th and 8th grade mathematics, respectively. The good news is that New Hampshire public school students outperform the nation as a whole. The bad news is that the average score for both the nation and NH does not even meet the level of proficient. Further, the average NH score is declining at a faster rate than the national average. While the data for the 4th grade is not as dire (flat over the past two years) as the data for the 8th grade, one would be hard pressed to be pleased with the 4th grade results. These are very concerning facts.
Figures 10 & 11 shows the 10th, 25th, 50th, 75th, and 100th average percentile score since 2003. While the 4th grade percentile scores were little changed, the majority of the 8th grade decline in the average score is a result of dramatic declines in average score for the 10th & 25th, and to a lesser degree the 50th percentile students. Once again, that is to say our lowest performing public school students are doing far worse than our higher performing students, who have declined in performance to a much lesser degree.
There is other New Hampshire specific data that we may consider in our analysis of New Hampshire public education. Before we go on we should mention a few important reminders. For the most part we are not concerned with the absolute values of assessment results when comparing results of different assessments. While they may be indicative, they are more qualitative than quantitative when used comparatively. Instead, a more appropriate use of the data is to look for trends over time to evaluate performance. Unfortunately, that is not reliably possible for data generated by the New Hampshire Department of Education over the past five years. We have New England Common Assessment Program (NECAP) data from 2006 through 2014, or nine years’ worth of data. The NECAP was well respected and relatively non-controversial. In 2015 the NHDoE switched to the Smarter Balance Assessment Consortium© (SBAC) and continued to use it until 2017, providing us with only three years of data points. The SBAC was a controversial pseudo-national assessment that few liked, and even fewer had any faith in the results provided. Another interesting point, that we will see shorty, is that the percentages of students below proficiency, and students at or above proficient apparently inverted, making comparisons with the previous NECAP results impossible. To add more mud to the water the NHDoE switched statewide assessment vehicle again in 2018 to the “locally developed” New Hampshire Statewide Assessment System (NHSAS), which it continues to use. “Locally developed” is in quotation marks because many believe that this is simply a local version of the SBAC. Currently we only have two years, or two data points for the NHSAS. For the following figures the data (lines) with breaks between 2014-2015, and 2017-2018, are NECAP, SBAC, and NHSAS, respectively. The continuous data that ranges from 2005-2019 is NAEP data. By overlaying the data from these sources we can more reliably look for trends in academic performance of NH public school students. We will consider three sets of graphs for ELA, and mathematics, respectively. We will follow the previous scheme of looking at the most general information, and drill down in successive graphs. After our consideration of ELA and mathematics, we will take a quick look at science results.
Figures 12 & 13 are considering both national and state level NAEP results, along with NECAP, SBAC, and NHSAS results. The national NAEP data are the raw scores divided by three to allow the data to fit on a zero to 100 scale, as percentage of students at or above proficient is not readily available. However, trends in raw score are indicative of performance over time. All other data represents the percentage of students deemed at or above proficient by the various assessment vehicles. Again, we are not concerned with absolute values, but rather trends over time. All data sets indicate that we are in a period of declining academic performance in both ELA and mathematics and the situation appears more dire with respect to ELA.
Figures 14 & 15 add in the data for the percentage of students below proficiency. The sum of the two data sets must equal 100, and therefore the two data sets appear to be mirror images. It is one thing to see a decline in percentage of students at or above proficient. It is another to also see the magnitude of increase in the percentage of students below proficient. It is certainly difficult to look at this data and believe that the New Hampshire public school system is moving in the correct direction with respect to ELA and mathematics education.
However, the previous four graphs only tell half of the story. Who is performing better, and who is performing worse is a valid, and responsible question to ask. Figures 16 & 17 will break the previous data set out into four categories. Please note the vertical scale change. These two graphs have a maximum Y value of 60%, as opposed to the 100% scale used on all other graphs. The change in scale was done intentionally to aid in the ease following the various data lines that are spaced very close together, and are much more difficult to resolve using a 100% scale. The NHDoE uses four possible scores: Level 1 – Does NOT meet Proficiency, Level 2 – Approaching Proficiency, Level 3 Proficient, and Level 4 – Exceeds Proficiency. I have color coded the various levels as noted above. It is clear from Figures 16 & 17 that declines in percentage of students at or above proficient are result of some stark increases in the percentage of students in the Level 1 population! This is combined with moderate increases in Level 2. The bulk of students making up the increase in Levels 1 & 2 are students that were previously at Level 3 – Proficient.
To say it another way, our declining scores are not primarily driven by shifts in the “approaching proficient” and “proficient” populations, but rather by stark increases in the “does not meet proficiency” category.
Let us now turn our attention to assessment results for science. SBAC did not have a science component. For that reason, NH used the NECAP for science assessment through 2017, and switch to the NHSAS for the last two years. Figures 18 & 19 show proficiency levels in science for New Hampshire students. The most remarkable thing to note is the incredibly low percentage of New Hampshire students proficient in science. This is even more startling given the focus on STEM (science, technology, engineering, & mathematics) education over the past decade!
While the data presented is pretty grim it is not all bad news. First, New Hampshire does very well as compared to the nation as a whole. Second, this analysis only considered the data for the state and national level. Data is available at both the New Hampshire school district, and local school level, by grade, for grades four through eight. There are school districts, and schools that perform above the state average, and should be models for other districts and schools. Third, there are school districts, and local schools that are showing improvement year on year. Those school districts, and local schools should be recognized, and commended.
Please feel free to contact the author with any questions, or comments.
Written by:
Kevin Verville
NH State Representative
Rockingham 2
Kevin.verville@leg.state.nh.us
(603) 731-0654
References:
1 New Hampshire Statewide Assessment System data source: State of New Hampshire Department of Education – https://www.education.nh.gov/instruction/accountability/index.htm
2 National Assessment of Educational Progress (NAEP), aka “The Nation’s Report Card” https://www.nationsreportcard.gov/
