In 2011, the Oklahoma Legislature approved an A-F system of grading schools that was pioneered in Florida by former governor Jeb Bush. An independent examination of Oklahoma’s system concludes that the system is flawed and the grades given to schools are “misleading” — not only because of the way the state calculates the scores but also because of problems generally associated with using standardized test scores as measures of school performance.
The conclusions speak not only to the specifics of Oklahoma, but to similar state systems that have been adopted to emulate Florida’s. It is a new challenge to the school reform movement that is using standardized test scores to assess not only students but teachers and schools.
The report, done by researchers from the University of Oklahoma and Oklahoma State University, says:
Some problems with the A-F Report Card are unique to methods used by the Oklahoma State Department of Education (OSDE) to calculate student achievement, student growth, and school performance indices. Other problems are longstanding conceptual and methodological constraints associated with aggregated test scores as measures of school performance. Although achievement data are obviously important for assessing schools, an accountability grade based almost exclusively on test scores does not account for numerous critical factors that contribute to school performance.
Performance measurement and accountability systems in many sectors take a more balanced approach to assessment. In other enterprises, executives and managers are not expected to make strategic decisions based on outcome data alone. Healthcare, for example, has made great strides adopting scientific process and outcome measures for evaluation. Manufacturing forecasts future profitability using measures of customer satisfaction, demand, internal processes, and innovation and growth, relying on much more than past financial reports. It is now standard accounting practice to evaluate companies as much by their intangible resources as by their physical capital. Sole reliance on outcome indicators produces biased assessment and does not depict fairly or accurately how school leaders and teachers respond to the dynamic needs of students.
Accountability systems are only useful if their measures are credible and clear. Despite good intentions, the features of the Oklahoma A-F grading system produce school letter grades that are neither clear, nor comparable; their lack of clarity makes unjustified decisions about schools. Further, A-F grades are not productive for school improvement because they do not explain the how or why of low performance. Building on what has already been done, Oklahoma can and should move toward a more trustworthy and fair assessment system for holding schools accountable and embracing continuous, incremental improvement.
And there’s more:
All three components of the A-F System have statistical limitations that jeopardize their validity, reliability, and usefulness. Moreover, combining flawed indicators creates a misleading measure in the form of a single grade for each school.
The grading system was adopted with the intention of giving schools “incentives” to challenge children to raise their achievement, the Oklahoma Department of Education website says. It also says:
The new A-F report cards will make school performance clear in a transparent manner easily communicated to the public.
Among the recommendations made by the researchers is this:
The accountability system is important to our State and it is essential that it be credible, accurate, and supported by policy-makers, school professionals, and the people of Oklahoma.
Action: Take the time to enlist the services of assessment and evaluation experts who can objectively build an exemplary Oklahoma accountability system directed at incremental and continuous school improvement.