The Program for International Student Assessment, better known in the education world as PISA, has come under fierce criticism in recent years. PISA is given every three years to 15-year-olds around the world in reading, math and science. PISA is influential in that many governments look at the scores as pronouncements on their education systems, though critics say the tests are flawed and the results should not be taken seriously. Last year, dozens of researchers and academics from around the world wrote an open letter to Andreas Schleicher, director of the Program of International Student Assessment, urging him to suspend administration of PISA until a new exam can be created. (You can read about that here.)
The following post looks at the controversy around PISA and asks whether it should be saved. The answer may surprise you. It was written by Pasi Sahlberg (@pasi_sahlberg) is visiting professor at the Harvard Graduate School of Education and the author of Finnish Lessons 2.0.Andy Hargreaves (@HargreavesBC) is professor in the Lynch School of Education and the Carroll School of Management at Boston College and the coauthor of “Uplifting Leadership.”
By Pasi Sahlberg and Andy Hargreaves
What if three-quarters of American school students voluntarily attended daily after-school classes to boost their knowledge of mathematics, literacy, and science? On top of that, imagine if American students were to spend more than two hours a day on homework related to these subjects. This is what their peers do in envied Shanghai, Singapore and South Korea. The United States is banging its head against a Great Wall when it aims to beat these countries in international student assessments.
Since the year 2000, the most developed countries have been able to compare knowledge and skills of their 15-year-old students in reading, mathematics and science in a periodic PISA (Programme for International Student Assessment) test. The test has had a towering influence on national educational policies. Many countries whose performance has fallen have gone into ‘PISA shock,’ and shifted direction dramatically to raise their standings and their standards.
As the influence of this international assessment administered by the Organization for Economic Cooperation and Development (OECD) has increased, so have the skeptical voices denouncing the nature and consequences of this test. Last year, over a hundred academics around the world called for a moratorium on PISA testing. The tower of PISA is leaning. But unlike some of our colleagues, we will not rejoice if it finally falls.
Critics have raised three concerns. First, they say, PISA ranking of countries has negative consequences for school systems. It provokes overreliance on standardized tests and narrowing of learning to what is easily measured. Second, OECD and therefore PISA are criticized for being biased in favor of economic interests in public education – especially alliances with global for-profit companies looking for business opportunities by selling PISA-like instruments for practice in schools. Third, some experts claim there are major technical flaws with the items that make up PISA tests, with how the tests are administered, with how samples of students are determined in some countries (especially in Asia) and with the (mis)use of statistical techniques to create country rankings.
We agree with these criticisms about PISA and its uses. In educational and moral terms, the tower of PISA is not only leaning, but in danger of toppling over completely. But PISA has also done many good things for students, schools and societies around the world and these should not only be acknowledged; they should also be protected.
Just think for a moment what would global education look like if PISA had never been launched? There would be, as there was in the 1990s, a number of countries that mistakenly believed their education systems are the best in the world and should set the direction for other nations. Were it not for the fact that these weaker performing countries that include the United States and England have not been successful in PISA, the worldwide pressures for more market competition between schools, less university-based training for teachers, and more standardization of the curriculum, would have had a far easier ride.
For instance, Sweden’s catastrophic decline in educational performance on PISA since its introduction of for-profit free schools has taken it away from the high performance of other Nordic countries and towards England and the United States, with declining equity and stagnated student achievement. This galvanized public awareness about, and political opposition to, these reforms that had a huge influence on the recent change of government in Sweden. This would not have been possible without PISA.
At the same time, were it not for their repeated high performance on PISA, nobody would have heard of or been able to celebrate the social and educational characteristics of Finland and Canada – not us, and not several of the academics who signed the petition opposing PISA. Much of the global campaign to improve the status and quality of teaching turns on the evidence of how stringently Finland selects and trains its teachers, how much it values them politically and throughout the society, and how it expects them to collaborate by creating most of the curriculum themselves in collaboration with their colleagues. Neither Finland nor Canada test their students relentlessly, subject after subject and grade after grade. Nor do they force schools to compete against one another for enrolment, students’ test scores, and, ultimately, financial resources.
OECD has also done far more than emphasize differences in overall performance. It has become a strong advocate of equity in education by reminding policymakers that the highest-performing education systems combine quality with equity. It has put equity high up on the reform agenda. Without the data that PISA has generated over the years, calls for enhanced equity would not be part of the education policy conversation in the countries that have suffered from inequitable education systems, including the U.S.
These things mean that PISA deserves preservation, not termination. But not enough is being done to discourage countries from using hasty and desperate measures to move towards the top of the PISA rankings. Instead, packages of PISA tests are now being sold to school districts and whole countries, pushing teachers and schools to teach to the PISA test that does not enhance equity or quality of education or the integrity and credibility of PISA. As an unintended consequence, schools that are successful in OECD’s PISA-based Test for Schools are celebrated as being like like Shanghai or Finland, and poorer performing schools are shamed as looking like Turkey or Chile.
Whether the tower of PISA will stand tall as a helpful beacon for policy-makers or whether it will fall over depends also on how much the governments and global education community can regard PISA as a trustworthy instrument that is immune to commercial and ideological interests. Since the beginning of PISA, OECD has placed its trust in the hands of international consortia of professional organizations that have controlled how PISA tests are designed, how data are collected and how results are analyzed.
In December 2013, however, OECD accredited textbook and testing giant McGraw-Hill Education as an exclusive administrator of PISA-based Test for Schools in the United States. One year later it awarded Pearson – another global education corporation – the contract to define what will be measured in PISA 2018. By admitting private corporations to design these tests and to have access to global data about students and teachers, PISA is closer to crossing the line that separates commercial interest for expanded markets on the one hand, and neutral and trustworthy independent measurements of the health of education systems on the other.
In addition, more and more mainly Asian countries and cities are being added to the top of the PISA tower by questionable methods – placing downward pressure on the previous highest performers to start copying these new Asian exemplars, even though they are generally weak on equity and backward in their approaches to special education. The tower that once stood straight up is becoming ideologically inclined to a dangerous degree.
Last, when countries learn from one other, PISA and other student achievement measures should not be the only ones that count. Other indicators like measures of child wellbeing or human rights where a number of countries outside Asia, the United States and the United Kingdom do especially well, also have to be considered.
What PISA shows to the United States is that its current course of education policies that rely on competition, standardization, testing and privatization of public education is a wrong way. Our goal should not be to take PISA down, but to get it or something like it upright again, so that by using a range of criteria, and by using them in a fair and transparent way, we can identify and learn from the true high performers who are strong on equity as well as excellence, and on human development as well as tested achievement.