In a recent guest post, Oklahoma State University Provost Robert Sternberg argues against high-stakes assessments in colleges. Here is a response to that post from Roger Benjamin, president of the Council for Aid to Education, publisher of the Collegiate Learning Assessment.
Robert Sternberg’s piece is fully in keeping with the Super PAC’s tendency to unleash misinformation about their opponents in this heated presidential race. Sternberg follows the standard line of attack: Distort the position of the opponents, and then demolish that position. He rubbishes standardized assessments used in higher education, including the College Learning Assessment (CLA). He implies the only use for such tests is for nefarious--and unwanted--No Child Left Behind (NCLB) high-stakes testing and top-down accountability systems in higher education. He dismisses the CLA as a narrow test of something he calls “general learning,” which, in his view, will narrow the curriculum because instructors will have to teach to it. Sternberg provides his preferred list of attributes to assess, such as: academic disciplines, creativity, ethical behavior, portfolios, and persistence. These qualities, he argues, are far more relevant to the success of students than the content measured by the standardized tests. Moreover, he asserts that reliance on a single test score at the institutional level to evaluate student learning is ludicrous. The case is closed for Dr. Sternberg. Please read on for a different perspective.
Sternberg’s argument falls prey to two shibboleths about assessment in higher education. However, I begin by describing the measurement goals of the CLA and how the test differs from multiple-choice tests and bears no resemblance to Sternberg’s caricature.
Critical thinking, problem solving (both qualitative and quantitative in nature), and writing--the skills the CLA measures--are at the top of both faculty and employer surveys. They are also at the core of most major education reforms, such as the common core standards movement, and are regarded as necessary for student success in college and work. Secondly, the CLA is not a multiple choice test. CLA performance assessments are realistic problems that require constructed essays and have no “correct answer.” The goal of the CLA is to stimulate students’ abilities to “form original, concise thoughts”; which is parallel to the goals that Dr. Sternberg approves of. In fact, performance assessments are tests worth teaching to because there are no “right” answers as with multiple-choice tests. Moreover, performance assessments are as useful as instructional tools as they are for assessment. Through our performance academies, thousands of professors have been taught to develop their own performance assessments along with the curriculum and text materials. These same faculty members encourage their colleagues to share best practice ideas to improve teaching and learning at their home institutions. They also profit from sharing best practices across colleges, which brings me to Sternberg’s conflation of standardized tests with the potential evils of bad education policy.
Without appropriate standardized assessment comparisons between institutions, colleges remain isolated silos with no objective way to set their level of teaching and learning in a larger context. What evidence do institutions have to back up their claims for the education they offer? Opponents of institutional comparisons argue that assessments within an institution are more than sufficient. However, suppose one finds that 35 percent of an institution’s students report reading a book during the past two weeks that was not assigned by their professors. Is that percentage good, bad, or indifferent? We cannot know the answer unless we can confirm that this percentage was one of the highest or lowest reported by students at colleges similar to this one. One needs a comparative-based benchmark to frame this finding in order to give the specific finding meaning. This is true of all of the assessments that Dr. Sternberg approves of.
Finally, Sternberg joins those who argue that one-size-fits-all measures to compare institutions are inappropriate. On this point he is correct. No one measure can capture the complexity of a college or university. The scores on the CLA and other tests at the institutional level should be set in the context of multiple indicators collected and analyzed by faculty at the institution itself. Faculty and administrators might respond to the initial institutional-level assessment scores in steps, such as examining the factors that are correlated with the results and placing the performance tasks in the hands of the faculty so they can use them in their classrooms.
Comparative-based assessments should be designed to be objective characterizations of institution-level performance on student learning outcomes, offering evidence that is judged to be reliable and valid, and thought to be authentic by the faculty. Such assessments offer a powerful reality check for institution-based formative assessment, which is, at the least, as important as standardized tests. The testing organization should report assessment results for the institutions it tests back to them only. Otherwise, why would an institution, department, or program want to permit comparative-based testing? It should play a critical role in making formative assessment more systematic. Of course, the questions of what to report and who gets to report it under what review conditions are central issues being debated today. For colleges, the peer-review concept is the best place to start such conversations.
Since it is possible to learn from other institutions that are producing advances in teaching and learning, there is no intellectual argument for not doing so. The stakes are too high. Peer review anchors the continuous improvement system that provides progress in research. We may never achieve such clearly positive results in teaching and learning. However, we can build a strategy to move the subject of teaching and learning forward to a more evidence-based systematic subject in which verifiable best practices are continually adjusted or changed and improvement of student learning demonstrably occurs. See The Case for Generic Skills and Performance Assessment in the United States and International Settings for the rationale behind performance assessment, a review of the many independent studies of the CLA, and links and reports on numerous best-practice examples of institutions engaging in comparison between best practices of teaching and learning and improving their teaching and learning, which should be the principal reason for doing any assessment.
[Benjamin appends the following note:This response is based on a larger argument about assessment in higher education in my “The New Limits of Education Policy: Avoiding A Tragedy of the Commons,” (London, Edward Elgar, 2012). My response represents CAE and the CLA only.]