New Data on AP's Impact

By Jay Mathews
Washington Post Staff Writer
Friday, January 9, 2009; 6:27 AM

On one wall of my cubicle is a large chart extracted from Tom Luce and Lee Thompson's 2005 book "Do What Works: How Proven Practices Can Improve America's Public Schools." It shows that a study of 78,000 Texas students found college graduation rates much higher for those who, while in high school, took Advanced Placement exams -- but failed them -- than those who took no AP exams at all.

At this point, you may be saying, "Huh?" We AP wonks are an odd breed. We often cite statistics that make no sense to normal people. But I will try to explain this one, and why it was greeted with such excitement by AP teachers four years ago.

AP courses are given in nearly 40 subjects. They allow high school students to earn college credit, or at least skip college introductory courses, if they do well on the final exams. Many AP teachers argue that students' grades on the three-hour exams, given in most U.S. high schools every May, are not as important as taking the college-level course and exam and getting a taste of college trauma. Many of their students who flunk the AP exam still report, when they come back to visit after their freshman year of college, that the AP experience made it easier for them to adjust to fat college reading lists and long, analytical college exams. They may have failed the AP exam, but by taking it, and the course, they were better prepared for the load of stuff dumped on them in college. When they took the college introductory course in the subject that had been so difficult for them in high school AP, they did much better.

The Texas study showing that failed AP students were more likely to graduate from college than non-AP students was thus greeted as proof that the AP teachers' view on this issue was correct. But the researchers who had done the work cautioned against putting too much weight on it. There were too many variables to reach hard conclusions.

For instance, as statisticians always remind me, correlation is not the same thing as causation. Just because students who struggled with AP exams did better in college did not mean that the AP experience produced those good college results. Perhaps the sort of students who took AP were more hard-working, and that explained their success in college.

Now, however, a new study has arrived that I am sure will once again send a buzz through AP Land, since it endorses with greater force the view that getting kicked around by an AP course and exam is good for you. The study, published on the College Board Web site, is "College Outcomes Comparisons by AP and Non-AP High School Experiences" by Linda Hargrove, Donn Godin and Barbara Dodd. Hargrove and Godin are researchers who work for the Texas Higher Education Coordinating Board. Dodd is a professor of educational psychology at the University of Texas at Austin. They have published several articles on the effects of AP, using Texas' state database, one of very few that allow scholars to follow the progress of students from public schools into state colleges. This new study uses data from five graduating high school classes, 1998 to 2002, that total 302,969 students.

For AP teachers and others who argue over the effect of AP and other college-level high school courses such as International Baccalaureate, this somewhat convoluted sentence in the summary section of the report will draw the most attention: "The preponderance of academic subject-specific analyses (AP English Language, AP English Literature, Calculus, Biology and History) showed that students, matched on SAT ranked score intervals (measuring ability or college readiness) and FRPL [free and reduced price lunch] status (family income measure), who took both an AP course and exam in the corresponding AP subjects significantly outperformed AP course only, dual enrollment only (excluded for AP Biology), and other courses students on all the college outcomes, even after gender and ethnicity were taken into account."

Whew. I also had to read it three times before I got it. Two footnotes might help. (1) Dual enrollment courses are classes offered to high school students by local colleges, who either invite the students to their campuses or train or send instructors to the high schools to teach courses that can earn college credit. (2) The college outcomes measured in this study included students' first- and fourth-year college grade point averages, the number of college credit hours earned and the percentage of students who graduated from college in four years.

What was most interesting about this study was that it attempted to compare students with similar academic characteristics so that it would be less likely that the results could be explained by the notion that students who took AP courses and exams were just naturally better students than those who did not take AP. As the long sentence above says, the study compared the college outcomes of AP students who had low SAT scores with the outcomes of non-AP students with low SAT scores, and found the AP students did better. Similarly, AP students from low-income families, who often struggle in college, did better than non-AP students from low-income families.

AP exams are graded on a five-point scale. According to College Board vice president and AP director Trevor Packer, a grade of 5 is equivalent to a college A, a 4 is equivalent to a high B, a 3 is a high C, a 2 is a D and a 1 is an F. The College Board gives AP exams to college students who have just completed a college introductory course in the same subject and compares their college grades to what they get on the AP. Some selective colleges give credit only for scores of 4 or 5, but the vast majority of incoming college students, who attend less selective state schools, can get credit for a 3. Often when talking about grades, we AP wonks say 3 to 5 are passing grades and below 3 are failing grades.

On pages 35 and 36 of their report, the Texas researchers revealed what was for me the most interesting of their many new disclosures. They show that even students who only get a 2 on their AP exams after taking the AP course have significantly better college outcomes than non-AP students. Students who get 1s on the exam do not do better than non-AP students, but as I have often heard AP teachers say, they have no chance to build those students up to a 2 or a 3 unless they are allowed in their courses.

These are complicated issues. This study is not the last word. Critics of AP may say that these researchers' work is tainted by the fact that the College Board, which owns the AP program, paid them for their study. But there is no question they are reputable, independent scholars, and their data is there for all to see.

It is extremely rare to have such a study with a sample that large, more than 300,000 living, breathing students. The fact that it confirms what many AP teachers have been seeing in their classes leads me to think that we are getting close to the truth, and that the vast majority of high schools that still bar average students from taking AP, who tell those kids they just aren't ready for a college challenge, should consider how much even struggling students get out of this experience they are denying their kids.

© 2009 The Washington Post Company