Students in a free online physics course from the Massachusetts Institute of Technology demonstrated roughly equal learning gains if they stuck with the class, regardless of previous academic experience, researchers reported Tuesday.
Their progress also was comparable to what some MIT students showed when they were required to take the introductory course on campus as a remedial measure.
The findings offer evidence that “massive open online courses,” or MOOCs, can be an effective way to teach a broad range of people from around the world who want to learn from top-flight universities without paying tuition.
“It’s an issue that has been very controversial,” David Pritchard, an MIT physics professor, said in a statement. “A number of well-known educators have said there isn’t going to be much learning in MOOCs, or if there is, it will be for people who are already well-educated.”
Pritchard led a team of academics from MIT, Tsinghua University in China and Harvard University who analyzed the results of an MIT course called Mechanics Review, which was offered in 2013 on the MOOC platform edX. Results of their study were published in the International Review of Research in Open and Distance Learning.
MOOCs have drawn widespread attention for more than two years as a potentially disruptive force in higher education. Web sites have sprung up to offer hundreds of courses free, covering subjects from computer science to poetry and produced by major colleges and universities.
Coursera, based in Mountain View, Calif., is a for-profit company that offers online courses from the Universities of Virginia and Maryland, among many others. MIT and Harvard founded nonprofit edX, based in Cambridge, Mass., in 2012; participants now include Georgetown University, among others.
There are dueling views on the significance of MOOCs. Advocates say the courses are lowering barriers to higher education around the world. Skeptics say they are chiefly a promotional vehicle for brand-name schools, pointing out that many people who enroll never finish the courses.
The MIT-led study aimed to quantify what students learn if they persist.
About 17,000 people registered for Mechanics Review, a MOOC patterned after a course offered on campus to MIT students who had received a D or lower in a fall physics course. Most of those who registered were just window-shopping. After the second assignment, fewer than 10 percent were on track to complete the MOOC. Eventually, about 1,000 people earned certificates of completion.
The researchers gave students an identical test before and after they took the MOOC, comparing results to assess gains. They also analyzed in detail the performance of 1,080 students who attempted more than half of the MOOC questions, using a statistical method called item response theory. That meant the MOOC “dropouts” were effectively excluded from the study.
What surprised Pritchard was that various types of MOOC students showed relatively equal progress. Those who had obtained a doctorate advanced about as much as those whose highest credential was a master’s degree, a bachelor’s degree or a high school diploma. Gains also were about equal for MOOC students regardless of their level of preparation in math and physics.
That did not mean all scored equally well. Some had superior results, and some failed. But the relative progress of those who persisted was roughly equal across groups.
“If you look at the final grades, you’re going to say [some] people did horribly,” Pritchard said in a telephone interview. “But they learned as much as everybody else. That’s an important point.”
The analysts also compared how the persistent MOOC students did with results for 35 MIT students who took a similar review course on campus in 2013 after struggling with the subject. Here, too, there was a surprise: Even though on-campus students had extra instruction, researchers found “no evidence of positive, weekly relative improvement of our on-campus students compared with our online students.”
Pritchard said this finding was puzzling, given how much faculty attention the MIT students receive. “I had hoped that because they went to our classes, they would learn a lot more,” he said.