US: Decoding learning gains

Throughout the world, interest in gauging learning outcomes at all levels of education has grown considerably over the past decade. In higher education, measuring 'learning outcomes' is viewed by many stakeholders as a relatively new method to judge the 'value added' of colleges and universities. The potential to accurately measure learning gains is also viewed as a diagnostic tool for institutional self-improvement, write Gregg Thomson and John Aubrey Douglass in a new paper for the Center for Studies in Higher Education at the University of California Berkeley.

Decoding Learning Gains: Measuring outcomes and the pivotal role of the major and student backgrounds, which is freely available on the CSHE site, compares the methodology and potential uses of three tools for measuring learning outcomes, according to its abstract: the Collegiate Learning Assessment (CLA), the National Survey of Student Engagement (NSSE), and the University of California's Undergraduate Experience Survey (UCUES).

"In addition, we examine UCUES 2008 responses of seniors who entered as freshmen on six of the educational outcomes self-reports: analytical and critical thinking skills, writing skills, reading and comprehension skills, oral presentation skills, quantitative skills, and skills in a particular field of study," write Thomson, who is Executive Director of the Office of Student Research and Campus Surveys at the University of California Berkeley, and Douglass, a Senior Research Fellow at the Center for Studies in Higher Education.

"This initial analysis shows that campus-wide assessments of learning outcomes are generally not valid indicators of learning outcomes, and that self-reported gains at the level of the major are perhaps the best indicator we have, thus far, for assessing the value-added effects of a student's academic experience at a major research university."

The authors advocate UCUES as the most effective approach for assessing and reporting learning outcomes, because it "offers more extensive academic engagement data as well as a much wider range of demographic and institutional data, and therefore an unprecedented opportunity to advance our understanding of the nature of self-reported learning outcomes in higher education, and the extent to which these reports can contribute as indirect but valid measures of positive educational outcomes.

"At the same time, the apparent differences in learning outcomes across the undergraduate campuses of the University of California, without controls for campus differences in composition, illustrates some of the limitations of self-reported data," conclude Thomson and Douglass in the abstract.