GLOBAL: How to measure citizenship outcomes

A full house of curious academics gathered to hear Carlos Mijares describe how Tecnológico de Monterrey (TM) in Mexico is going about measuring the outcomes of citizenship education among students.

A growing number of universities are encouraging their students to learn citizenship skills but few have a system in place to measure how well they are doing it.

"The easiest thing to measure is inputs and outputs such as the number of students, institutions or hours spent studying, but I need to measure their soft skills; that is much more difficult but it can be done," said Mijares, vice-president of academic affairs at TM, during the Talloires Network Leaders' Conference held in Madrid's Autonomous University on 14-16 June.

TM started the evaluation process two years ago as part of its drive to improve its citizenship education and boost overall quality. Defining the skills and competences to be measured was the first task and it was decided to produce two definitions or indicators each for citizenship skills and for ethical values. Those for citizenship skills consisted of, first, being aware of the social, economic and political conditions of Mexico, and second, being capable of taking action in order to improve the life of community, especially for people living in hardship.

Three main methods are used to teach citizenship skills during the nine semesters of an undergraduate programme at TM. These are compulsory courses on ethics and social responsibility, including ethical dilemmas in courses across the range of disciplines - medicine is easy, according to Mijares, engineering can be more of a challenge - and giving students opportunities to get involved in civic activities. Students at TM are required to complete 480 hours of service to the community in order to graduate. This activity is evaluated although it does not count towards their final qualification.

Students were tested at the end of their first year and just before graduation by evaluators from another TM campus using detailed questionnaires based on the National Survey of Student Engagement in use at American and Canadian universities. Initial results from the 2010 exercise showed that citizenship education at TM still has some way to go, according to Mijares. Only 34% of students knew enough to be able to evaluate the political, economic and social conditions of Mexico, for instance.

"In general we found that students were more predisposed to act but were not so interested in amassing sufficient knowledge to inform their actions," he said. Scores varied according to disciplines, with students of engineering being more likely to take action than social science students.

This first attempt at measuring how good TM is at instilling ethical values and a sense of active citizenship in students will be repeated every year for the next nine years, but Mijares was already able to draw some conclusions and give some tips to his audience in Madrid.

"First, you must be very careful about how much data you compile as otherwise you can end up with lots of data but few results," he warned.

Being able to get across the aims of the exercise to people in as accessible and engaging a way as possible is also important and, seeing as staff support for your initiative is vital, this message should be communicated to managers and lecturers first and then to students.

Next it is important to avoid only preaching to the converted. "It is useful to work through your offices for civic engagement," said Mijares, "but it is important to make sure that you involve the whole of the university in this exercise."

Finally, to produce useful results, you must be in it for the long term and this means organising long-term funds. "This can't be something that is done in a day and you mustn't pass on the message that this is just a passing trend," said Mijares. "At Tecnológico de Monterrey, we have budgeted for 10 years."