Crisis has shown virtual quality assurance can work well

Higher education quality assurance is in its infancy in Ukraine. Although many of the challenges faced by the university system – including increased academic autonomy, reduction of corruption and improvements in academic integrity – have been tackled over the past six years since the adoption of the 2014 Law on Higher Education (adopted immediately after the Maidan protests), implementation of quality assurance at the programme level in universities has been sporadic at best.

The launch of the National Agency for Higher Education Quality Assurance (NAQA) in 2019 was aimed at improving the situation. The launch was successful, but then came the COVID-19 pandemic.

Ironically, the challenges presented by the global lockdown seem to have accelerated quality improvements in Ukraine – even to the point where Ukrainian practice, when it comes to video-conferencing, has now overtaken that of more established agencies in other countries.

Throughout the world, quality assurance agencies have thus far implemented accreditation through video-conferencing in a very limited way.

Key international quality assurance organisations (ENQA, EQAR, INQAAHE, CEENQA and CHEA) have issued recommendations on adapting quality assurance (QA) processes to quarantine conditions and to the protocols of individual agencies (for example, AQU Catalunya, the Catalan University Quality Assurance Agency).

But most agencies have postponed site visits until after the lockdown, limiting their online activities to internal meetings and consultations. This is despite members of QA associations being urged to show flexibility in their own review processes and, more explicitly at a time when site visits cannot be organised, being encouraged to use video-conferencing and other appropriate formats.

Hundreds of site visits

This advice was taken literally by NAQA, and the results have been somewhat revolutionary.

Over the past four months, accreditation experts from Ukraine’s NAQA have conducted over 430 site visits to universities using video-conferencing technology, and the experience of such programme accreditations has proved highly effective – both as a means of gauging the technical readiness of universities to implement blended learning technologies and their understanding of the essence of quality, as laid out in the Standards and Guidelines for Quality Assurance in the European Higher Education Area 2015 (ESG 2015).

The widespread use of telecommunications and video-conferencing to carry out accreditation site visits is a new experience not only for Ukraine. No European agency has ever done it (certainly not at such scale).

Of course, being a pioneer can be stressful. It requires organisation and mutual support from all accreditation process participants – both within the National Agency and between it and higher education institutions. In the case of Ukraine’s National Agency, all its members jointly coordinated with all of the departments of the secretariat, with a central role played by the experts department.

According to the head of the experts department, Kateryna Kunytska, the video-conference format requires more planning and preparation from all participants in the process: experts, higher education institutions and the National Agency.

The level of responsibility is high and everyone is well aware of it. The agency has developed a step-by-step list of instructions and has mandatory preparatory meetings with participants and mandatory briefings conducted by the chair and-or deputy chair(s) of the agency.

At least one responsible staff member of the secretariat is assigned to each video-conference call. Members of NAQA are actively involved as observers during the online site visits. At any time, they are ready to provide assistance and support to ensure the effective outcome of such meetings.

How ‘quarantine’ accreditation is different

Firstly, quarantine accreditation is cheaper than face-to-face accreditation: higher education institutions save on experts’ travel costs. Secondly, whereas face-to-face site visits do not always allow physical participation by members of the National Agency, video-conferencing technology enables this.

In the words of accreditation expert Alexei Barybin: “The presence of agency representatives during meetings gives greater objectivity and transparency to the whole procedure.”

In all other aspects site visit procedures by teleconference are identical to a face-to-face visit; the process is simply transferred online.

Video-conference site visits are not a whim. They are compulsory. On the other hand, technology presents great advantages. All video-conference sessions are recorded for internal use by the National Agency, thus making the procedure more transparent. The aim is to overcome potential subjectivity in experts’ evaluations and to deal with controversial issues, if they arise.

The video-conferencing format is also convenient for stakeholders who can now easily join meetings using their smartphones from anywhere that has an internet connection.

This means that an employer will have less reason to refuse a request by an educational programme’s guarantor (the person who is responsible for the programme) to take part in an accreditation meeting if it is online.

The video-conferencing format also means information can be obtained more quickly: the parties can easily show and discuss documents, links, videos, etc, during meetings.

Strict rules for meetings and the need to separate expert group internal meetings from other meetings mean that disruptions while online (for example, to ask an extra question) are minimised.

Among the difficulties of the new format, the most obvious is the problem of reliable internet connections. But this problem is not widespread, nor permanent.

Difficulties with internet connections can be caused by technical problems or the temporary shutdown of electricity due to an accident. In general, NAQA’s experience (the first video conference site visit was launched on 25 March 2020) has proved that such difficulties are minor and easily resolved.

Video-conference accreditation site visits will definitely not replace the face-to-face format after the current crisis is over. As accreditation expert Nataliia Ausheva says: “Face time with people is just not the same as distance communication.”

And even though sometimes it can be difficult, in this emergency the ability to combine efforts, demonstrate high levels of professionalism and flexibility in quality assurance is of great value. We look forward to opportunities to share these experiences with other quality assurance agencies.

Mychailo Wynnyckyj is head of the secretariat at the National Agency for Higher Education Quality Assurance in Ukraine.