The Principles Behind the Success of AUN-QA Assessments in the New Normal

A staff member from Universitas Andalas answering questions from AUN-QA Assessors during the online/remote site visit.

By Buranond Kijwatanachai, Programme Officer
7 October 2020

COVID-19 hit the pause button for humanity. The world screeched to a halt and everyone scurried indoors to wait until it was safe. It has been almost six months since then. The world has begun to resume marching. The gears of the economy have restarted. People have become mini-germaphobes and have begun to discover a new world with new norms. But there is one thing that has yet to start up again—international travel. Which means that most international cooperative projects, with the exception of meetings and seminars, are still suspended in limbo.

Not all have chosen to be stifled by these conditions.

The AUN-QA Online/Remote Assessment Task Force coming up with the process and procedures for implementing the new online/remote site visits.

While many are struggling to keep their heads above the water, the AUN Secretariat, in its work for the AUN-QA, has been anticipating and preparing for the situation presented by COVID-19 ever since mid-January, before COVID-19 was even declared by the WHO as a global pandemic. The AUN Secretariat in close consultation with the AUN-QA Online/Remote Assessment Taskforce, the AUN-QA Council, and the AUN-QA Technical Team, prepared a blueprint for how the AUN-QA can continue to provide its classic quality assurance services in the new normal.

The 6th AUN-QA Council Meeting approving the implementation of online/remote site visits in the AUN-QA Programme Assessment.

As of the end of September 2020, the AUN-QA has completed its first four rounds of programme assessments in the new normal, proving that the network will not only be unrestrained by COVID-19 but to also use the pandemic as an opportunity to elevate its practices to higher standard.

Its official name is “Online/Remote Site Visits in the AUN-QA Programme Assessment” and it is the AUN-QA’s solution to transnational operations amid the COVID-19 pandemic. It delivers the classic AUN-QA quality assurance service, with added flexibility, without sacrificing any of its spirit or essence by going online. This is all made possible through the network’s meticulous approach towards preparation, uncompromising dedication towards assessor well-being, and devotion to continuous improvement.

To put into context the success and productivity of the AUN-QA in the new normal, a total of four assessments have been conducted which evaluated 11 study programmes involving 22 AUN-QA Assessors. Over 1000 stakeholders have been interviewed, the physical facilities of 11 programmes virtually visited, and countless documents examined and verified. And all of this without any major disruptive issues despite all assessments being conducted virtually.

Here’s how they did it.

Maintain the essence of the classic AUN-QA Assessment

Simply doing the assessment but virtually does not guarantee that it will be effective. Each step of the assessment must be carefully crafted to ensure that the online/remote site visit retains the spirit of its on-site counterpart. Here are three examples of how this was put into practice:

1.) Dividing the interview session into smaller groups

During on-site assessments, interview sessions were usually conducted in large rooms with a large amount of interviewees. However, the AUN Secretariat, through consultation with the AUN-QA Council and Technical Team, identified that interactivity in large online conference calls will be significantly lower based on their extensive experience in organizing virtual meetings. Therefore, the secretariat opted to divide up some of the interview sessions into smaller, logically grouped sessions to maintain the high level of interactivity which is the true core of the interview.

2.) Assigning dedicated web cameras for each interviewee

Each interviewee would have one web camera dedicated to that individual. This makes the interview authentic, allowing for the AUN-QA Assessor to reliably read the facial expressions and body language of the interviewee as if the interview was conducted in person.

3.) Live streaming the site visit

In order to create a true site visit, close collaboration and preparation with the university being assessed results in good camera work from a video camera that is livestreaming to our assessors which gives them an in-depth and up close look at the facilities they wish to inspect. The level of interaction between the assessor team and the facilitators is still very high, no different than being on-site. Because of the online medium, the detailed information that assessors will request can also be more readily provided in an easy to understand format.

Be flexible within reason

The AUN-QA operates transnationally. This means that it goes beyond national borders to deliver its services. Its assessor teams are always diverse and international. Its member institutions are located all over the ASEAN region. Because of this, the secretariat must deal with varying local protocols and national policies regarding COVID-19 that differ from country to country.

This means that the AUN-QA must be able to deploy its services in a flexible manner that befits the specific circumstances of the institution being assessed as well as everyone involved in the process.

In essence, this means that interviewees can join in either from home or on campus. Overall, AUN-QA Assessments in the new normal have had a hybrid approach where certain interviewees will join in from home while others are on campus depending on the local COVID-19 restrictions and their own convenience.

Employ exhaustive and meticulous preparation

Transnational operations are extremely complicated to pull off. Factor in the online medium and COVID-19 pandemic and you have a highly elaborate and involving challenge to tackle. Having organized and facilitated countless virtual meetings in the past six months to remain productive, the AUN Secretariat has an abundance of experience that is necessary to organize effective virtual operations.

The secretariat has identified several key anticipated issues based on their experience and prepared logical solutions that will alleviate, if not outright prevent, these issues from occurring. These issues and their respective solutions are the following:

1.) Equipment Readiness

On top of the usual preparation for the classic AUN-QA Assessment, all parties involved in the assessment, from assessors to university staff, will be using equipment that they may or may not be accustomed to. They may not even have the necessary technology to be able to handle conducting this type of assessment.

To solve this, the AUN Secretariat has prepared a suggested equipment specifications list which includes items like computer hardware and camera resolution that will be sent to the university ahead of time to allow the university to prepare accordingly. Likewise, the secretariat also prepares equipment for the assessors in order for them to perform their duties at maximum efficiency. A dry test run is also deployed to ensure smoothness on assessment day.

Virtual facility visit Readiness Test for the 179th AUN-QA Programme Assessment

Furthermore, the secretariat also has separate orientations for assessors and university staff to prepare them for the assessment itself. This includes familiarizing both parties with the Zoom application and how the AUN-QA uses it so that during the assessment, there is little to no confusion about usage.

A step-by-step guide to the entire readiness preparation process is provided to both the university and assessors.

The virtual closing ceremony of the 178th AUN-QA Programme Assessment at Universitas Andalas (below). The ceremony was attended physically by those on campus while adhering to social distancing guidelines (above).

2.) Internet Latency and Stability

The AUN Secretariat places extra emphasis on internet latency and stability. To ensure that the internet connection is responsive and stable, the secretariat requests all parties to check their connection following the internet speed checking procedure that the secretariat has provided.

A chart compiling the results of the internet speed test from all the interviewees.

3.) Mid-operation disruptions

From prior experience with virtual meetings, the AUN Secretariat knows exactly what can disrupt online meetings as they happen. The specially trained and prepared secretariat is prepared to facilitate the assessment to be as smooth as possible by providing instant and non-intrusive technical assistance and preventing potentially disruptive noises from live microphones or echoes.

The AUN Secretariat and AUN-QA Assessors in extensive discussions with Universiti Brunei Darussalam in the preparation for the 180th AUN-QA Programme Assessment.

Use local verifiers

Local verifiers are a new addition to the AUN-QA Programme Assessment. Because currently the site visit is conducted virtually, these local verifiers act as additional eyes and ears for the assessor team on-site.

The role of these local verifiers is to act as a third-party observer that will double-check documents provided by the universities and corroborate university facility site visits.

Local verifiers are selected through mutual agreement between the AUN Secretariat, AUN-QA Assessors, and the institution being assessed. After they are selected, the local verifier will be on standby at the campus throughout the assessment.

The local verifier performing his duties during the 181st AUN-QA Programme Assessment at Chulalongkorn University

Be thoughtful towards the well-being of assessors

At the end of the day, assessors will be hard at work during an assessment. The AUN Secretariat has their health and well-being in mind because these assessors are performing at their best when they are healthy and not overworked. This is part of the reason why AUN-QA Assessments in the new normal have been extended to five days instead of the usual two and a half days but with a shorter duration per day—to help reduce fatigue.

Always reflect and improve

Perhaps the most important principle of all is the value of continuous improvement that is embedded into the working culture of the AUN-QA. This forms the bedrock for all the other principles detailed above. It also means that the AUN-QA will never remain complacent in its quest to develop the ASEAN higher education sector.

Despite its success thus far, the AUN-QA is still hard at work to improve its online/remote site visits. COVID-19 has rendered most of the world’s certainties null, but the AUN-QA can guarantee that it is always working to build a blueprint that is flexible enough to accommodate any of its over 200 member institutions as well providing consistently high quality service. What the AUN-QA will guarantee, based on this example of the online/remote site visit deployment, is that it is demonstrably quick to react to changing circumstances and will always work to produce an effective solution to whatever situation that may arise to ensure that our members will always enjoy the service that the AUN-QA brand is committed to deliver.