Higher education’s unique storage and data requirements

Data storage is a key requirement for any organisation. However, the needs of the higher education sector are wholly distinct compared to traditionally defined commercial business entities. Those needs include vast volumes of research data and multiple departments requiring access to data, as well as sensitive personal information on students and faculty.

COVID-19 has also brought with it a fresh set of data challenges, driven by the need for remote working and learning and the sheer amount of e-learning video data generated as a result. Across the board, the ongoing pandemic has forced a whole reappraisal of institutional data storage needs within the education sector.

Old architectures and processes suddenly no longer fit the current educational environment. These factors bring storage considerations to ‘life’, prompting discussions about how to manage diverse data types, data needs and architectures.

COVID response

In the early stages of the pandemic, IT practices were largely reactive, due to the lack of ‘coping’ methods, such as policies for long-term remote working, e-learning and data access within pre-existing IT strategies. And although things have settled, IT leaders may be dealing with the aftershock of this for some time – creating yet more complexity in their roles.

Universities and research institutes alike have unique signifiers and sensitivities that define their approach to data management. These attributes, explored in a new industry report carried out by Western Digital with IT leaders from universities and research institutes across North, Central and Eastern Europe, fall into four distinct categories:

• Research data volume. As opposed to the lighter touch approach of agile business organisations, universities often find themselves contending with vast quantities of data related to research.

From enrolment to graduation, these rich learning environments are exponentially generating unpredictable amounts of different data types – smart video, video conferences, courses, tests, results, research, images, lectures and unstructured data, to name but a few – that need to be properly protected and stored. As a result of this, individual file sizes, like video, can be enormous.

This places additional strain on the data storage infrastructure within which they are managed. Naturally, this impacts the storage solutions IT leaders choose to work with, and the capabilities these technologies are required to provide in terms of security, performance, sharing and compliance.

• Organisational diversity. Universities are distinct from general business organisations because they perform a myriad of additional functions in addition to core research and operational management.

Educational organisations process personal and sensitive information about their staff and students, such as financial data, medical and personal records and so on.

All that information can be subject to specific local compliance requirements, most notably the General Data Protection Regulation (GDPR) that all European institutions need to abide by. Therefore, having storage solutions that can comply with the proper encryption is crucial.

• Committee decision-making. Unlike the hierarchical management structure of large businesses, educational institutions are decentralised and flatter. Any decision-making on the types of data storage solutions to use tends to be made on a committee basis, as opposed to purely by senior management.

This type of decision-making challenges conventional ways where the divide between ownership and stewardship of data is more distinct.

While IT may not always know where the value resides in the data, and academics and faculty may also not understand the intricacies of data storage and management, in universities there is an effort to make decisions with an understanding of each other’s perspective.

• Collaboration culture. The traditionally open ethos of universities and their culture of collaboration and sharing of data and research is again distinct from the commonplace strategies pursued in businesses, which adopt a more secretive approach to data in order to maintain a market advantage over their competitors.

Data sharing is a fundamental requirement in academia and so can drive the requirements in choosing more bespoke IT solutions.

The transactional process of sharing data between universities is particular to each institution. While most universities take full advantage in the exchange of data across their departments, the way they share it externally differs.

For instance, a representative of a public university in northern Italy spoke of using shared cloud storage, whereas their IT counterpart at a public university in Scotland instead relies on the platforms of other universities.

Cloud storage and data governance

Some IT leaders give a lot of autonomy to ‘super users’ of data storage, such as researchers within universities, as their work drives the value of the educational institution.

A university in London spoke of modelling their data storage architecture according to the needs of their ‘super user’ researchers. Their sophisticated use of the IT infrastructure means that this data-literate cohort needs to have a key role in infrastructure decision-making.

Universities are taking advantage of cloud technology to a degree. However, given the data attributes that education IT leaders work with, and their storage priorities, on-premises solutions are still used.

According to the report, currently under a third of data in total is stored in the cloud – even less sometimes for research institutes.

One research institute in Germany stored 15 times more data on-premises as compared to the cloud. And while some are looking to move the dial in favour of the cloud, both scale and the sensitivity of data are hampering greater cloud storage adoption.

Naturally, this varies between the types of institutes and the data attributes. For example, the research found that cloud use is more heavily favoured in cases where there is much data sharing and in smaller universities.

In addition to the four categories detailed above, there is another distinct challenge facing the education sector’s data governance. Most universities surveyed have some element of data governance in place. However, policies vary in complexity, and frequently ‘tier’ data types according to sensitivity and value.

These policies often also determine whether data can be stored in the cloud, or on-premises, which can be linked to GDPR, a notable concern for European universities.

For example, operational data (financial data) and student data would likely be placed in the GDPR-sensitive tier, whereas archival data that is not often used would likely fall into the low priority camp.

Data storage is complex and there’s no one-size-fits-all solution, and both cloud and on-premises have a place. According to the survey, cloud is generally viewed as the future among most IT leaders surveyed, but on-premises solutions are still widely used.

Regardless of the solution, it must demonstrate value, scale easily and be an enabler of the future priorities and plans of the university. It must also balance levels of security and compliance, as well as smooth workflows and capacity requirements for students, academia and faculty.

Davide Villa is director of business development for Europe, Middle East, Africa and India at Western Digital.