Impact and REF

Written by Dr Ben Nichols, Research Policy and Governance Administrator, Research Policy and Operations

I’m sure that anyone working in research has heard their colleagues complain about the Research Excellence Framework (or REF). And if you’ve been reading this blog you’ll have seen a number of references to it. The REF is the most recent incarnation of the periodic assessment of research quality in UK universities that has happened every 6 or 7 years since 1986. The results of this assessment translate directly into how much money institutions get from the Government, so universities put a lot of effort into making good quality submissions. But many have criticised the exercise for introducing unnecessary competition, for its administrative burden, and for creating undue pressure on researchers.

Ahead of the REF that took place in 2014, HEFCE (the body responsible for formulating and running the REF) introduced an aspect of the exercise for assessing the “impact” of research. Whereas previous exercises had assessed research outputs (mostly publications) and research environment (the infrastructure and culture that supports research), they wanted a system for assessing how research makes things happen outside academia. The guidance published by HEFCE ahead of the last REF defined “impact” in brief as “an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.” Universities were invited to submit case studies in largely narrative form that described how excellent research that they had demonstrably contributed to had generated this kind of impact. These case studies were then awarded scores on the basis of their “reach and significance” by panels of subject experts and the scores, in turn, made up 20% of a unit’s, and therefore an institution’s, overall “quality profile.”

This focus on research impact introduced by REF 2014 has of course generated differing opinions. Some have welcomed it, arguing that researchers are funded by public money and should therefore show how their work directly benefits the public who pay for them. But as with other aspects of REF, “impact” has come under fire. Some argue that it encourages applied over theoretical research, disadvantages those in precarious positions (such as contract research staff or ECRs), and creates cults of personality around certain impactful academic “stars.” Moreover, some have asked why researchers have been asked to demonstrate their impact when the same burden is not always placed on other activities also funded by public money. At the same time, there is often a recognition that the assessment of impact in REF 2014 has generated a large database of impact case studies (6,975 in fact), useful for demonstrating the tangible benefits of university research to those who might not necessarily be convinced of its intrinsic value.

Whatever is the case, it looks like the assessment of impact is here to stay. HEFCE recently held a consultation on the criteria for the next REF, scheduled to take place in 2021. Proposals for the assessment of impact in the future include: creating a unified definition with the UK Research Councils, introducing institutional- as well as unit-level impact case studies, and widening and deepening the sense of what research impact is. The preliminary results of this consultation are due any day now and we’ll know more once they’ve been announced. One thing we know for sure, though, is that the formal assessment of research impact is unlikely to go anywhere soon.

Comments are closed.