Impact and REF

Written by Dr Ben Nichols, Research Policy and Governance Administrator, Research Policy and Operations

I’m sure that anyone working in research has heard their colleagues complain about the Research Excellence Framework (or REF). And if you’ve been reading this blog you’ll have seen a number of references to it. The REF is the most recent incarnation of the periodic assessment of research quality in UK universities that has happened every 6 or 7 years since 1986. The results of this assessment translate directly into how much money institutions get from the Government, so universities put a lot of effort into making good quality submissions. But many have criticised the exercise for introducing unnecessary competition, for its administrative burden, and for creating undue pressure on researchers.

Ahead of the REF that took place in 2014, HEFCE (the body responsible for formulating and running the REF) introduced an aspect of the exercise for assessing the “impact” of research. Whereas previous exercises had assessed research outputs (mostly publications) and research environment (the infrastructure and culture that supports research), they wanted a system for assessing how research makes things happen outside academia. The guidance published by HEFCE ahead of the last REF defined “impact” in brief as “an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.” Universities were invited to submit case studies in largely narrative form that described how excellent research that they had demonstrably contributed to had generated this kind of impact. These case studies were then awarded scores on the basis of their “reach and significance” by panels of subject experts and the scores, in turn, made up 20% of a unit’s, and therefore an institution’s, overall “quality profile.”

This focus on research impact introduced by REF 2014 has of course generated differing opinions. Some have welcomed it, arguing that researchers are funded by public money and should therefore show how their work directly benefits the public who pay for them. But as with other aspects of REF, “impact” has come under fire. Some argue that it encourages applied over theoretical research, disadvantages those in precarious positions (such as contract research staff or ECRs), and creates cults of personality around certain impactful academic “stars.” Moreover, some have asked why researchers have been asked to demonstrate their impact when the same burden is not always placed on other activities also funded by public money. At the same time, there is often a recognition that the assessment of impact in REF 2014 has generated a large database of impact case studies (6,975 in fact), useful for demonstrating the tangible benefits of university research to those who might not necessarily be convinced of its intrinsic value.

Whatever is the case, it looks like the assessment of impact is here to stay. HEFCE recently held a consultation on the criteria for the next REF, scheduled to take place in 2021. Proposals for the assessment of impact in the future include: creating a unified definition with the UK Research Councils, introducing institutional- as well as unit-level impact case studies, and widening and deepening the sense of what research impact is. The preliminary results of this consultation are due any day now and we’ll know more once they’ve been announced. One thing we know for sure, though, is that the formal assessment of research impact is unlikely to go anywhere soon.

Top Tips to Impact the World Around You

Written by Nudrat Siddiqui, Research Staff Development Officer 

Further to 67.5% of you expressing an interest in undertaking impact training in the Careers in Research Online Survey (CROS), The Centre for Research Staff Development (CRSD) was delighted to run a workshop on impact on 3rd July. The workshop was a great success and featured sessions delivered by various experts on impact, including Dr Kathy Barrett, University Lead for Research Staff Development, CRSD, Dr Richard Matthewman, Faculty of Natural and Mathematical Sciences Research Manager, Stephen Roberts, Research Engagement Manager, Engagement Services, and Dr Jenni Chambers, Head of Public Engagement with Research, Research Councils UK (RCUK). Participants also had the opportunity to engage with staff from the Science Gallery, Policy Institute, Cultural Institute and Entrepreneurship Institute to find out how these departments could support their impact plans and create and receive feedback on Pathways to Impact Statement templates from staff from the Research Development & Pre-Award Team.

In case you missed the workshop, here are some key points that were raised by speakers:

Why do impact?

How can you have impact and when should you do it?

  • Start thinking about and embedding impact into your research practices early, instead of waiting until you have to apply for funding.
  • The PESTLE analysis and Logic Model are useful tools that can help you consider how your research can have impact.
  • RCUK define impact in 2 ways: Academic Impact and Economic & Social impact
  • Academic impact –explores how your research has impacted your field. Ways to have academic impact include publishing papers, presenting at conferences, running workshops, and being invited to deliver talks.
  • Economic impact – Activities that can lead to economic impact include networking activities, such as knowledge transfer networks, IP and patents, and placements. Exchanging people and knowledge is one of the most powerful ways to have impact, e.g. if you spend a few days working in the environment of the end user.
  • Social Impact – can be generated by having an influence on health (e.g. working with healthcare professionals), policy (e.g. getting involved with professional bodies) cultural sectors (e.g. holding exhibitions), within local communities or raising public awareness and understanding (e.g. public engagement and outreach in schools, museums, etc).
  • Consider the timeline of your impact in terms of the Preparation stage, the Project activities stage, and the Continue stage, which may involve further funding and exploring the contacts you’ve made.

What stakeholders should you engage?

  • Consider the level of interest that different stakeholders will have in your research and the influence they will have in supporting your impact activities.
  • Assess whether your research will have a positive or negative impact.
  • Groups of stakeholders you might want to consider engaging could include patients and healthcare professionals, service providers, funding agencies, NGOs and charities, government and policymakers, environmental practitioners, education practitioners and students.
  • Think about how you prioritise stakeholders with limited time and resources.
  • Ways to engage stakeholders could include face-to-face meetings, through visual materials, or exhibitions.

Tips to write a strong RCUK Pathways to Impact Statement

  • If you plan to apply for RCUK funding in the future, you will have to complete a 2-page template for a Pathways to Impact Statement outlining how your research will have impact.
  • RCUK’s main consideration for Pathways to Impact Statements is that your research should have a significant impact on economy and society.
  • Different funding calls have different requirements for Pathways to Impact Statements and there are council specific guides on how to do this.
  • Pathways to Impact Statements are assessed under peer review.
  • Keep Statements simple but thorough and be project specific rather than generalised.
  • Indicate the audiences and sectors you will be engaging and how you will engage them.
  • Consider how your impact will be evaluated.
  • Don’t hesitate to request money or staff time.
  • Include costs related to proposed impact activities.
  • Choose impact activities that are two-way and that will engage stakeholders.

This workshop is also being developed into an e-learning module which will be available internally on KEATS in the next academic year. Keep an eye out for details of when the module will become available in the Research Staff News newsletter.

Impact: A Short Introduction

Written by Nadia Xarcha, Research Information Coordinator, Faculty of Arts & Humanities

Yes, you’ve heard about it! Your PI or line manager has mentioned it at some point and your research colleagues talk about it during their coffee break. Questions like – who came up with impact in the first place? Why do I have to think about it? Why can’t I just concentrate on my research? – have definitely crossed your mind a few times.  I know, it might be overwhelming but I have news for you: You’re already delivering impact!

What is it exactly and when was it introduced?

‘Impact’ was introduced a few years ago for the purposes of the REF[1] and there are several definitions but impact, in simple words, is the benefit of research to society. What has changed (behaviour, practice, etc.) as a result of your research? Examples of impact might be the alteration of public policy, the introduction of a new school curriculum, the improvement of patients’ health and many, many others that cannot be demonstrated in a short post. Just browse through the old impact case studies and you will rediscover the beauty of conducting research.

And why does it matter so much?

In REF terms, impact was developed in order to showcase the reach and significance of research and to allow external audits of how public money is spent. But impact is not and should not be just about the REF. It is the link that makes researchers’ work known to people from outside research who otherwise might not have been engaged, even though it might deeply affect them. It helps researchers reflect on their research questions and outcomes and conduct better research in the long term. It motivates them to communicate their research in simple words to a wider audience.

Should an early career researcher engage with impact?

It’s a personal decision but I would encourage you to do so as it is helpful for the development of your research career. As an early career researcher, you definitely have a lot in mind and impact might seem to be the cherry on the cake, but I think this is not the way to see it! Impact can actually be satisfying. What I would suggest is that you focus on doing exceptional research and enjoy that journey. The ‘impact element’ of your research will then emerge naturally. And we have plenty of resources (people, training, seed funds, etc.) here at King’s to support you with it at every stage of your research (the earlier the better). Just talk to us. Remember, you’re not alone in this

[1] REF: Research Excellence Framework is the system for assessing the quality of research in UK higher education institutions.