For the first time, Higher Education Institutions (HEIs) were required to submit impact case studies as part of the 2014 Research Excellence Framework (REF). In total, 6679 non-redacted case studies were submitted, and today, we publish a report of the results of our text mining analysis of these data.
The case studies are now available to read online in a searchable database developed by Digital Science, providing a rich resource that has enabled us to demonstrate that UK research has thousands of different applications worldwide. The analysis of the case studies, led by the Policy Institute and department of digital humanities at King’s College London, used text mining techniques leading to the identification of 60 impact topics or areas where research influences society, such as medical ethics, climate change, clinical guidance and women, gender and minorities. Automated text mining was also supplemented with ‘deep mines’, where more than 1000 case studies were read to provide a deeper picture of the data – looking at specific questions such as ‘what is the impact and value of research on clinical practice and health gain?’ and ‘what has been the impact of research on BRIC countries?’.
The results of this analysis are fascinating and also discussed in Research Fortnight and on the HEFCE blog. The take home message is that the benefits from research are multi-impactful. For example, the case studies submitted for the unit of assessment on psychology, psychiatry and neuroscience showed that these disciplines made a contribution to 49 of the 60 impact topics. These included obvious applications such as in mental health, but also in impact topics including transport, schools and education and crime and justice. We also discovered that the case studies contained more than 3700 individual pathways to impact presenting a real challenge to anyone interested in producing impact metrics.
Many of the case studies were able to provide a clear illustration of the contribution that universities make to society, in a way that has not been revealed before. For example, one case study reported on research showing that the painkiller co-proxamol was the most common drug used for suicides in the UK. This finding led to its withdrawal, and has been estimated to have led to approximately 600 fewer deaths by 2012 in the UK alone.
Our analysis has also shown that impact of UK universities is truly global with the Details of the Impact sections of the case studies mentioning every country in the world – with the US being the most frequently mentioned, followed by Australia, Canada and Germany.
The case studies provide a rich resource demonstrating the breadth and depth of research impact. They can also help us to change perceptions. For example, the largest impact topic was informing government policy, which was associated with 1233 case studies. The word ‘policy’ was mentioned in 3206 case studies. This is reassuring, given the pre-REF scepticism about whether case studies could capture and articulate impacts on public policy. It was suspected that researchers mostly influenced policy through personal contacts and under-the-radar advisory channels, rather than through specific research that could be described in a case study.
Our analysis is really just the start. There are limitations to using these case studies as research material, such as the universally positive sentiment in their language and the fact that researchers could carefully select which case studies to use and the number of identical or near-identical submissions. However, this rich resource provides numerous avenues for future research – to enable us to really dig deeper into the global impact of UK University research.