Now that the dust has settled, institutions have posted their interpretations of the results of the 2014 Research Excellence Framework (REF), and people have gone through the results and institutional interpretations in various blogs, I thought it time to put my oar in. This is the first of a series of posts that will look at the REF 2014 score of the Institute of Integrative Biology, how RAE/REF scores for this academic enterprise have fared over time, and whether REF/RAE is of any use.
I had an inside view for REF 2014, in that I was REF wallah for our Institute of Integrative Biology, and a member of the Faculty’s Clinical Medicine subpanel. My long-term involvement with these assessments, which started with the departmental debriefing held post RAE 1992, when I was a newly appointed Lecturer, gives me some perspective.
How did we do?
The Institute of Integrative Biology covers a broad span of biology, from electron transfer in proteins to ecosystems. This is exactly what makes the Institute a great place to work, since such breadth is essential to stimulate lateral thinking. From a REF standpoint we decided to split three ways, into Biological Sciences (largely ecologists and environmentalists) Clinical Medicine (biochemists and some microbiologists), and Agriculture, Veterinary and Food Science. This makes a comparison with previous assessments a little more complex, since on two of these panels, there were a lot of staff from other Institutes in the Faculty. However, the comparison is no more difficult than comparing the original three constituent pre-1996 departments (Biochemistry, Ecology and Evolutionary Biology, and Genetics and Microbiology) with the subsequent single School/Institute, or RAE 2001 with RAE 2008, which used different scoring systems.
Across the 3 subpanels the aggregate scores were remarkably similar, 80 to 82% 3* and 4*. Since there were at least two independent readers for every paper and impact case study and the reading assessment was then double checked by the Faculty subpanels collectively reading these outputs on a huge screen in a boardroom, then the output quality is pretty much normalised. Thus, any contributing component, such as our Institute, can lay claim to what is the Faculty score.
Just over 80% of Institute staff were submitted to REF 2014. We MUST remember two points. First, the Institute is a collective exercise, running from undergraduate admissions to publishing papers. So those with lots of great papers are leaning on those doing admissions and leading teaching programmes, and vice versa. Second, many staff who were not submitted had papers in the 3*/4* bracket, but the limits imposed by the number of Impact Case Studies meant that we couldn’t return more people. The impact case studies imposed a numerus clausus on the number of staff who could be submitted.
So in REF 2014, if we (incorrectly, since there are good papers there, but we have no data) give a 1* or 2* to those not submitted, we have ~65% of the institute with a score of 3* or 4*. This is progress, in 2008 we submitted everyone and the score was 40% at 3* and 4*.
Given that in the 6 years of REF 2014 we continued to both teach (including a full curriculum review and revamping every single course) and research seriously, then this is in my book a tremendous result and a real testimony to colleagues’ ability to work extraordinarily hard to the highest standards.