Will the REF add up to good science?

30 November 2009

By Paul Wilson

The Research Assessment Exercise (RAE) of 2008 was the last of its kind. Since then, the Higher Education Funding Council for England (HEFCE) has been looking at ways of replacing the RAE. Many academics complained that the burden of compiling an RAE submission was too onerous. The solution was the Research Excellence Framework (REF), which would make heavy use of bibliometric indicators based on journal citation statistics. The theory was that this would obviate the need for detailed peer review of submissions.

Well, the consultation document for the REF has recently been published, and it seems more of an evolution of the RAE than the wholesale changes that were predicted. Peer review will stay as a major component of the system, but bibliometric data will be used to “inform” that review, depending on the research field. But the use of such data is notoriously tricky, and has been criticised by a number of prominent researchers (see, for example, here and here).

The other major change is that “Impact” will make up 25% of the assessment for the REF. Exactly what this will entail is not entirely clear, but the consultation document suggests a wide definition, including “economic, social, public policy, cultural and quality of life” impacts. In some ways, it seems reasonable for researchers to show how they have contributed to society. The problem is that actually doing this is not straightforward, and this is likely to add considerably to the burdens of the research assessment process.

In the light of current disquiet about the effects of research assessment and the bureaucracy faced by young researchers (see, for example, this report or this recent article), is the REF a good thing? Or is it likely to lead to gaming of the system that is detrimental to good research?