Impact in the REF - why I won't sign the UCU petition

19 November 2009

By Simon Smith

Moving beyond simplistic assumptions about science and knowledge

Much of the debate around the treatment of 'impact' in the REF portrays the issue in black-and-white terms. This inevitably makes me suspicious!

The UCU petition 'Stand Up for Research' (over 12,000 signatures and counting, see https://www.ucu.org.uk/index.cfm?articleid=4207) seeks to draw a clear line between science and its social or ecomomic applications, stating, " As academics, researchers and higher education professionals we believe that it is counterproductive to make funding for the best research conditional on its perceived economic and social benefits". 

At this point, it's worth introducing a couple of principles from two authors who had a more nuanced perspective on the history of science and the nature of scientific knowledge:

- “Scientific knowledge is at once theoretical and empirical, pure and applied, objective and subjective, exact and estimative, democratic (open for all to confirm) and elitist (experts alone confirm), limitless and limited (to certain domains of knowledge).” (Gieryn 1983: 792) Gieryn goes on to describe how quite often both versions of science have been deployed simultaneously in relation to different audiences: thus Victorian scientists, still trying to establish the credibility of scientific knowledge, were apt to stress empiricism and social utility in order to differentiate science from religion, but to stress theoretical depth and disinterestedness in order to differentiate science from engineering.

- Paraphrasing Bourdieu, we can say that science is a 'field' of activity that is at its best when it enjoys relative autonomy from other fields (e.g. politics, economics, the media), but this autonomy is only ever relative (science is always subject to 'external constraints'), and moreover, if science has a duty to share its knowledge, this can only be done by encroaching into other fields such as the three mentioned above. Splendid isolation is not an option.

Recognising different units of analysis

Most criticisms of the impact criteria in the REF, however, start from the idea that scientific research has value in and of itself, and that the status of curiosity-driven research, whose influence (at least in the short term) is confined to the academic community, needs defending. When applied, say, to the introduction of impact criteria in Research Council funding processes, this argument has merit, and regardless of how strenuously the Councils reassure researchers that 'zero-impact' research applications will be treated equally, it seems likely that the behaviour of researchers, in formulating proposals, will be shaped by their awareness of the 'added value' they can give to their application by writing a good impact statement, or by securing co-funding from potential research users (BBSRC, MRC and NERC prioritise applications with pledges of co-funding).

Thus the UCU petition would be a valid position to take in opposing (perceived) Research Council conditionality, although even in this case it needs to be recognised for what it is: an ideological intervention in the ongoing 'credibility contest' which characterises any professional field (as described by Gieryn). In this instance, the intervention is seeking to diminish the credibility of 'applied' research relative to 'pure' research.

Yet the petition is actually about the 25% weighting given to the impact domain of the proposed REF. This overlooks a crucial difference between the operationalisation of impact by Research Councils and by the REF, which concerns the level of analysis. The impact domain of the REF focuses on the research unit (the Unit of Assessment). At this level, it may be quite legitimate to expect impact to be embedded and demonstrable across the full range of research activity, because this does not imply that each individual researcher or research project must produce impact. Rather, it assumes that a successful academic unit will have a collective consciousness and a capacity to act as a unit, and may even be characterised by a division of labour in relation to impact¹. There is an assumption that impact case studies will often cover collaborative work by research teams, with perhaps some members of those teams taking primary responsibility for exploitation or application of findings (HEFCE 2009: paragraph 61).

It is also significant that the case studies envisaged are not case studies of projects, but instead ask research units to perform a sort of tracking-back exercise from an identified economic or social impact to the contribution made by research within the unit. This could promote an aggregate/integrative perspective, and may even encourage more collaborative working within research units in future. This is because case studies that link back to more than one research project would likely score higher given that panels are asked to consider “how far a submitted unit has successfully achieved impact across the full range of activities and contexts appropriate to its field of activity; it should not be possible to achieve the highest score by concentrating narrowly on only a part of the territory that the unit might have been expected to cover.” (HEFCE 2009: paragraph 72)

Achieving impact on our own terms

If this is the case, then the REF seems to be trying to incentivise types of behaviour at the level of academic units which look surprisingly similar to proposals for the French Institute for Agricultural Research (INRA) made by Bourdieu, who was a fervent advocate of academic freedoms, but also an advocate of the political engagement of intellectua. He believed that 'exporting' academic authority into the political field, as well as listening to and responding to the ‘social demand’ for scientific knowledge, are inextricable parts of the life of the scientific community.

But, he went on, they are best done as a collective endeavour, beginning from a defence of the very conditions of scientific knowledge production. Connecting science to user communities and social demand is firstly about overcoming the internal divisions within the scientific field between theoreticians and practitioners, basic and applied scientists, or academics and research administrators (Bourdieu 1997: 67). First you need to construct a kind of 'collective intellectual', which means performing inclusive rather than exclusive 'boundary-work' (Gieryn 1983) – narrativising scientific knowledge in ways that unite, rather than divide, established players and 'outsiders' in a common defence of scientific authority and autonomy. Then science as a whole is better-placed to ensure that the uses to which it is put in the social world are at least partially within the control of scientists. To illustrate how such collective engagement could work, Bourdieu (1997) described how a scientific community such as a research institute might put forward appropriate experts to speak on particular policy issues or engage with government in policy processes, rather than waiting for government or the mass media to select their own preferred experts.

Being proactive and acting collectively are the conditions under which scientists can intervene in the practical world from a position of autonomy. Taken at face value, there is little in HEFCE's proposals for the impact domain that undermines these conditions, and they may even encourage them.