Performing our way out of the impact impasse?

This commentary follows on from a discussion at the plenary session of the STS stream at this year’s British Sociological Association conference on April 13th. It was a discussion that helped pull together some thoughts I’d been grappling with for a while. I’m a cautious optimist when it comes to the impact agenda in general and the proposals to assess impact for the 2014 REF in particular. And I want to argue that the performative approach in STS can help us respond to this agenda in some creative and empowering ways.


For those not familiar with the notion of performativity, it’s a concept that sociology borrowed from linguistics, and in its simplest form it expresses the idea that a statement (or knowledge claim) is performative if it installs that which it describes - if it literally does what it says. A simple example is that of a judge passing sentence or a priest declaring a couple man and wife: their mere words change material reality because they have an inaugurating power. STS has used this idea to explore the way academic theories have (deliberately or otherwise) intervened in material reality and altered the conditions under which we live. Many of the most well-researched examples come from economics. My personal favourite is the strange tale of ‘individual transferable quotas’. Invented by economists as a tool to manage fish stocks sustainably, they have had mixed results as a measure to limit over-fishing. But a paper by Holm provides a captivating description (believe me!) of how the economic instrument of quotas turned fish (still swimming in the sea) into property, and how this fundamentally altered the way fishermen behaved as economic actors and even changed the emotional relationship between fishing communities and the sea’s natural resources.

Apparently we’re subjected to at least 100 different metrics, each of which becomes a performance management tool in the hands of all those with the power to hold researchers to account

But performativity can also be applied to academia itself. At the BSA conference, Roger Burrows detailed the extraordinary extent to which the professional lives of researchers are now governed by numbers. Apparently we’re subjected to at least 100 different metrics, each of which becomes a performance management tool in the hands of all those with the power to hold researchers to account (from research managers within universities to bureaucrats at the funding agencies and Research Councils, right up to government ministers). And the galling thing is that academics - and in particular those working in the field of STS - are partly responsible for the power of the ‘numbers’ that now have a hold over our careers.For a simple example I can refer to my comment on Sarah’s blog about ‘academic slackers’. President Sarkozy’s jibe at French researchers for being ‘30-50% less productive’ than their British counterparts was only possible because generations of STS researchers have invented, refined or given credence to the bibliometric statistical tools that ‘prove’ the superior academic productivity of the Brits over the French. We are co-responsible for literally enacting the world in which these sorts of statistics mean anything, so we can’t complain if we are then hoist by our own pétard (I’m steering clear the obvious national rivalries that you could read into the tale of this particular example of performativity).

Quick-witted readers may have noticed that the above are perfect examples of ‘impact case studies’, in REF jargon. The catch is that while the impact of economists or STS researchers in these cases is demonstrably profound, it’s more dubious whether the changes have been socially, economically, culturally or environmentally beneficial.

Quick-witted readers may have noticed that the above are perfect examples of ‘impact case studies

What do performative narratives typically do? They describe how knowledge passes from a restricted arena to an enlarged arena - from the lab (literally or metaphorically speaking) into some real-world context or other. Let me try to apply a performative analysis to the REF itself. When researchers mobilised their collective intelligence to intervene in the public consultation on the REF, they performed a series of arguments about the likely effects of measuring impact within a restricted arena. The consultation was the equivalent of a science lab. Then some more researchers and research managers ‘co-performed’ the REF in a not-quite-so-restricted arena during the impact pilot exercise. According to HEFCE’s positive assessment (which is largely what counts, as HEFCE has the inaugurating power) they succeeded in making that world act as it was supposed to - they credibly evaluated impact. The REF is now on the point of transfer to the enlarged arena where all British universities will be asked to play by its rules, and ‘for real’.

For those who contest some or all of those rules the key question is at what point in this journey are resistance and counter-proposals most effectively expressed? Did participation in the restricted arenas simply legitimise the tool? The tangible effects of performing the REF 'in the lab' were just a few technical adjustments such as the reduction in the weighting of the impact element by 5% and a reinforcement of the principle that only research judged ‘excellent’ by disciplinary peers would be eligible for the impact test. Some of the more radical responses to the REF propose to intervene further ‘downstream‘. For example, advocates of participatory research approaches have advocated ‘working between the lines’ of the REF once it‘s already up and running. Will they be any more effective? Or will they be thwarted by the power of research offices in their universities? Will the conventions for using the tool of the REF still be open to this kind of creative adaptation at the point at which they need or want to use it to gain consecration for the types of methods and approaches they wish to promote?

The performative approach carries the optimistic message that technologies (and the REF can be thought of here as a managerial technology) are adapted as they are adopted, and while they tend to become more irreversible as they diffuse, this process never achieves finality: there is always scope to re-perform knowledge-in-practice. I hope there is room for creativity when impact case studies come to be written. We have a new tool, which we might not all like, but its effects depend very much on how the ‘user manual’ gets written, and we can influence that more than we might think.

Comments

Tennie Videler
Still relevant to this is Peter Lawrence's 2007 article in current biology on how measuring H-factors and impact factors are particularly detrimental on 'gentle people'.

Simon Smith
Thanks for the link Tennie. It's a nice piece, and he sums up the main message of my blog in the following: "Scientists of all ranks, senior as well as junior, are also to blame as we have meekly allowed this to happen. But can we now start to fight back? We need to raise awareness of the problems and make changes locally."

Cathy Gibbons
Simon, I read your blog with interest, as an educational researcher I especially enjoyed your elegant and brief explantion of performativity, even though my original reason for reading your blog was because of its title. Would  you like to comment on my blog? http://impactwhen.wordpress.com/ My blog really depends on researchers (and those allied to research) telling their stories. Tennie and others might like to have a look too. There's an invite to all on my Vitae blog, but I thought you mioght be particularly interested.