The cruel world of funding peer review

This article began life as a presentation for Peer review and grant funding: From evidence to practice at Melbourne University, 17 November 2017. Thanks to Adrian Barnett and Philip Clarke for inviting me to speak.


Before I begin, I should point out that I write from a position of incredible privilege. I’m not an academic – I’m a university administrator. I am securely employed, and have been for most of my working life. My job is to help academics find funding for their research.

In that role, I work with Australian academics from RMIT University. I work with artists, designers, educators, social scientists and humanities scholars, primarily on their Australian Research Council applications. A significant number of the academics that I have worked with over the last seven years have been early career researchers, generally trying to win their first major grant.

Early career researchers face a cruel world these days. Even though they are an increasingly diverse cohort, they are still generally imagined as young, full-time academics without significant outside commitments. They aren’t. Many of them have significant responsibilities outside of work, taking care of children and elderly parents or working on limited visas, far from home. Most of them have no secure work, while being expected to take on increasing levels of accountability. Their research outputs, their teaching performance and even their scholarly engagement with the world are under intense scrutiny and evaluation.

All this has interesting ramifications for the peer review system that we use for government grants.

A professor stands at the top of a pyramid of scholars and students. Advice flows downwards and cites flow upwards. When funding runs out, the scheme collapses.

Beware the Profzi Scheme, on “Piled Higher and Deeper” by Jorge Cham http://www.phdcomics.com

The cruel world

Casualisation of the workforce means that most early career researchers are on similar employment contracts to supermarket employees. Those that have contracts (as opposed to casual employment), often find themselves stuck at the lowest levels of the academic strata (level A and B in Australian university employment terms) on rolling 12- or 18-month contracts. The number on three-year fellowships is vanishingly small.

Static or falling government funding and industry support for research, along with increasing expectations on ‘academic superheroes‘, means that research funding is increasingly competitive. Most of the funding goes to academics in secure employment (permanent or tenured employees), who have often been in the academic workforce for some time.

Taken together, this means that we are hollowing out the university research system. It just keeps getting worse. Fewer and fewer people have the secure employment that allows them to lead significant research grants. Those that are secure in the system are facing static or falling funding envelopes, which often means smaller individual grants.

The government’s Impact Agenda is changing (or at least confusing) what type of research that academics feel that they should undertake. My own university has had a long history of applied research and education. It is now returning to that focus, having spent the greater part of the last two decades encouraging staff to compete for less applied, more ‘pure’ funding schemes.

This is particularly confusing for early career academics. I’m seeing applications where people are over-emphasising the value of their research to government agendas, and making wild claims about the potential impact that their work will have.

On top of all this, the ‘Metric Tide‘ of academic scrutiny means that they must make every single thing count, because every single thing that they do is being counted. As a result, early career researchers are either becoming disillusioned, or being turned into highly focused research machines. Neither is healthy for strong, curiosity-driven research.

Advice for individuals

So, what should an early career researcher do in such a cruel world? I don’t know. What I do know is that none of the suggestions below will do you any harm.

Here are some things that you can do as a reviewer:

  • Learn how the system works. Register to review for granting agencies. If you are dreaming of working in another country in the future, register to review their grants, too. Granting agencies are often hungry for international reviewers. Being a reviewer gives you the best education that you can get in what makes a competitive funding application, and it is good for the system overall, too.
  • Because most funding agencies don’t pay for reviews (or provide a token payment), you should set clear limits, both in how many reviews you will do, and how long you will spend on each one. Keep in mind that the funding agencies will want a relatively quick turn-around.
  • Since you live in a world of measurement, record your effort. Publons provides academics with a service to track their peer review contributions. If you don’t want to be that public, consider using an academic log to track your own work.
  • Don’t just learn by doing. As with any activity like this, seek out mentoring, including peer mentoring. I think that mentoring is particularly important for early career female academics. It isn’t just a cruel world – it’s a gendered cruel world.
  • Look for advice. A lot of advice on peer reviewing for publications applies to grant reviewing, too.
  • Be kind (or at least be the change that you want to see). It doesn’t have to be a cruel world.

Here are some things that you can do as an early career academic, competing for funding in this cruel world:

  • Choose balance. Don’t let unrelenting pressure to ‘do more’ ruin your passion.
  • Build a rejection wall. Peer review has two sides, reviewing and rejection. A rejection wall will help you cope with the inevitable rejection that comes with peer review.
  • Consider writing ONE grant application for someone else. I hesitate to recommend this, because so many people find this an abusive situation. I think that it can be useful, but only under the following conditions.
    • Only ever write a grant application for someone else as a learning exercise (hence, only ever do it once).
    • Only write it for someone who has enormous experience at winning grants, and is willing to share that experience.
    • Only write it if there is some clear advantage to you, such as being written into the budget (hello, three-year contract!) or being listed on the front page.
  • Let other people do the work. Not everybody has to lead. Not everybody can lead. At this stage of your career, being a collaborator in a successful grant is really, really useful, and probably more realistic than leading one.
  • Reject the peer review system altogether. There is a world of funding that isn’t peer reviewed. Industry funding, philanthropic funding, crowdfunding, micro-patronage… Get creative with your funding opportunities. It might be more suitable for your research, and it will be more sustainable over the long term. Think like a kindergarten.
  • Reject academia altogether, at least for a short time. Go work in industry, or set up your own entrepreneurial research endeavour. Experience with how the outside world works can help enormously if you return to academia. Besides, you will know that you can survive on the outside, which will be useful when the next restructure comes around.
  • Set a time limit on how long you will bang your head against your rejection wall. Consider how long you are willing to work extremely long hours in insecure conditions, with multiple competing demands, in a cruel world. Then…
  • Leave.

Advice for granting agencies

Funding agencies are very aware of the cruelty of your world, although they probably won’t express it that way. They generally talk about the ageing population of researchers (the ever-looming ‘retirement crisis’), the rising number of applications, falling success rates and the increasing difficulty in finding enough peer reviewers.

They might create specific funding awards for early career researchers. However, given that funding levels will remain flat for the foreseeable future, they are never going to be able to put enough funds aside.

I think that the main thing that funding agencies could do is make the grant application process less onerous. This would recognise that most early career researchers have significant other duties to attend to.

  • Look to the changes in peer review for publishing and experiment with more inventive methods of review. Open peer review, voting for funds and matched crowdfunding all provide models for funding that change the process dramatically.
  • In particular, consider changing attitudes to the confidentiality of grant applications. You might choose to build a library of successful applications, or even make all your applications public. That would provide significant benefits to people learning to write good applications.
  • Funding agencies should incorporate Publons profiles into their reviewer selection process. Publons is a database of people who value the review process. Given that the base of securely employed academics is shrinking, it makes sense to turn to the people who value the review process.
  • Change the very nature of the application. Personally, I like the idea of video grant applications. I don’t think that it would be particularly easy for the applicants, but it would reduce the burden on the reviewers considerably.
  • Failing that, follow the Gates Foundation’s lead and reduce grant applications to two pages, concentrating on the core idea. Fixed budgets, double-blind review and no administrivia. You can collect all those details once you’ve awarded the grant, as part of the contracting process. This would save an enormous amount of effort across the whole system.
  • Finally, give better feedback. “After the amount of work that most grant applications entail, it’s always anticlimactic to get a ‘yes/no’ result.”

Moreover, it isn’t just up to funding agencies to address this problem. Learned academies, discipline bodies, university grants offices, research centres and national conferences could all provide training in peer review, both for grants and publications. Research is becoming more cross-disciplinary, and researchers are becoming more mobile, so we can’t depend on people learning this skill by osmosis.

In the end, more permanent positions (or fewer casual positions, which isn’t quite the same thing) is the only way to bring about significant change. Universities need to be brave and stop hollowing out the centre. Unfortunately, I don’t see that happening any time soon.

Advertisements

About Jonathan O'Donnell
Jonathan O'Donnell helps people get funding for their research. To be specific, he helps the people in the College of Design and Social Context at RMIT University in Melbourne, Australia. He has been doing that, on and off, since the 1990's (with varying degrees of success). He loves his job. He loves it so much that he has enrolled in a PhD to look at crowdfunding for research. With Tseen Khoo, he runs the Research Whisperer blog and @ResearchWhisper Twitter stream, about doing research in academia.

7 Responses to The cruel world of funding peer review

  1. Peter Bentley says:

    Thanks for the thorough and timely post.

    I often wonder if ECA/ECR’s who do not get full-time academic positions would be better off working outside academe on a 0.6-0.8 FTE basis, and dedicating the remaining 1-2 days towards research on an unpaid or contract basis for a set period of time (taking the same approach with casual teaching would be less advisable for obvious reasons). For many people with PhDs, the potential earnings outside academe will be better than an Level A/B position, leaving them no worse off financially. Many full-time academic positions at Level A/B provide less than 1-2 days per week of real research time anyway, with the exception of postdoc/RA work (but this work can also be restricted to doing other people’s research, rather than independent). Being outside academe would also not count against people in their ROPE sections. The problem is that many professional jobs outside academe are not offered on a part-time basis, but that is not the case for all positions.

    It is something to seriously consider.

    Liked by 1 person

    • Jonathan O'Donnell says:

      It is something to seriously consider, Peter. I’m a bit dubious about the ‘unpaid’ bit, but the rest makes a lot of sense to me.

      You can see this pattern in my friend Kate Bagnall’s CV.
      https://www.linkedin.com/in/katebagnall/
      She is a historian, who finished her Phd in 2006, and then worked for the National Archives, a fellowship at the National Museum, the government and as a freelancer, before landing a DECRA in 2016.

      She was full time, and then returned to work part time after maternity leave.

      Her example gives me faith that the ARC Research Opportunity Performance Evaluation (ROPE) system does help assessors to compare different CVs. It also reiterates that people don’t need to be in the university sector to be contributing to the academic world.

      Like

  2. Peter Bentley says:

    Trust a historian to take a long-term view!

    My point about the unpaid bit is that many people have PhD chapters to be written up as articles, or draft articles to be submitted to journals, etc. Taking a non-academic job but dedicating a day or two to writing up research could get this research done without much financial cost. My biggest concern with doing paid research on the side is, apart from the lack of available funding, it is very time consuming to apply/pitch/beg for the funding. If it takes a week to apply for funding and negotiate a contract, you may just be better off spending that week doing the research. The opportunity costs of seeking funding are directly evident when one is outside the academic sector.

    Like

  3. ZTF says:

    Peter – in a ideal world this might be possible, however most Unis will not give people the .6 or .8 contract that you are suggesting. There will always be that other candidate who wants the 1.0 position who will squeeze their time to do the teaching, admin, research, writing AND the grant application. In the current climate (certainly in the UK, but I suspect everywhere) the competition for jobs is fierce and no-one would deliberately disadvantage themselves. Universities are the drivers behind the push for funding so they will want the employees who are prepared to do it all.

    Like

    • Peter Bentley says:

      Hi ZTF,

      I think you misunderstood my point. I was suggesting working 0.6-0.8 outside the university sector and then use those free days to publish research. An example I often consider is working in government administration.

      I would rarely advocate to work 0.6-0.8 in the university sector because for many they will just be giving up their research day

      Like

      • Peter, there is certainly some merit in this idea, particularly for more industry-exposed disciplines where such experience is particularly valuable, and it is good to be reminded we can contribute to academic research from outside the academy. However, there are other factors to consider, too. Most importantly, those who are not currently in academic positions will often not be eligible to apply for research grants.

        We should also consider the impact of isolation from disciplinary communities and networks, not only in terms of getting research done and published, but also access to opportunities for collaboration and team building, HDR student supervision, and other aspects of track record development. It is certainly possible to build track record outside academia, as per the example above, but there are still formidable obstacles to address on this path.

        It is pleasing to see some institutions introducing short-term postdoctoral fellowships that allow recent graduates to focus on disseminating their research via conferences and publication, and building their academic skills. Where this is accompanied by active mentoring, it can provide excellent preparation for an early career researcher. We need more innovative solutions like this, and greater flexibility from institutions and funding agencies.

        Liked by 1 person

  4. Thanks for this refreshingly upfront and comprehensive post, Jonathan — it intersects nicely with issues I’m thinking and writing about at the moment, around ECR career development strategies.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: