The cruel world of funding peer review

This article began life as a presentation for Peer review and grant funding: From evidence to practice at Melbourne University, 17 November 2017. Thanks to Adrian Barnett and Philip Clarke for inviting me to speak.

---

Before I begin, I should point out that I write from a position of incredible privilege. I’m not an academic – I’m a university administrator. I am securely employed, and have been for most of my working life. My job is to help academics find funding for their research.

In that role, I work with Australian academics from RMIT University. I work with artists, designers, educators, social scientists and humanities scholars, primarily on their Australian Research Council applications. A significant number of the academics that I have worked with over the last seven years have been early career researchers, generally trying to win their first major grant.

Early career researchers face a cruel world these days. Even though they are an increasingly diverse cohort, they are still generally imagined as young, full-time academics without significant outside commitments. They aren’t. Many of them have significant responsibilities outside of work, taking care of children and elderly parents or working on limited visas, far from home. Most of them have no secure work, while being expected to take on increasing levels of accountability. Their research outputs, their teaching performance and even their scholarly engagement with the world are under intense scrutiny and evaluation.

All this has interesting ramifications for the peer review system that we use for government grants.

Beware the Profzi Scheme, on “Piled Higher and Deeper” by Jorge Cham http://www.phdcomics.com

The cruel world

Casualisation of the workforce means that most early career researchers are on similar employment contracts to supermarket employees. Those that have contracts (as opposed to casual employment), often find themselves stuck at the lowest levels of the academic strata (level A and B in Australian university employment terms) on rolling 12- or 18-month contracts. The number on three-year fellowships is vanishingly small.

Static or falling government funding and industry support for research, along with increasing expectations on ‘academic superheroes‘, means that research funding is increasingly competitive. Most of the funding goes to academics in secure employment (permanent or tenured employees), who have often been in the academic workforce for some time.

Taken together, this means that we are hollowing out the university research system. It just keeps getting worse. Fewer and fewer people have the secure employment that allows them to lead significant research grants. Those that are secure in the system are facing static or falling funding envelopes, which often means smaller individual grants.

The government’s Impact Agenda is changing (or at least confusing) what type of research that academics feel that they should undertake. My own university has had a long history of applied research and education. It is now returning to that focus, having spent the greater part of the last two decades encouraging staff to compete for less applied, more ‘pure’ funding schemes.

This is particularly confusing for early career academics. I’m seeing applications where people are over-emphasising the value of their research to government agendas, and making wild claims about the potential impact that their work will have.

On top of all this, the ‘Metric Tide‘ of academic scrutiny means that they must make every single thing count, because every single thing that they do is being counted. As a result, early career researchers are either becoming disillusioned, or being turned into highly focused research machines. Neither is healthy for strong, curiosity-driven research.

Advice for individuals

So, what should an early career researcher do in such a cruel world? I don’t know. What I do know is that none of the suggestions below will do you any harm.

Here are some things that you can do as a reviewer:

• Learn how the system works. Register to review for granting agencies. If you are dreaming of working in another country in the future, register to review their grants, too. Granting agencies are often hungry for international reviewers. Being a reviewer gives you the best education that you can get in what makes a competitive funding application, and it is good for the system overall, too.
• Because most funding agencies don’t pay for reviews (or provide a token payment), you should set clear limits, both in how many reviews you will do, and how long you will spend on each one. Keep in mind that the funding agencies will want a relatively quick turn-around.
• Since you live in a world of measurement, record your effort. Publons provides academics with a service to track their peer review contributions. If you don’t want to be that public, consider using an academic log to track your own work.
• Don’t just learn by doing. As with any activity like this, seek out mentoring, including peer mentoring. I think that mentoring is particularly important for early career female academics. It isn’t just a cruel world – it’s a gendered cruel world.
• Look for advice. A lot of advice on peer reviewing for publications applies to grant reviewing, too.
• Be kind (or at least be the change that you want to see). It doesn’t have to be a cruel world.

Here are some things that you can do as an early career academic, competing for funding in this cruel world:

• Choose balance. Don’t let unrelenting pressure to ‘do more’ ruin your passion.
• Build a rejection wall. Peer review has two sides, reviewing and rejection. A rejection wall will help you cope with the inevitable rejection that comes with peer review.
• Consider writing ONE grant application for someone else. I hesitate to recommend this, because so many people find this an abusive situation. I think that it can be useful, but only under the following conditions.
  • Only ever write a grant application for someone else as a learning exercise (hence, only ever do it once).
  • Only write it for someone who has enormous experience at winning grants, and is willing to share that experience.
  • Only write it if there is some clear advantage to you, such as being written into the budget (hello, three-year contract!) or being listed on the front page.
• Let other people do the work. Not everybody has to lead. Not everybody can lead. At this stage of your career, being a collaborator in a successful grant is really, really useful, and probably more realistic than leading one.
• Reject the peer review system altogether. There is a world of funding that isn’t peer reviewed. Industry funding, philanthropic funding, crowdfunding, micro-patronage… Get creative with your funding opportunities. It might be more suitable for your research, and it will be more sustainable over the long term. Think like a kindergarten.
• Reject academia altogether, at least for a short time. Go work in industry, or set up your own entrepreneurial research endeavour. Experience with how the outside world works can help enormously if you return to academia. Besides, you will know that you can survive on the outside, which will be useful when the next restructure comes around.
• Set a time limit on how long you will bang your head against your rejection wall. Consider how long you are willing to work extremely long hours in insecure conditions, with multiple competing demands, in a cruel world. Then…
• Leave.

Advice for granting agencies

Funding agencies are very aware of the cruelty of your world, although they probably won’t express it that way. They generally talk about the ageing population of researchers (the ever-looming ‘retirement crisis’), the rising number of applications, falling success rates and the increasing difficulty in finding enough peer reviewers.

They might create specific funding awards for early career researchers. However, given that funding levels will remain flat for the foreseeable future, they are never going to be able to put enough funds aside.

I think that the main thing that funding agencies could do is make the grant application process less onerous. This would recognise that most early career researchers have significant other duties to attend to.

• Look to the changes in peer review for publishing and experiment with more inventive methods of review. Open peer review, voting for funds and matched crowdfunding all provide models for funding that change the process dramatically.
• In particular, consider changing attitudes to the confidentiality of grant applications. You might choose to build a library of successful applications, or even make all your applications public. That would provide significant benefits to people learning to write good applications.
• Funding agencies should incorporate Publons profiles into their reviewer selection process. Publons is a database of people who value the review process. Given that the base of securely employed academics is shrinking, it makes sense to turn to the people who value the review process.
• Change the very nature of the application. Personally, I like the idea of video grant applications. I don’t think that it would be particularly easy for the applicants, but it would reduce the burden on the reviewers considerably.
• Failing that, follow the Gates Foundation’s lead and reduce grant applications to two pages, concentrating on the core idea. Fixed budgets, double-blind review and no administrivia. You can collect all those details once you’ve awarded the grant, as part of the contracting process. This would save an enormous amount of effort across the whole system.
• Finally, give better feedback. “After the amount of work that most grant applications entail, it’s always anticlimactic to get a ‘yes/no’ result.”

Moreover, it isn’t just up to funding agencies to address this problem. Learned academies, discipline bodies, university grants offices, research centres and national conferences could all provide training in peer review, both for grants and publications. Research is becoming more cross-disciplinary, and researchers are becoming more mobile, so we can’t depend on people learning this skill by osmosis.

In the end, more permanent positions (or fewer casual positions, which isn’t quite the same thing) is the only way to bring about significant change. Universities need to be brave and stop hollowing out the centre. Unfortunately, I don’t see that happening any time soon.