Research proposals are assessed by committees of experts, when it’s known that this isn’t always the fairest way. Columnist Dirk-Jan Scheffers says there’s only one solution: a lottery.
When my Spanish in-laws sent us a Christmas card, they included a tenth of a ticket for the Spanish Christmas lottery, which had a hefty first prize. Because you can only buy a tenth of a ticket, they’re often shared by entire families, neighbourhoods, or villages, which only increases the joy when a ticket wins.
Spaniards love lotteries and other games of chance. One time, we watched an ox being walked over a grid of numbered squares. The square the ox pooped on won a prize. Of course, the biggest lottery winner is whoever organises it. In the case of El Gordo it’s the Spanish government, in the case of the ox it’s the local bull club.
Right now, most funds for academic research are distributed through competition. Academic staff get judged on their capability to win these competitions. It determines whether research can be started or continued and is an important criterion whenever someone is up for promotion.
These competitions work as follows: An academic, or group of academics and other partners, write up a research proposal. This proposal is assessed by experts (peer reviewers). Sometimes, applicants are allowed to respond to the peer review, or provide extra information during an interview.
In reality, there are more great proposals than there is money
Next, a committee of other academic will assess which proposals are the most excellent. Because the reality is that there are more great proposals than there is money, many an applicant has complained that the entire process is akin to a lottery.
In the meantime, quite a bit of academic research on giving out money on the basis of excellence. It turned out that it is anything but excellent.
Committees and peer reviewers are not very good at predicting how successful an application will be, and interdisciplinary research, which everyone loves so much, actually scores lower because people aren’t very good at looking beyond the limits of their own areas of expertise. Genuinely groundbreaking proposals don’t perform very well because committees prefer things they are familiar with.
Not to mention the (subconscious) prejudices the assessors have. Women and minorities are automatically at a disadvantage, while a bad proposal by an established researcher stands a better chance of being funded. Nobel Prize winner Katalin Karikó was denied research funds for years and was even dismissed by her own university: no one believed in her idea for developing mRNA vaccines.
No committees, and no one will whine about the results
There is a simple, academically sound solution for this time-consuming and expensive circus, and that’s a lottery. After all the proposals have been checked by experts to ensure they’re okay, it’s time for the lottery.
It saves a lot of people a lot of time. You don’t need committees, and no one will whine about the results. Advantages and favouritism are eliminated. It will also save a lot of people a lot of frustration; hearing you’ve lost a lottery is different than being told your proposal was good, just not good enough.
In spite of evidence to the contrary, many academics still believe in the meritocratic funding model; occasionally winning is good for their ego. But research financiers are worried what people will think when they start ‘gambling’ with the funds they’ve been trusted with.
That’s a shame, because even the former NWO chair has called these competitions a lottery. Perhaps they should put their money where their mouth is.
DIRK-JAN SCHEFFERS