Research

Leaving the Frontiers

On 14 August 2014 I was invited to become an associate editor (AE) of the Open-Access Frontiers in Human-Computer Interaction journal (part of the 'Nature' publishing group)

I support Open Access, and while I was curious what this collaboration with Nature could do for it, I have become increasingly uneasy being associated with Frontiers. Nature itself documented how the journal is now added to Beal's List of questionable publishers. There were significant issues with the editorial independence of Frontier's medical section, written up well by Leonid Scheider. The same author also penned a piece describing the problems with interns at Frontiers.

But in particular, I feel the peer review process is toothless and that it is entirely setup to effectively act as a pay-to-publish model. That some of Frontier's sections also publish good papers is a matter of luck, in my opinion. I only supported the review process of a couple of papers, mostly because almost none of the papers submitted to this journal were close to my area of expertise. But in both cases I felt there was no way in which the reviewers' comments could ever lead to the paper not being accepted. In fact, the entire peer review process, including its web-interface, seems geared towards slowly but surely progressing a manuscript to publication. The official Associate Editor guidelines issued by Frontiers say as much: reviewers should not gauge a paper on the basis of 'novelty', but on 'correctness'. The idea is that posterity will signify a paper's significance, presumably measured by e.g. citation counts. This basically means that anything can get published, as long as you correct your errors and persevere. The insistence that reviewers respond to each new comment of authors turns it into a match of who gives up first. Persistent authors will always be able to publish. Provided they pay Frontiers, of course.

While there are many things wrong with the standard peer review process, like Democracy, I think it's the least bad system we have. Yes, it's based on a limited number of experts. And yes, the process is unduly lenghty, un-interactive and the comments and demands of reviewers (and AEs) can sometimes be downright petty. But, in the end, I believe that it provides readers iwth a solid form of trust that a paper is correct, and worthwhile to read. Good journals will be able to call upon better reviewers and AEs, and as such the whole process is more solid for the more elite journals as well, which is also good.

In this new era of populism and fake news stories circulating on social media, I believe going down the Frontiers route that basically says 'if it's mentioned/cited/liked by people, it must be good' is simply the wrong way to go. I believe in the value of expert reviews, and I believe in the power of a trust-based society. Therefore I am today standing down as an Associate Editor for Frontiers.

The problem with monetarising academic science

On Tuesday the 24th of May, I attended the Royal Institution’s event titled ’Science careers: has the science establishment let down young researchers?’. Dr Evan Harris chaired a panel discussion on careers in science at the Royal Institution. On the panel were Dr Jenny Rohn and Professor Dame Athene Donald, FRS, and they were joined by Rt Hon David Willetts MP, Minister of State for Universities and Science. The event was masterfully led by Dr Evan Harris, who made use of the late running of the minister to make a survey of what topics the audience wanted to discuss, and to make a selection of what questions and opinions should definitely be put before Willetts.

Many issues were raised, ranging from the two body problem induced by the pressure on postdocs to gather experience at different universities, to the leaky pipe syndrome experienced by female scientists for whom the lack of job security is a problem if they are planning for a family. Perhaps the biggest problem for young scientists to advance their career, i.e. to become a PI, is the current structure of the scientific community. The ratio of PIs to postdocs and PhDs is sometimes described as a pyramid, but as Dr Jenny Rohn put it, it’s more like a big spike in the middle of a flat surface. That is a nice property for a signal in machine learning, but it’s not a desirable property for your career prospects. A recent study by the RA showed that only 4% of PhDs ended up in an academic position. The minister accepted that such problems existed, but made it clear that in the government’s opinion such problems should be addressed by the academic community, and not regulated by government.

While I agree with Willetts that it is the academic community that needs to change its ways, I do think there’s a great role for the government to play, albeit not necessarily by setting up new laws and regulations. But I will get back to that at the end of my discussion. In my opinion, the fundamental problem is with how academic research is monetarised. That is, how do we value academic research and how do we then proceed to distribute research funding. Funding requests are proposed by an individual or a small group of individuals. At the moment, the prevailing measure for research quality of these individuals is their publication record. Someone with a lot of publications and citations must be a good researcher, and is thus worthy of the funding requested.

This mentality, if not properly constrained, generates groups where a single PI hires as many PhDs and PostDocs as possible, who do the actual research described in the funding proposals, and publish papers on which the PI is invariably a co-author. This increases the research quality of the PI, resulting in more funding, resulting in more PhDs and PostDocs, resulting in more papers... you get the picture. There is no incentive in this mechanism not to have a big group of low-paid eager employees led by a single PI. It simply works.

So what’s wrong with this picture? A few things, I think. While this mechanism may function, it does not necessarily generate the best possible research for the money spend, and may thus not be in the best interest of the country and the government. Why not? First of all, both PhDs and PostDocs are supposed to be in training by the PI. PhDs more so than PostDocs, but even they are supposed to receive training as a PostDoc is supposed to be a preparatory stage before becoming a lecturer. It’s not hard to see that the more spiky the employment structure, i.e. the more PhDs/PostDocs a PI has, the lower the quality of this training, and thus the lower the quality of the resulting research. Spiky structures also mean low career prospects, and this has a huge demoralising effect on the very people expected to conduct the actual research. Yet there is no incentive in the current monetarising mechanism to limit the number of PhDs/Postdocs for each PI, in fact, the opposite is encouraged.

Secondly, it is reasonable to assume that the best researchers end up being PIs, eventually. Yet in the current system, all their time is consumed in writing grant proposals, and managing their research groups. This is a waste of their potential: they are supposed to be the best researchers available to the community yet they cannot perform any actual bench work! One argument often heard against this is that they at least still generate the ideas that form the basis for grant proposals. Yet it is a well known fact that proposals do not consist of entirely novel ideas. Because of the harsh reviews governments and research councils put in place these days, PIs promise to research things that have already been attained, or are at least within short reach. So many research proposals are not stellar new proposals for groundbreaking research, but bland continuations of existing research, designed to have easily evaluated performance measures to suit the bureaucratic reviews.

And that forms also the basis for the third issue of spiky research groups. While PostDocs are supposedly independent researchers, mentored by their professor, in reality they are performing the research set out by their PI and formulated in part by the grant proposals. This means that they are not fully able to pursue their own ideas, which can be truly paradigm shifting, and again a lot of potential research excellence is lost.

So, how can we go about changing this in a structured manner, that is, by changing the actual mechanism of monetarising academic research? I see two things that can be done by the scientific community. First of all, universities can set a limit to the number of PhDs/PostDocs that a PI can have, based on their obligation to properly supervise. This would ensure that PhDs and PostDocs are properly trained, and that would in turn improve their research output. Universities may be afraid of this because it would limit their star funding attractors, but this is not an objection if they would increase the ratio of PIs to PostDocs, which is the principal problem we are addressing anyway, a problem which I hope the universities acknowledge exists.

Secondly, universities and funding bodies should make it clear that it is ‘not done’ to be a co-author on a paper merely on the merit of having procured the funding for a subset of the authors. Some funding bodies in other countries (e.g. Germany) already make this explicit in their funding rules, and increasingly prominent journals and conferences ask the authors to declare that all authors have made a significant contribution to the research described in the paper (not the research funding). If these guidelines were upheld, they would force PIs to spend more time on actual research for fear of losing their measured research quality, which would mean they would have less time to manage huge groups and write loads of funding proposals, which in turns makes place for more PIs with better structured (i.e. less spiky) groups.

One obstacle remains: recently there has been a push by funding bodies to reduce the number of grant proposals coming in, as a cost cutting measure. The proposals I made above will both lead to an increase of proposals, at least there will be more proposals by more people. This is a good thing though, and the fallacy of reducing funding requests must be tackled. This is one thing where Willetts can act: provide a larger percentage of the budget to the research councils for dealing with grant proposals.

The other way in which Rt. Hon. Willetts can influence this process is by influencing universities and research councils that a de-spiking of the research community structure is a good thing, and that the ideas outlined above should be supported.