Post-Peer-Review Open Access, Commentary and Metrics versus Post Hoc "Peer Review"

David Colquhoun (2011) is quite right on practically every point he makes: There is too much pressure to publish and too much emphasis on journal impact factors. Too many papers are published. Many are not worth publishing (trivial or wrong). Peer reviewers are overworked. Refereeing is not always reliable. There is a hierarchy of journal peer review quality. The lower levels of the quality hierarchy are practically unrefereed.

But the solution is not to post everything publicly first, and entrust the rest to post hoc "peer review," including anonymous peer review.

First, the situation is not new. Already a quarter century ago Stephen Lock (1985), former editor of the British Medical Journal, noted that everything was getting published, somewhere, in the hierarchy of journals.

And journals' positions in the hierarchy serve a purpose: Their names and public track-records for quality are important filters for potential users, helping them decide what to invest their limited time in reading, how much to trust it, and whether to risk trying to apply or build upon it. This is especially true in medicine, where it is not just researchers' time and careers that are at risk from invalid results.

Classical, prepublication peer review is answerable: The submitted paper is answerable to the referees. The referees are answerable to the editors. The editors and journal are answerable to the readership. In the higher quality journals, if revisions cannot be made to bring a paper up to its standards, it is rejected. Peer review is a means of raising paper quality, for authors, and of filtering paper quality, for users.

Self-appointed post hoc peer review is not answerable. No editor's or journal's public track record is at stake to ensure that qualified referees assess the papers, nor that their recommendations for revision are valid, heeded or followed.

And referee anonymity is a two-edged sword. Yes, it protects junior researchers and rivals from vindictiveness, but it also allows anyone to say anything about anything, immunized by anonymity. (Look at the unevenness in the quality of the comments on Professor Colquhoun's article here in the Guardian. This is not peer review.) Journal referees are anonymous to authors, but not to editors.

No, the solution is not that everything should be publicly posted, unrefereed, and then to hope that open commentary will somehow take care of the rest. The solution is to post all peer-reviewed, revised and accepted papers online, free for all (Open Access) and to allow postpublication open peer commentary (anonymous and onymous) to complement and supplement classical peer review.

And to end the arbitrary tyranny of journal impact factors (which just means the journal's average number of citations per article), let 1000 new Open Access metrics bloom -- a metric track-record, public and answerable.

Colquhoun, D. (2011) Publish-or-perish: Peer review and the corruption of science Guardian September 5 2011.

Harnad, S. (1997) Learned Inquiry and the Net: The Role of Peer Review, Peer Commentary and Copyright. Learned Publishing 11(4) 283-292.

Harnad, S. (1998) The invisible hand of peer review. Nature [Web Matters]

Harnad, S. (2003) Valedictory Editorial. Behavioral and Brain Sciences (Journal of Open Peer Commentary) 26(1): 1

Harnad, S. (2009) Open Access Scientometrics and the UK Research Assessment Exercise. Scientometrics 79 (1)

Lock, Stephen (1985) A difficult balance : editorial peer review in medicine London : Nuffield Provincial Hospitals Trust.