We recently carried out an online survey of current peer review practice.
We got 200 replies, representing many more than 200 journals (some
respondents were multi-journal editors, or publishers). You can find the
results at www.alpsp.org/pub4.htm
Sally
Sally Morris, Secretary-General
Association of Learned and Professional Society Publishers
South House, The Street, Clapham, Worthing, West Sussex BN13 3UU, UK
Phone: 01903 871686 Fax: 01903 871457 E-mail: sec-gen_at_alpsp.org
ALPSP Website
http://www.alpsp.org
Learned Publishing is now online, free of charge, at
www.learned-publishing.org
----- Original Message -----
From: "Stevan Harnad" <harnad_at_coglit.ecs.soton.ac.uk>
To: <AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM_at_LISTSERVER.SIGMAXI.ORG>
Sent: 26 January 2001 14:05
Subject: Re: ePrint Repositories [+ Peer Review]
> On Fri, 26 Jan 2001, Simon Buckingham Shum wrote:
>
> > I disagree with your terminology. Ask any academic what
> > "classical peer review" is, and they will tell you about the
> > "anonymous reviewer, no right of reply for author" process.
>
> The one to ask is not "any academic" but journal editors. You will find
that
> most journals offer referee anonymity as an option (which can be waived by
the
> referee, and often is). And most competent editors will mediate, even for
> several rounds, if there is a substantive basis for interaction between
authors
> and referees.
>
> It is only to some (disappointed) authors that the system sometimes
> looks rigid, conspiratorial and biassed: In reality, editors would like
> to publish the best papers, and will do whatever it takes to make sure
> they do, including following up author/referee disagreements where it
> looks as if it is warranted.
>
> If you remove the option of referee anonymity you invite more bias than
> you remedy: How is one to do an honest negative review (if that is what
> is called for) of someone more senior than oneself, or someone who may
> be refereeing one's next grant proposal? The short answer to why
> referees need the anonymous option is: for exactly the same reason we
> vote anonymously in a democracy: to make sure we can exercise our
> choice without fear or favor. Sure, this can be abused in the other
> direction too, but that's what a competent editor is there for: the
> reviewer is never anonymous to the editor.
>
> > Our conversational model places author-reviewer interaction at the
> > centre right from the start, makes this 'intellectual trace'
> > accessible to wider peers, and preserves an edited, open version of
> > it with the publication.
>
> I misunderstood about referee anonymity then; I thought anonymity was
> possible within the "closed" phase of author/referee interaction,
> mediated by the editor, in your system: If referee anonymity is not
> possible, then yours is not classical peer review. If the interactions
> are, in addition, visible not only to the author/editors/referees but
> to the wider community as well, then this your system is still further
> removed from classical peer review, and needs empirical testing to
> determine the resultant quality it delivers before it can be declared
> ready for adoption.
>
> Note that the archiving of the successive embryological phases of
> preprint/revision/postprint provides the "intellectual trace" you
> desire, even with classical peer review (plus self-archiving of all
> those embryological stages, if the author wishes), but it leaves that,
> rightly, voluntary. The same is true with the archiving of referee
> reports, comments, responses, and postpublication revisions: These too
> are possible in the classical system, but voluntary.
>
> As soon as you make variants that could have substantive effects on
quality
> (e.g., no anonymity option, public review) mandatory, you have to test and
> demonstrate what effects that has on quality; till then, everyone is best
> advised to stick with tried-and-tested classical peer review.
>
> > BBS has implemented commentary and
> > author response as best it could in the paper medium, but as Stevan
> > himself has argued, the net enables a more rapid tempo for scholarly
> > discourse. This is simply not practical in paper.
>
> Correct, and that is why Psycoloquy was founded in 1989.
>
> > CogPrints restricts this benefit to "pre-journal" status
> > contributions.
>
> CogPrints is not a journal but an archive. Any embryological phase can
> be self-archived therein, be it pre-refereeing preprint, revised draft,
> refereed final draft, postpublication update, referee report,
> commentary, or response -- as long as the author of each type of item
> chooses to self-archive it.
>
> > JIME gives it
> > first class status as a mode of discourse suitable for journal
> > submissions, particularly in a new interdisciplinary context. The
> > context of use (disciplinary factors and user expertise may be
> > variables that determine success of more radical models).
>
> But we will need a few years to find out exactly what the quantity and
> quality of the papers undergoing any new, untested system like this
> will actually turn out to be. Till then, no conclusions can be drawn,
> and certainly no new system should be adopted.
>
> > We are certainly agreed that there are no proven substitutes for
> > organised peer review. (Future technologies may appear to provide
> > automatic analyses of quality, but you can't beat humans.) To make
> > review happen, you need to engage reviewers in a social contract. To
> > make this happen requires some form of coordination, up until now the
> > role of the journal.
> >
> > However, it is not inconceivable that the net will enable
> > decentralised coordination of some sort, which is able to create the
> > social contract to review (or provide some other kind of incentive?),
> > without any notion of a Journal or other Learned Body to establish
> > the contract. Sort of Napster peer review model.
>
> Not inconceivable, but until demonstrated, this is merely untested
> speculation.
>
> > But what's the
> > payback for the reviewer? Short of paying reviewers, I don't know of
> > anyone who's successfully devised other motivations yet.
>
> Paying reviewers is yet another untested and potentially corruptive
> variant on classical peer review.
>
> --------------------------------------------------------------------
> Stevan Harnad harnad_at_cogsci.soton.ac.uk
> Professor of Cognitive Science harnad_at_princeton.edu
> Department of Electronics and phone: +44 23-80 592-582
> Computer Science fax: +44 23-80 592-865
> University of Southampton http://www.ecs.soton.ac.uk/~harnad/
> Highfield, Southampton http://www.princeton.edu/~harnad/
> SO17 1BJ UNITED KINGDOM
>
> NOTE: A complete archive of the ongoing discussion of providing free
> access to the refereed journal literature online is available at the
> American Scientist September Forum (98 & 99 & 00 & 01):
>
> http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
>
> You may join the list at the site above.
>
> Discussion can be posted to:
>
> american-scientist-open-access-forum_at_amsci.org
Received on Wed Jan 03 2001 - 19:17:43 GMT