Open Access Archivangelism

Here is an important message from the American Alliance for Taxpayer Access, posted in Peter Suber's Open Access News, about the need for US citizens to contact their senators in order to support the NIH Green OA Self-Archiving Mandate against the anti-OA publishing lobby:

Added: Charles Bailey has just provided a generic text that you need merely cut/paste as your letter.

---

   Time to contact the Senate

An alert from the Alliance for Taxpayer Access:

As the Senate considers Appropriations measures for the 2008 fiscal year this fall, please take a moment to remind your Senators of your strong support for public access to publicly funded research and ­ specifically ­ ensuring the success of the National Institutes of Health (NIH) Public Access Policy by making deposit mandatory for researchers.

Earlier this summer, the House of Representatives passed legislation with language that directs the NIH to make this change.  The Senate Appropriations Committee approved a similar measure.  Now, as the Appropriations process moves forward, it is critically important that our Senators are reminded of the breadth and depth of support for enhanced public access to the results of NIH-funded research. Please take a moment to weigh in with your Senator now.

Contact information for your Senator is here. Please fax a letter with your support no later than Friday, September 28, 2007.

Feel free to draw upon the following talking points:

• American taxpayers are entitled to open access on the Internet to the peer-reviewed scientific articles on research funded by the U.S. government. Widespread access to the information contained in these articles is an essential, inseparable component of our nation's investment in science.
• The Fiscal Year 2008 Labor/HHS Appropriations Bill reported out of committee contains language directing the National Institutes of Health (NIH) to change its Public Access Policy so that it requires NIH-funded researchers to deposit copies of agency-funded research articles into the National Library of Medicine's online archive.
• Over the more than two years since its implementation, the NIH's current voluntary policy has failed to achieve any of the agency's stated goals, attaining a deposit rate of less than 5% by individual researchers. A mandate is required to ensure deposit in NIH's online archive of articles describing findings of all research funded by the agency.
• We urge the Senate to support the inclusion of language put forth in the Labor/HHS Appropriations bill directing the NIH to implement a mandatory policy and ensuring free, timely access to all research articles stemming from NIH-funded research ­ without change ­ in any appropriate vehicle.

(We'll be making additional resources for patient advocates ­ including the recording of our August 30 Web cast and specific talking points ­ available shortly as well. Watch the ATA Web site or email me directly for updates.)

Again, please take a moment to express your support for public access to research to your Senator as soon as possible and no later than September 28. As always, we'd appreciate it if you'd let us know of what action you're able to take, or send a copy your letter to the ATA through (202) 872-0884 (fax). Thank you!

Comments by Peter Suber:

Publisher associations are lobbying hard against this bill.  For example, the AAP/PSP launched PRISM, the behemoth Copyright Alliance weighed in, and Elsevier hired another lobbying firm.  It's critical that we show the Senate our support for a stronger OA policy at the NIH.  If you're a US citizen, please contact your Senators and spread the word to others who could do so.

I've omitted the list of Senators with their fax numbers, but it's in the ATA message if you need it.  For other kinds of contact info for your Senators (DC office, DC phone, local offices, local phones, email), use CongressMerge.  Remember to act before September 28!

Peter Suber

---

---

SUMMARY: There are many valid arguments for Open Access (OA), but the claim that researchers or their universities are currently "double-paying" to "buy back" access to their own research output -- by (1) paying for the conduct of the research, giving it to journals, and then (2) paying for subscriptions to access it -- is invalid. Researchers already have their own results, and so does their own university. What their university library subscriptions are paying for is to buy-in (not buy-back) the research output of other universities, exactly as with books written by authors from other universities. (Some research, universities and university libraries are funded by public funds, some not. Unless OA is to be limited to only that subset, that is not a sufficient rationale for OA either.)
   There is a valid "double-payment" argument, though, when it is not the payer (the researcher, funder, or university) paying twice, but the payee -- the publisher -- getting paid twice (not necessarily by the same payer): This is the "Trojan Horse" of paying a "hybrid Gold OA" publisher (i.e., not a pure Gold OA publisher but a subscription-based publisher who offers the option of making individual articles OA in exchange for a fee) at a time when the potential funds for doing so are still tied up in the university subscriptions that are already paying that publisher's costs in full.
   Publishers need only be paid once. If mandated Green OA should ever make cost-recovery from subscription payments unsustainable (because it makes university subscription demand disappear), then the resultant university subscription windfall savings can be redirected to pay for the peer review on the Gold OA model, with all access-provision and archiving off-loaded onto the distributed network of university OA Repositories.

---

There are many valid arguments for Open Access (OA): It maximizes research usage and impact, it enhances research and university visibility as well as researcher and university income, it accelerates national and international R&D productivity and progress, it helps developing countries, it helps inform the general public, it is useful for education and training, it might eventually reduce university library subscription costs.

But the widespread claim that researchers or their universities are currently "double-paying" to "buy back" access to their own research output -- by (1) paying for the conduct of the research, giving it to journals, and then (2) paying for subscriptions to access it -- is both invalid and unnecessary.

If you ask those who are making this argument exactly what they mean, you will get two different answers, neither of them coherent or defensible:

(1) "I am salaried, and government-funded, to do my research. I give the report of my research findings to my publisher for free. Then I or my university must buy a subscription to the journal in order to access my results."

This is incorrect. You already have your own research results, and so does your university. What your university library subscriptions are paying for is to buy in the research output of other universities. And the ones to whom your publisher is selling your research are other universities, not your own.

This is no more "buy-back" or "double-paying" than it is to pay for the books written by authors from other universities. Or any other output that the university both itself generates and also buys in from other universities.

Nor does the fact that books sometimes generate royalty revenue change this picture: Most books generate negligible royalty revenue or none at all.

(It is relevant however -- not as a buy-back/double-pay argument, but as a conflict of interest -- that researchers, unlike other authors, ask for no royalty income from their publisher, and publish their articles purely for the sake of their research impact, not their royalty income. Hence this royalty-free author give-away writing is not a "work for hire," and publishers are unjustified in trying to prevent or embargo its authors' attempts to maximize access to it by self-archiving it free for all online.)

(2) "My research is funded by public funds; so is my university, and so is my university's library. Hence it is somehow buy-back/double-pay for them to be paying for subscription access to it too."

This no longer sounds like buy-back/double-pay but something much more complicated, with some grains of truth. But those grains of truth are only being buried or distorted in calling this buy-back/double-pay. Not only is a lot of published research not publicly funded, not only are neither university researcher salaries nor institutional library budgets all or wholly derived from public funds, but again, much the same argument could be made for books and other university outputs (even commercial ones) if the underlying problem were indeed the buy-back/double-payment one.

So unless you think that OA should only be accorded to research if -- and in proportion to the degree to which -- public funders or universities are literally drawing twice from the same pot, you may be narrowing OA to a smaller and more difficult-to-determine subset of OA's real target, which is all of peer-reviewed research. (And you may also be making an argument against either the public funding of R&D or the selling of its output: That may be arguable, but it is a far, far bigger argument than OA!)

There is, however, one "double-payment" argument that is valid, but it is not the one the buy-back/double-payment complainants have in mind: rather the opposite. For in this valid double-payment argument it is not the payer (the researcher, his funder, or his university) who is paying twice, but the payee -- the publisher -- who is getting paid twice (but not necessarily by the same payer)

I am referring to the "Trojan Horse" of paying an extra fee to a "hybrid Gold OA" publisher (i.e., not a pure Gold OA publisher but a subscription-based publisher who offers the option of making individual articles OA in exchange for a fee) at a time when the potential funds for paying for that fee are still tied up in the university subscriptions that are already paying that publisher's costs. Hence universities or funders who opt for optional-Gold in a hybrid OA journal today must find extra funds to pay that optional-Gold OA fee (e.g., by redirecting money from research funds), even while all publishing costs are still being fully covered by subscriptions.

But this is not the kind of "double-payment" that the buy-back/double-pay complainants are agitating against: On the contrary, this is the kind that they are (prematurely, and unwittingly) agitating for.

The resolution of the "double-payment" puzzle is very simple: Publishers need only be paid once. Today they are mostly being paid (once, via subscriptions), for providing both (1) the peer-review service and (2) the paper and online editions. As long as the subscription market covers the costs, the only thing researchers need do for immediate OA is to self-archive the final, peer-reviewed drafts of their own published articles in their own university OA Repositories (Green OA); and the only thing their universities and funders need do is to mandate that they do so.

If and when Green OA should ever make cost-recovery from subscription payments unsustainable (because university subscription demand disappears), then the resultant university subscription windfall savings can be redirected to pay for the peer review (on the Gold OA cost-recovery model). And both the paper and online editions, for which there is (ex hypothesi) no longer any demand at that time, can be terminated (along with their expenses), off-loading all access-provision and archiving onto the distributed network of university OA Repositories that has already assumed the access-provision role de facto, in virtue of providing Green OA.

That's a tad more complicated than "buy-back/double-pay," but it is coherent, reflective of the facts and factors (actual and hypothetical) involved, and leads to the same outcome: 100% OA (first Green, and perhaps eventually Gold), but without any fuzziness, double-talk, or unsupportable arguments.

Stevan Harnad
American Scientist Open Access Forum

---

Update Jan 1, 2010: See Gargouri, Y; C Hajjem, V Larivière, Y Gingras, L Carr,T Brody & S Harnad (2010) “Open Access, Whether Self-Selected or Mandated, Increases Citation Impact, Especially for Higher Quality Research”
Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

---

SUMMARY: Kurtz & Henneken (2007) report that the citation advantage of astrophysics papers self-archived as preprints in Arxiv is caused by (1) Early Advantage (EA) (earlier citations for papers self-archived earlier) and (2) Quality Bias (QB) (a self-selection bias toward self-archiving higher quality papers) and not by (3) Open Access (OA) itself (being freely accessible online to those who cannot afford subscription-toll access).
   K & H suggest: "[T]here is no 'Open Access Advantage' for papers from the Astrophysical Journal" because "in a well funded field like astrophysics essentially everyone who is in a position to write research articles has full access to the literature."
   This seems like a perfectly reasonable explanation for K&H's findings. Where there is no access problem, OA cannot be the cause of whatever higher citation count is observed for self-archived articles.
   We (Hajjem & Harnad 2007) have conducted a similar study, but across the full spectrum of disciplines, measuring the citation advantage for mandated and unmandated self-archiving for articles from four Institutional Repositories that have self-archiving mandates, each compared to articles in the very same journal and year by authors from other institutions (on the assumption that mandated self-archiving should have less of a self-selection Quality Bias than unmandated self-archiving).
   We again confirmed the citation advantage for self-archiving, and found no difference in the size of that advantage for mandated and unmandated self-archiving. It is likely that the size of the access problem differs from field to field, and with it the size of the OA citation advantage. Evidence suggests that most fields are not nearly as well-heeled as astrophysics.

---

Kurtz & Henneken (2007) report a very interesting new result:

"We demonstrate conclusively that there is no 'Open Access Advantage' for papers from the Astrophysical Journal. The two to one citation advantage enjoyed by papers deposited in the arXiv e-print server is due entirely to the nature and timing of the deposited papers. This may have implications for other disciplines."

Earlier, Kurtz et al. (2005) had shown that the lion's share of the citation advantage of astrophysics papers self-archived as preprints in Arxiv was caused by (1) Early Advantage (EA) (earlier citations for papers self-archived earlier) and (2) Quality Bias (QB) (a self-selection bias toward self-archiving higher quality papers) and not by (3) Open Access (OA) itself (being freely accessible online to those who cannot afford subscription-toll access).

Kurtz et al. explained their finding by suggesting that:

"in a well funded field like astrophysics essentially everyone who is in a position to write research articles has full access to the literature."

This seems like a perfectly reasonable explanation for their findings. Where there is no access problem, OA cannot be the cause of whatever higher citation count is observed for self-archived articles. Moed (2007) has recently reported a similar result in Condensed Matter Physics, and so have Davis & Fromerth (2007) in 4 mathematics journals.

Kurtz & Henneken's latest study confirms and strengthens their prior finding: They compared citation counts for articles published in two successive years of the Astrophysical Journal. For one of the years, the journal was freely accessible to everyone; for the other it was only accessible to subscribers. The citation counts for the self-archived articles, as expected, were twice as high as for the non-self-archived articles. They then compared the citation-counts for non-self-archived articles in the free-access year and in the toll-access year, and found no difference. They concluded, again, that OA does not cause increased citations.

But of course K&H's prior explanation -- which is that there is no access problem in astrophysics -- applies here too: It means that in a field where there is no access problem, whatever citation advantage comes from making an article OA by self-archiving cannot be an OA effect.

K&H conclude that "[t]his may have implications for other disciplines."

It should be evident, however, that the degree to which this has implications for other disciplines depends largely on the degree to which it is true in other disciplines that "essentially everyone who is in a position to write research articles has full access to the literature."

We (Hajjem & Harnad 2007) have conducted (and are currently replicating) a similar study, but across the full spectrum of disciplines, measuring the citation advantage for mandated and unmandated self-archiving for articles from 4 Institutional Repositories that have self-archiving mandates (three universities plus CERN), each compared to articles in the very same journal and year by authors from other institutions (on the assumption that mandated self-archiving should have less of a self-selection quality bias than unmandated self-archiving).

Figure 1. Self-Selected Self-Archiving vs. Mandated Self-Archiving: Within-Journal Citation Ratios (for 2004, 4 mandating institutions, all fields). S = citation counts for articles self-archived at institutions with (Sm) and without (Sn) a self-archiving mandate. N = citation counts for non-archived articles at institutions with (Nm) and without (Nn) mandate (i.e., Nm = articles not yet compliant with mandate). Grand average of (log) S/N ratios (106,203 articles; 279 journals) is the OA advantage (18%); this is about the same as for Sn/Nn (27972 articles, 48 journals, 18%) and Sn/N (17%); ratio is higher for Sm/N (34%), higher still for Sm/Nm (57%, 541 articles, 20 journals); and Sm/Sn = 27%, so self-selected self-archiving does not yield more citations than mandated (if anything, it is rather the reverse). (All six within-pair differences are significant: correlated sample t-tests.)
(NB: preliminary, unrefereed results.)

We again confirmed the citation advantage for self-archiving, and found no difference in the size of that advantage for mandated and unmandated self-archiving. (The finding of an equally large self-archiving advantage for mandated and unmandated self-archiving was also confirmed for CERN, whose articles are all in physics -- although one could perhaps argue that CERN articles enjoy a quality advantage over articles from other institutions.)

A few closing points:

(1) It is likely that the size of the access problem differs from field to field, and with it the size of the OA citation advantage. Evidence suggests that most fields are not nearly as well-heeled as astrophysics. According to a JISC survey, 48% of researchers overall (biomedical sciences 53%, physical/engineering sciences 42%, social sciences 47%, language/linguistics 48% and arts/humanities 53%) have difficulty in gaining access to the resources they need to do their research. (The ARL statistics on US university serials holdings is consistent with this.) The overall access difficulty is roughly congruent with the reported OA access advantage.

(2) In Condensed Matter physics (and perhaps also mathematics) there is the further complication that it is mostly just preprints that are being self-archived, which reduces the scope for observing any postprint advantage, as opposed to just an Early Advantage (EA). (In fields that do have a significant access problem, the "Early Advantage" for postprints is the OA Advantage!) OA self-archiving mandates (and the OA movement in general) target refereed, accepted postprints, not unrefereed preprints.

(3) The positive correlation is between citation counts and the number of self-archived articles in each citation-range (Hajjem et al 2005). This could be caused by Quality Bias (QB) (higher quality articles being more likely to be self-archived) or Quality Advantage (QA) (higher quality articles benefiting more from being self-archived) or both. (The top 50% of articles tend to be cited 10 times as often as the bottom 50% [Seglen 1992]. This has nothing to do with OA or self-archiving.)

(4) Citations are of course not the only potential advantage of self-archiving. Downloads can increase too (Brody et al. 2006).

(5) We are now in the Institutional Repository (IR) era. Arxiv is a venerable Central Repository, one of the biggest and oldest and most successful of them all. But, institutions are the sources of OA content, institutions are in the best position to mandate self-archiving, and institutions share with their authors the benefits of enhancing the accessibility, usage and impact of their research output. Moreover, the nature of the web is distributed local websites, harvested by central service providers, rather than central websites.

Stevan Harnad

Brody, T., Harnad, S. and Carr, L. (2006) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 57(8) pp. 1060-1072.

Davis, P. M. and Fromerth, M. J. (2007) Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles? Scientometics, Vol. 71, No. 2.

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47.

Hajjem, C. and Harnad, S. (2007) The Open Access Citation Advantage: Quality Advantage Or Quality Bias? Technical Report, Electronics and Computer Science, University of Southampton.

Kurtz, M. J. and Henneken, E. A. (2007) Open Access does not increase citations for research articles from The Astrophysical Journal. Preprint deposited in arXiv September 6, 2007.

Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C. S., Demleitner, M., Murray, S. S. (2005, The Effect of Use and Access on Citations. Information Processing and Management, 41, 1395-1402)

Moed, H. F. (2007) The effect of 'open access' on citation impact: An analysis of ArXiv's condensed matter section, Journal of the American Society for Information Science and Technology (JASIST) , August 30, 2007.

Seglen, P. O. (1992) The skewness of science. Journal of the American Society for Information Science 43:628-38

The UK's Arts and Humanities Research Council (AHRC) is now the 6th of the 7 UK Research Councils to adopt a Green Open Access Self-Archiving Mandate

That makes AHRC's the 18th funder OA mandate worldwide, in addition to 14 university and departmental mandates, 2 proposed multi-university mandates, and 4 proposed funder mandates, for a total of 38 Green OA mandates adopted or proposed so far.

Like most of the mandates adopted so far, the AHRC has some needless, easily-corrected flaws, but first, let us (with Dr. Johnson) applaud the fact that it has been adopted at all: Bravo AHRC!

Now the mandate's altogether unnecessary and ever-so-easily-corrected flaw:

In their anxiety to ensure that their policy is both legal and not needlessly worrisome for publishers, AHRC (and many of the other funder mandates, including yesterday's CIHR mandate from Canada) have allowed an embargo period before the article is made OA, if the publisher wishes.

That is fine. But it is a huge mistake to allow the time at which the article must be deposited to be dictated by the publisher's embargo.

The deposit should be required immediately upon acceptance for publication, without exception. If there is no publisher embargo, that deposit is also immediately made Open Access at that same time. Otherwise it is made Closed Access for the duration of the embargo period. (Only the bibliographic metadata are visible and accessible via the web, not the article itself.)

It may seem pointless to require an article to be deposited immediately if it cannot be made OA immediately. But the point of requiring immediate deposit either way is to close a profound loophole that could otherwise delay both deposit and OA indefinitely, turning the mandate into a mockery from which any researcher can opt out at the behest of his publisher.

The early mandators have been very progressive and helpful in having adopted OA mandates at all, but now that mandates are spreading, it is important to optimize them, and plug the needless loopholes. Otherwise the mandates will suffer the same fate as the ill-fated NIH Public Access Policy, which failed so badly that its self-archiving rate was even lower than the spontaenous baseline for random self-archiving, in the absence of any policy at all. (The proposed NIH policy upgrade to a mandate is now one of the 4 pending funder mandate proposals).

Optimizing OA Self-Archiving Mandates: What? Where? When? Why? How?

The Immediate-Deposit/Optional Access (ID/OA) Mandate: Rationale and Model

OA mandators (and those proposing or contemplating OA mandates): Please consult the above links, as well as Peter Suber's critique below, and then do the minor tweaks that are the only thing needed to transform your policies into reliable, effective mandates, setting an example worthy of emulation by others.

---

Peter Suber in Open Access News wrote:

The UK Arts & Humanities Research Council announced its long-awaited OA policy today. You can find it on the AHRC access policy page and in Appendix 9 of its lengthy (111 pp.) Research Funding Guide for 2007/08:

It is the AHRC’s position that authors choose where to place their research for publication. It is for authors’ institutions to decide whether they are prepared to use funds for any page charges or other publishing fees. Such funds could be part of an institution’s indirect costs under the full economic costing regime....

The AHRC requires that funded researchers:

   -- ensure deposit of a copy of any resultant articles published in journals or conference proceedings in appropriate repository

   -- wherever possible, ensure deposit of the bibliographical metadata relating to such articles, including a link to the publisher’s website, at or around the time of publication

Full implementation of these requirements must be undertaken such that current copyright and licensing policies, for example, embargo periods and provisions limiting the use of deposited content to noncommercial purposes, are respected by authors.

The final paragraph is emphasized (in bold type) in the Funding Guide but not emphasized on the access policy page.

Comments.

-- I applaud the mandatory language. But the policy is sketchy on most other important details. It doesn’t indicate which version should be deposited or what counts as an appropriate repository. It urges immediate deposit for metadata but doesn’t do so for the text itself. It gives no timetable for depositing the text and no maximum length for the delay or embargo.

-- It gives nearly as much space to the exception as it does to the policy, and creates the same gigantic loophole as the new CIHR policy and the older ESRC policy. If publishers don’t want their authors to make any version of their articles OA, they only have to adopt a house rule to that effect and suddenly the AHRC policy does not apply to AHRC grantees who submit work to that publisher.

--The AHRC is the sixth of the seven Research Councils UK to announce its OA policy. If this kind of mandatory language qualified by a vitiating exception can be called a mandate, then it’s also the sixth to adopt a mandate. The other five are at the BBSRC, ESRC, MRC, NERC, and STFC. The EPSRC is still deliberating. Of the six RCUK OA policies, three allow authors to use grant funds for publication fees at fee-based OA journals (MRC, NERC, STFC) and three do not (AHRC, BBSRC, ESRC).

Peter Suber, Open Access News

---

Stevan Harnad
American Scientist Open Access Forum

Faculty Attitudes and Behaviors Regarding Scholarly Communication: Survey Findings from the University of California

The UC Faculty Survey results are summarized in a somewhat misleading way:

"There is limited but significant use of alternative forms of scholarship, with 21% of faculty having published in open-access journals, and 14% having posted peer-reviewed articles in institutional repositories or disciplinary repositories."

(1) The practise in question is making published articles open access (not "alternative forms of scholarship").

(2) 21% of UC Faculty published articles in OA journals and 14% posted their published postprints in repositories.

(3) But 31% posted their published postprints on personal or departmental websites (and 29% posted their preprints).

So the comparison between the proportion of OA publishing (Gold OA) and OA self-archiving (Green OA) is not 21% vs. 14%. It's 21% vs. either 31% or anything up to 74% (if the 3 forms of self-archiving were additive).

UC should correct these summary figures. Otherwise it is giving a very misleading picture of the actual proportions at UC between the two ways of providing OA. This is important, because it is OA self-archiving that has the greatest scope for growth and acceleration, as Gold OA cannot be mandated, but Green OA can (and should be).

Stevan Harnad
American Scientist Open Access Forum

The Canadian Institutes of Health Research (CIHR) has just announced the official adoption of the Green Open Access Self-Archiving Mandate it had proposed last year.

This is the 31st Green OA Mandate adopted worldwide, but the 1st in North America. (Indeed, only one North American University -- l'Universite du Québec à Montréal -- has signed the Berlin Declaration.)

In all, 14 departmental and university self-archiving mandates plus 17 funder mandates have so far been adopted worldwide. In addition, 2 large multi-university mandates (Brazil and Europe) are in the proposal stage, as are 4 proposed funder mandates (two of them in the US and very big).

The UK is still substantially in the lead for OA mandates adopted, but if the pending US and European mandate proposals are adopted, OA will have prevailed unstoppably worldwide.

The next big growth area will be the sleeping giant of university Green OA mandates, fueled by both the OA movement and the Institutional Repository movement. The UK universities and the European ones are moving in concerted directions here. Time for US university provosts (who signed in support of the FRPAA Green OA mandate proposal) to go into action too!

Stay tuned...

Stevan Harnad
American Scientist Open Access Forum

---

SUMMARY: (1) Peer review just means the assessment of research by qualified experts.
(2) Peer review, like all human judgment, is fallible, and susceptible to error and abuse.
(3) Funding and publishing without any assessment is not a solution:
   (3a) Everything cannot be funded and even funded projects first need some expert advice in their design.
   (3b) Everything does get published, eventually, but there is a hierarchy of journals with a hierarchy of peer-review quality standards, their names and track-records serving as an essential guide for users about what they can take the risk of trying to read, use and build upon.
(4) So far, nothing as good as or better than peer review (i.e., qualified experts vetting the work of their fellow-experts) has been found, tested and demonstrated.
(5) Peer review's efficiency can be improved in the online era.
(6) Metrics are not a substitute for peer review, they are a supplement to it.

---

"Help Wanted: A pall of gloom lies over the vital system of peer review. But the British Academy has some bright ideas". The Guardian, Jessica Shepherd reports, Tuesday September 4, 2007

The 4 Sept Guardian article on peer review (on the 5 Sept British Academy Report, to be published tomorrow) seems to be a good one. The only thing it lacks is some conclusions (which journalists are often reluctant to take the responsibility of making):

(1) Peer review just means the assessment of research by qualified experts. (In the case of research proposals, it is assessment for fundability, and in the case of research reports, it is assessment for publishability.)

(2) Yes, peer review, like all human judgment, is fallible, and susceptible to error and abuse.

(3) Funding and publishing without any assessment is not a solution:

(3a) Everything cannot be funded (there aren't enough funds), and even funded projects first need some expert advice in their design.

(3b) And everything does get published, eventually, but there is a hierarchy of journals, with a corresponding hierarchy of peer-review quality standards. Their names and track-records are essential for users, guiding them on what they can take the risk of trying to read, use and build upon. (There is not enough time to read everything, and it's to risky to try to build on just anything that purports to have been found -- and even accepted papers first need some expert advice in their revision.)

(4) So far, nothing as good as or better than peer review (i.e., qualified experts vetting the work of their fellow-experts) has been found, tested and demonstrated. So peer review remains the only straw afloat, if the alternative is not to be tossing a coin for funding, and publishing everything on a par.

(5) Peer review can be improved. The weak link is always the editor (or Board of Editors), who choose the reviewers and to whom the reviewers and authors are answerable; and the Funding Officer(s) or committee choosing the reviewers for proposals, and deciding how to act on the basis of the reviews. There are many possibilities for experimenting with ways to make this meta-review component more accurate, equitable, answerable, and efficient, especially now that we are in the online era.

(6) Metrics are not a substitute for peer review, they are a supplement to it.

In the case of the UK, a Dual Support System of prospective funding of (i) individual competitive proposals (RCUK) and (ii) retrospective top-sliced funding of entire university departments, based on their recent past research performance (RAE), metrics can help inform and guide funding officers, committees, editors, Boards and reviewers. And in the case of the RAE in particular, they can shoulder a lot of the former peer-review burden: The RAE, being a retrospective rather than a prospective exercise, can benefit from the prior publication peer review that the journals have already done for the submissions, rank the outcomes with metrics, and then only add expert judgment afterward, as a way of checking and fine-tuning the metric rankings. Funders and universities explicitly recognizing peer review performance as a metric would be a very good idea, both for the reviewers and the researchers being reviewed.

Harnad, S. (2007) Open Access Scientometrics and the UK Research Assessment Exercise. In Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics 11(1), pp. 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds.

Brody, T., Carr, L., Gingras, Y., Hajjem, C., Harnad, S. and Swan, A. (2007) Incentivizing the Open Access Research Web: Publication-Archiving, Data-Archiving and Scientometrics. CTWatch Quarterly 3(3).

Shadbolt, N., Brody, T., Carr, L. and Harnad, S. (2006) The Open Research Web: A Preview of the Optimal and the Inevitable, in Jacobs, N., Eds. Open Access: Key Strategic, Technical and Economic Aspects. Chandos.

Harnad, S. (ed.) (1982) Peer commentary on peer review: A case study in scientific quality control, New York: Cambridge University Press.

Harnad, Stevan (1985) Rational disagreement in peer review. Science, Technology and Human Values, 10 p.55-62.

Harnad, S. (1986) Policing the Paper Chase. [Review of S. Lock, A difficult balance: Peer review in biomedical publication.]Nature 322: 24 - 5.

Harnad, S. (1996) Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals. In: Peek, R. & Newby, G. (Eds.) Scholarly Publishing: The Electronic Frontier. Cambridge MA: MIT Press. Pp 103-118.

Harnad, S. (1997) Learned Inquiry and the Net: The Role of Peer Review, Peer Commentary and Copyright. Learned Publishing 11(4) 283-292.

Harnad, S. (1998/2000/2004) The invisible hand of peer review. Nature [online] (5 Nov. 1998), Exploit Interactive 5 (2000): and in Shatz, B. (2004) (ed.) Peer Review: A Critical Inquiry. Rowland & Littlefield. Pp. 235-242.

Peer Review Reform Hypothesis-Testing (started 1999)

A Note of Caution About "Reforming the System" (2001)

Self-Selected Vetting vs. Peer Review: Supplement or Substitute? (2002)

Stevan Harnad
American Scientist Open Access Forum