Open Access Archivangelism

Research Evaluation, Metrics and Open Access in the Humanities
Coimbra-Group Workshop
Trinity College Dublin
18-20 September 2008

-- Aimed at Arts and Humanities researchers, Deans of Research, Librarians, research group leaders and policy makers within the Coimbra-Group member universities and the Irish University sector...

-- To compare established and innovative methods and models of research evaluation and assess their appropriateness for the Arts and Humanities sector...

-- To assess the increasing impact of bibliometrical approaches and Open Access policies on the Arts and Humanities sector...

The European Commission has now mandated Green OA self-archiving for 20% of its 7th Framework Funding. This is the 51st Green OA Mandate worldwide (and the 26th funder mandate: The European Research Council (ERC), another European research funder, had earlier likewise mandated Green OA.)

See ROARMAP: Institution's/Department's OA Self-Archiving Policy

The pilot covers approximately 20% of the FP7 budget and will apply to specific areas of research under the 7th Research Framework Programme (FP7): Health, Energy, Environment, Information and Communication Technologies (Cognitive Systems, Interaction, Robotics), Research Infrastructures (e-Infrastructures), Socio-economic Sciences and Humanities, Science in Society.

New grant agreements in the areas covered by the pilot will contain a clause requiring grant recipients to deposit peer reviewed research articles or final manuscripts resulting from their FP7 projects into their institutional or if unavailable a subject-based repository... within six or twelve months after publication, depending on the research area.

The Humanities and Social Sciences branch of France's Agence Nationale de la recherche has just announced its Green OA self-archiving mandate -- France's first funder mandate (France' second mandate overall, and the world's 50th). See ROARMAP

Note that the situation in France with central repositories is very different from the case of NIH's PMC repository: France's HAL is a national central repository where (in principle) (1) all French research output --from every field, and every institution-- can be deposited and (again, in principle) (2) every French institution (or department or funder) can have its own interface and "look" in HAL, a "virtual" Institutional Repository (IR), saving it the necessity of creating an IR of its own if it does not feel it needs to.

The crucial underlying question -- and several OA advocates in France are raising the question, notably, Hélène Bosc, in a forthcoming article (meanwhile, see this) -- is whether the probability of adopting institutional OA mandates in France is increased or decreased by the HAL option: Are universities more inclined to adopt a window on HAL, and to mandate central deposit of all their institutional research output, or would they be more inclined to mandate deposit in their own autonomous university IRs, which they manage and control?

Again, the SWORD protocol for automatic import and export between IRs and CRs is pertinent, because then it doesn't matter which way institutions prefer to do it.

---

Agence Nationale de la recherche (ANR) (Humanities and Social Sciences Branch) (FRANCE*
funder-mandate)

Institution's/Department's OA Eprint Archives

Institution's/Department's OA Self-Archiving Policy

[Paraphrase by T. Chanier]

"The Humanities and Social Sciences branch of the French National Research Agency (ANR) mandates [requires] researchers involved in projects that it funds to deposit their scienfically validated (refereed postprint) publications in the HAL-SHS open archive, without any delay.

"HAL is a nation-wide open archive supported by all public French Research Insitutions. Hal-SHS is a sub-part dedicated to Humanities and Social Sciences

"In November 2007, the general ANR agency had merely invited its researchers to deposit in HAL

"This time (July 2008), the ANR's SHS branch mandates that its researchers deposit (and requests project leaders to confirm that the deposit is done by everyone)."

** text extracted (July 2008) from the ANR SHS text **

"Par un communiqué :en date du 14 novembre 2007, l’ANR incite les chercheurs, porteurs ou partenaires de projets financés par elle, à intégrer leurs publications dans le système d’archives ouvertes HAL avec lequel elle collabore.

"Le Département SHS de l’ANR et la cellule-support de l’ENS LSH souhaitent donner un écho particulier à cette directive qu’ils jugent essentielle pour une visibilité accrue de la recherche française en SHS.

"La communauté des porteurs et partenaires de projets ANR en SHS doit ainsi se mobiliser autour d’un objectif commun qui est celui d’un dépôt systématique de leurs productions scientifiques dans HAL- SHS, interface SHS de l’archive HAL.

"Il est demandé aux porteurs et responsables de projets ANR de s’assurer, au sein de leurs équipes (chercheurs, universitaires, post-doc, doctorants, qu’ils soient français et étrangers), de l’intégration dans HAL de l’ensemble des publications (articles, communications, contributions à ouvrages collectifs, ou autres productions éligibles) réalisées dans le cadre du projet, et ce, au fur et à mesure de leur élaboration (par exemple dès la soumission à une revue puis à nouveau au moment de la publication effective)."

Registered by: Thierry Chanier (Professor, leader of ANR SHS-funded project) on 29 Jul 2008

I am not at all sure that Kudos are in order for Oxford University Press (OUP), just because they offer authors at subscribing institutions a discount on their hybrid Gold OA fee:

Unlike the American Psychological Association (yes, the much maligned APA!), the American Physical Society, Elsevier, Cambridge University Press and all the other 232 publishers (57%) of the 6457 journals (63%) that are on the side of the angels -- fully Green on immediate post-print self-archiving -- OUP is among the Pale-Green minority of 48 publishers (12%) of 3228 journals (32%) (such as Nature, which back-slid to a postprint embargo ever since 2005).

OUP's post-print policy is:

12 month embargo on science, technology, medicine articles
24 month embargo on arts and humanities articles
Pre-print can only be posted prior to acceptance
Pre-print must not be replaced with post-print, instead a link to published [toll] version
Articles in some journals can be made Open Access on payment of additional charge

Should we really be singing the praises of each publisher's discount on their hybrid Gold OA fee for the double-payment they are exacting (from the subscribers as well as the authors)?

I would stop applauding as progress for OA every self-interested step taken by those publishers who do not first take the one essential OA-friendly step: going (fully) Green.

Yes, OUP are lowering fees annually in proportion to hybrid Gold OA uptake, but they are meanwhile continuing to hold the post-print hostage for 12-24 months.

In reality, all the fee reduction means is an adjustment for double-dipping -- plus a lock-in on the price of Gold OA, and a lockout of Green OA.

Stevan Harnad
American Scientist Open Access Forum

Stan Katz has blogged a summary of the OA session at the Association of American University Presses (AAUP) Meeting in Montreal in the the Chronicle of Higher Education Review. Here are six little clarifications:

(1) SK: "Open Access [OA] [means that] all scholarly articles ought to be mounted on free public access websites maintained by their universities.

" This brief definition is fine as a 1st cut.

(2) SK: "[T]he obligation [i.e., recent mandates like Harvard's] to “publish” by mounting articles on free websites will make it impossible for nonprofit presses and learned societies to sustain themselves."

The mandates are for authors to mount (i.e., self-archive) the final, refereed draft of their published article, not to "publish" it by mounting it. This self-archived "Green OA" draft is a supplement, provided for all users whose institutions cannot afford access to the publisher's version. It is not itself another publication.

The publisher might be nonprofit or commercial; that is not the relevant question. The relevant question is whether or not the supplementary OA draft causes cancellations, rendering journal subscriptions unsustainable for covering costs. So far, in the few fields where OA self-archiving has been taking place the longest (15 years) and most extensively (100%) -- e.g. high-energy physics, published by the American Physical Society (APS), the Institute of Physics (IOP) and Reed-Elsevier -- the publishers report that they find no detectable subscription cancellations associated with self-archiving:

However, if and when OA self-archiving ever does cause catastrophic cancellations, making subscriptions unsustainable, then, and only then, journals can (a) offload all their former access-provision and archiving functions, along with their costs, onto the distributed network of institutional repositories, (b) downsize to peer review alone, and (c) convert to the "Gold OA" publication-cost-recovery model, charging the author-institution, per outgoing paper, for peer review and certification instead of charging the user-institution, per incoming journal, for access. The institutions will (on the very same hypothesis, of catastrophic cancellations) have more than enough annual windfall subscription cancellation savings out of which to cover those charges.

(3) SK: "Harnad’s suggestion is that the universities transfer the payments they are currently making to their academic presses to subsidize peer review and archiving of their faculty scholarly output."

No, my suggestion is only that universities should mandate self-archiving. Then, if and when the resulting universal OA should ever make subscriptions become unsustainable, the universities' subscription savings will, by the same token, be freed to pay for the university's peer-review costs. That is no subsidy: It is direct payment for a service, out of the very same funds formerly used to purchase a product.

(4) SK: "[U]niversities increasingly expect their presses to be self-sustaining economically, and are unlikely to put up the necessary funding"

Extra funding by whom for whom for for what? While universities are subscribing to journals, that pays for peer review. If and when journal subscriptions collapse, the university savings will pay for the peer review. (And again this has nothing to do with university presses in particular.)

(5) SK: "[N]ot all scholars (and nonprofit publishers) are connected to universities. Who will subvene their publications?"

Unaffiliated scholars are rare enough, and the per-paper costs of peer review alone are low enough, so that a small surcharge on the charges of the affiliated authors (the vast majority) will take care of these outliers.

And as noted, neither whether publishers are nonprofit or commercial publishers, nor whether they are university or learned-society publishers, is relevant to any of this.

(6) SK: "[OA has] different... implications...for the humanities and social sciences"

OA pertains to refereed journal articles publication in all disciplines. Humanities and social sciences are not exceptions in any way. All research, in all scholarly and scientific disciplines, benefits from maximizing its uptake, usage, applications and impact by eliminating the access-barriers that have been made obsolete and unnecessary by the advent of the PostGutenberg Galaxy.

Stevan Harnad
American Scientist Open Access Forum

---

SUMMARY: OA IRs provide free supplemental copies of published, refereed journal articles. The way to access them is via a harvester/indexer. Direct searching of the IR is more relevant for (1) institution-internal record-keeping, (2) performance assessment, (3) CV-generation, (4) grant application and fulfillment, and perhaps also some window-shopping by prospective (5) faculty, (6) researchers or (7) students. The main purpose of depositing refereed journal articles is (8) so they are accessible to all would-be users, not just to those whose institutions happen to have a subscription to the journal in which they were published. That way (9) the usage and impact of the institutional research output is maximized (and so is (10) overall research progress).
   What content an IR accepts is an entirely different matter from what content an IR mandates. Harvard is mandating OA target content, which is the refereed, accepted final drafts of peer-reviewed journal articles. Harvard is not publishing its journals articles. The journals are publishing them, and providing the peer review and copy-editing. Harvard is merely providing supplementary access to the peer-reviewed final drafts for those would-be users who cannot afford access to the publishers published (and copy-edited) version.
   OA is needed for researcher (peer to peer) access. The lay public benefits indirectly from the enhanced research productivity, progress, impact and applications generated by OA, not from direct public access to esoteric, technical reports. But whatever is one's primary rationale -- public access or peer access -- the other comes with the OA territory.
   Are would-be users whose institutions cannot afford subscription access to the publisher's copy-edited version better off with a refereed final draft, not copy-edited, or are they better off without it? (The answer is obvious.)
   If and when Green OA self-archiving (of refereed, non-copy-edited final drafts of journal articles) and Green OA self-archiving mandates should ever make journal subscriptions unsustainable, author-institutions can pay for peer review by the article out of their windfall subscription cancellation savings on the Gold OA cost-recovery model. If they find it worth paying for too, the copy-editing service can be bundled with the peer review service.

---

[See also: No Such Thing As "Provostial Publishing": I]

---

On Fri, 30 May 2008, Sandy Thatcher wrote in liblicense:

ST: "[O]ne is not likely to start with an IR to find the most important work in a discipline, unless one happens to follow the work of a particular scholar, in which case one would likely go to the scholar's own web site first, not the IR."

OA IRs provide free supplemental copies of published, refereed journal articles. The best and most likely way to find and access them is via a harvester/indexer that links to the item, not by directly searching the IR itself. (Direct searching of the IR is more relevant for (1) institution-internal record-keeping, (2) performance assessment, (3) CV-generation, (4) grant application/fulfillment, and perhaps also some window-shopping by prospective (5) faculty, (6) researchers or (7) students.)

The main purpose of depositing refereed journal articles is (8) to maximize their accessibility, so they are accessible to all would-be users, not just to those whose institutions happen to have a subscription to the journal in which they were published. That way (9) the usage and impact of the institutional research output is maximized (and so is (10) overall research progress).

ST: "But I do continue to question what the institution gains from its IR."

It seems to me that (1) to (10) above is quite a list of institutional gains from their IRs.

ST: "Does Harvard really need, or will it gain, any more "prestige" by having its faculty's work deposited there?"

Prestige is only a small part of it. All universities want and need (1) - (10) (and not all universities are Harvard for prestige either!).

ST: "It seems equally likely that it will lose some respect if too many scholars post articles that are first drafts or occasional pieces that would never appear in any peer-reviewed forum. It could easily become a grab bag of miscellany that will not reflect well on Harvard's presumed reputation for quality."

What content an IR accepts is an entirely different matter from what content an IR mandates. Harvard is mandating OA target content, which is the refereed, accepted final drafts of peer-reviewed journal articles.

It does not seem to have done the prestige of high energy physics any harm to have been (for over a decade and a half) self-archiving their pre-refereeing preprints too, even before their refereed, final drafts. But that is an individual and disciplinary choice, not one that the Harvard mandate is making for its faculty, and not the primary purpose of an OA IR.

ST: "Harvard authors, on the whole, are no better writers than scholars elsewhere, I would suggest, and their unedited prose will not do any good for the institution."

That may or may not be a good argument against depositing unrefereed preprints, but it has nothing to do with OA, OA mandates, or the primary purpose of OA IRs.

ST: "And, as for the general public, what members of that public are really going to bother spending their time pouring over esoteric scholarship when they can go to Wikipedia to get the information they need? This seems to me as false an assumption as the expectation that somehow members of the public are going to benefit greatly from reading the technical articles posted on PubMed Central under the new NIH program."

I think you are quite right about that, Sandy. I myself have been somewhat uncomfortable all along with the heavy emphasis that some of the advocacy for OA has placed on the putative need of the general tax-paying public for access to peer-reviewed research journal articles. The need is there for health-related research, and perhaps a few other areas, but I have always felt it weakens rather than strengthens the case for OA to argue that its primary purpose and urgency is for the sake of lay public access. It is not. OA is needed for researcher (peer to peer) access. The lay public benefits indirectly from the enhanced research productivity, progress, impact and applications generated by OA, not from direct public access to esoteric, technical reports.

However, I have to admit that my worries may have been misplaced, because the Alliance for Taxpayer Access (which I strongly support) managed to get through to a lot more people (lay public, academics and congressmen) with the message of taxpayer access than they would have done if they had stressed only peer-to-peer access. So they perhaps overgeneralized the special case of health-related research, suggesting that it was a matter of similar urgent public interest in all public-supported research. In reality, it is a matter of similarly urgent interest for the public -- but an interest in maximizing research progress and applications, by making research openly accessible to its real intended users, namely, other researchers. That is the way the tax-paying public maximizes its benefits from the research it supports -- not by reading it for themselves!

This public access argument, however, applies more to the public funding of research grants (such as those of NIH) than to university research access policy. In the university case, it is clear that the objectives of OA mandates are mostly peer-to-peer (or, if you like, university-to-university access) rather than a burning need for public access.

It is to be noted, though, that whatever is one's primary rationale -- public access or peer access -- the other comes with the OA territory. So the outcome is exactly the same.

Harnad, S. (2007) Ethics of Open Access to Biomedical Research: Just a Special Case of Ethics of Open Access to Research. Philosophy, Ethics and Humanities in Medicine, 2 (31).

ST: "I imagine that very few members of the public are going to be able to understand the vast majority of these articles, let alone derive any useful lessons for life from them. There seems to be a general fantasy that the whole world is somehow waiting breathlessly for access to all this highly specialized knowledge. I speak as director of a press that has a hard time selling books that we think to be of "general interest," compared with our monographs. The audience just isn't there, folks! And institutions that believe their reputations are going to soar because of what their faculty post on their IRs are just kidding themselves."

I agree with most of this, as you see.

(But be careful, Sandy! Some might be tempted to say you have a hard time selling books, but you might have less of a hard time giving them away free (OA)! As you know, I do not make this argument for books in general, because I know that there are true and unavoidable production costs to cover there -- whereas with Green OA IRs, journal publication reduces to just the costs of implementing peer review alone. Moreover, book authors are more inclined to seek royalties than to pay production costs and give their books away free. But for some esoteric monographs, an online-only OA edition, with costs covered by the author's institution, may very well turn out to be the optimal model, much as it will probably end up for journal article publication.)

Harnad, S., Varian, H. & Parks, R. (2000) Academic publishing in the online era: What Will Be For-Fee And What Will Be For-Free? Culture Machine 2 (Online Journal)

Ceterum Censeo: The notion of "provostial publishing" is utter nonsense:

No Such Thing As "Provostial Publishing": I.

ST: "On what basis do you [claim journal copy-editing is minimal]? Have you surveyed journals to find out how much copyediting they do? Are you basing this on your own personal experience with copyediting done by the journals to which you have submitted your own work primarily?"

It is based on the CUP journal I edited for 25 years, plus many other journals for which I have refereed and in which I have published. It is also a judgment shared by many not only in the research community but also in the publishing community: Journal article copy-editing today is down to which-hunting and reference-querying, for the most part. Software can do the latter and we can do without the former, I think.

ST: "I, of course, cannot claim sufficiently wide knowledge to make sweeping generalizations about the degree and level of copyediting done for journals compared with books at all publishing houses. But as director of a press that publishes 11 journals in the humanities, and a past employee of another press that published three (including one in mathematics), I can attest that the copyediting done for these journals is at the same level as done for books, which in university presses is pretty high. I suspect that other university presses operate in this respect the same way we do--which would mean that at least 1,000 scholarly journals get far more than 'minimal copy-editing'."

That may well be. But there are only two underlying issues here, so it's best not forget them:

(1) Are would-be users whose institutions cannot afford subscription access to the publisher's copy-edited version better off with a refereed final draft, not copy-edited, or are they better off without it? (The answer is obvious, I think.)

(2) If and when Green OA self-archiving (of refereed, non-copy-edited final drafts of journal articles) and Green OA self-archiving mandates should ever make journal subscriptions unsustainable, author-institutions can pay for peer review by the article out of their windfall subscription cancellation savings on the Gold OA cost-recovery model. If they find it worth paying for too, the copy-editing service can be bundled with the peer review service.

Nothing hangs on either of these things, and all other questions (such as how much copy-editing is really being done on journal articles today) are irrelevant to the Green OA and Green OA mandate issue.

ST: "I can also attest, from my own years of experience as a copyeditor, that the job does not just involve polishing prose and improving grammar. Not uncommonly, copyeditors will find and correct egregious factual and other errors, thus sparing the authors from considerable embarrassment. Without their "value added" services, much will get published in Green OA form that will NOT serve either the authors' peers or the general public well."

Right now, that value-added is being paid for by subscriptions. If subscriptions ever vanish, and the value is still desired, it can be paid for along with peer review on the Gold OA cost-recovery model.

ST: "Hence, I conclude, Harvard and others that follow its example and are content to publish less than the final archival version will be opening themselves to the exposure of all the flaws of scholarly writing that now get hidden from public view by the repair work done by copyeditors. Caveat lector!"

Harvard is not publishing its journals articles. The journals are publishing them, and providing the peer review and copy-editing. Harvard is merely providing supplementary access to the peer-reviewed final drafts for those would-be users who cannot afford access to the publishers published (and copy-edited) version. To complete the circle again, please refer to (1) above...

Stevan Harnad
American Scientist Open Access Forum

---

SUMMARY: Talat Chaudhri and I agree that Green OA via self-archiving is feasible and desirable, and that publication will eventually consist of peer review alone. The only points of disagreement are about how to get there from here. I advocate Green OA mandates, whereas Talat advocates direct transition to peer-review-only, administered by university consortia. What Talat does not explain, however, is how we are to get the 25,000 journal titles that are currently being published by their current publishers to migrate to (or be replaced by) such consortia. Nor does Talat explain how the consortia's true peer-review expenses would be paid for, even if the 25K journals titles did miraculously migrate to such consortia of their own accord (although the answer even then is obvious: via Gold OA author-institution fees, paid out of their subscription cancellation savings).

---

Talat Chaudhri, Repository Advisor, Aberystwyth University, wrote (in the American Scientist Open Access Forum):

TC: "Gold OA [1] isn't popular and, I suspect, [2] never will be."

Talat Chaudhri is right about the first point [1], and the reason is partly the current price of Gold OA and partly the fact that Gold OA is not yet necessary, because Green OA (self-archiving) can provide OA.

But whether the price of Gold OA once it amounts to nothing more than the cost of peer review will be "popular" [2] -- if and when it becomes a necessity (i.e., if and when universal Green OA makes subscriptions unsustainable) -- is not a matter of either popularity or suspicion: As long as peer review is necessary, paying the true costs of implementing it will be necessary, if one wants to publish (peer-reviewed research) at all.

The good news is that the cost per paper of peer review alone then will be far less than the cost per paper of (subscription) publishing now, and the subscription cancellations will release many times the amount of money needed to pay for peer review alone.

For the perplexed reader: Talat and I are not disagreeing on most of these points. We both agree that Green OA via self-archiving is feasible and desirable, and that publication will eventually consist of peer review alone.

The only points of disagreement are about how to get there from here.

I advocate Green OA mandates, whereas Talat advocates direct transition to peer-review-only, administered by university consortia.

What Talat does not explain, however, is how we are to get the 25,000 journal titles that are currently being published by their current publishers to migrate to (or be replaced by) such consortia.

Nor does Talat explain how the consortia's true peer-review expenses would be paid for, even if the 25K journals titles did, mirabile dictu, migrate to such consortia of their own accord (although the answer even then is obvious: via Gold OA author-institution fees, paid out of their subscription cancellation savings).

But apart from not wanting to call this sort of payment "Gold OA" (even though that's exactly what it is!), Talat also does not like Green OA mandates.

The trouble is that Talat has no better way -- nor even an equally good alternative way -- to get the 2.5 million articles published annually in the 2.5K journals to "migrate" to their authors' Green OA IRs -- any more than he has a way of getting the university peer review consortia created, or of getting the journal titles to migrate over to (or be replaced by) them.

So let's focus on the substantive points of agreement: (1) Universal Green OA and (2) publishing costs reduced to just peer review alone. Talat can leave the problem of generating that Green OA to the Green OA mandates, and he can call the funding of the peer review something other than "Gold OA" if he likes -- it all comes to the same thing anyway...

TC: "On "downsizing" to Gold OA, I'm afraid that I agree with the original point in the [Letter to Nature] to which N. Miradon posted a link recently. The developing world doesn't want it."

Reply: Downsizing is for publishers (not researchers) to do, under Green OA cancellation pressure. The only thing the developing world need do is to provide Green OA to its own article output by self-archiving the accepted, refereed final drafts (postprints) in their Institutional Repositories (IRs) (which is exactly the same thing that the developed world needs to do).

TC: "Neither, I submit, does anybody in the developed world want to pay for it."

They needn't. They need only mandate and provide Green OA. The rest will take care of itself. Institutions are already paying for publication (via subscriptions). When they no longer have to pay for subscriptions by the incoming journal, institutions' savings will be more than enough to pay for peer review by the outgoing article instead.

TC: "In terms of diverting currently subscription funds progressively to OA, any librarian such as myself will tell you that getting management agreement for what looks to them like a hypothetical new publishing model is going to be complex and very possibly unworkable, leaving only the few universities that have created funds for the purpose. None to my knowledge has agreed to allocate money on a yearly basis, as the costs are currently unknown."

But I have not said anything whatsoever about libraries needing to progressively divert subscription funds to OA.

I said universities and funders should mandate and provide OA (as 44 universities and funders, including Southampton and Harvard, ERC and NIH have done) and that IF and WHEN that should ever make subscriptions unsustainable (i.e., they are all cancelled), THEN a small portion of their windfall institutional savings can and will be redirected to pay for peer review.

No one is asking libraries to divert anything anywhere now, instantaneously or progressively. (If and when the time of universal, unsustainable cancellations comes, Necessity will be the Mother of Invention. No need to speculate or counterspeculate about it in our imaginations now, pre-emptively; let's just concentrate on mandating and providing universal Green OA.)

TC: "Why will Gold OA not catch on? Because it is unjust! Only those academics whose institutions can afford to pay will be able to publish, unlike the present situation where anybody can."

You are talking about Gold OA now, Talat, at current asking prices, and I agree.

So focus instead on mandating and providing Green OA for now, and worry about the question of converting to Gold OA if and when it becomes an actual matter of necessity, not just a hypothetical matter of possibility. (By that time the asking price will be so low, and the cancellation savings so high, that the decision will become a no-brainer.)

TC: "As I am presently a librarian, not an academic, I would be very likely unable to publish in my field of research on the basis of these centrally allocated funds, like retired academics and those in the developing world. Nobody will want this model, quite simply. They don't want it now!"

To repeat, you are thinking of Gold OA today, at today's asking prices, while all the money that can potentially pay for it is still tied up in paying the subscriptions. This is unilluminating and irrelevant: Forget about Gold OA if you wish. Publish wherever you like, self-archive your postprint, and let nature take care of the rest,

(Once Green OA is universal and only peer review needs to be paid for, the cost will be low enough so these needless hypothetical worries will look risible. And provisions for the minority of researchers who are retired, institutionally affiliated or otherwise unable to pay the low costs of peer review will be made. We don't need to retain the present access-denial juggernaut in order to take care of that small minority of special cases.)

TC: "[T]here was no "plausible path" for print to electronic publishing, yet it happened. If people as well placed as yourself were advocating... [university peer-review consortia] I am sure it might have a strong chance of catching on."

(1) If I were anywhere near as "well placed" as you imagine I am, dear Talat, we would have had 100% Green OA a decade and a half ago.

(2) Electronic publishing did not face the regenerating heads of the 34-headed monster responsible for the "Zeno's Paralysis" that besets Green OA -- a syndrome to which you, Talat, are alas not immune either:

#1. Preservation
#7. Peer review
#9. Downsizing
#10. Copyright
#17. Publishers
#18. Libraries
#30. Redirecting Cancellation Savings
#32. Poisoned Apple

The only effective medication, apparently, is Green OA mandates, and, luckily, relief is on the way:

Harnad, S. (2006) Opening Access by Overcoming Zeno's Paralysis, in Jacobs, N., Eds. Open Access: Key Strategic, Technical and Economic Aspects. Chandos.

TC: "...'relieving' journals of costs also "relieves" them of profits, which they won't want. It's myopic, to use your word, to suggest that this won't cause problems fairly soon."

Fairly soon? Self-archiving at 100% levels in high energy physics has not yet begun to be felt in cancellations in 17 years (both APS and IOP have confirmed this publicly: see the publications of Alma Swan). I am optimistic about mandates, but not so optimistic that I think that 100% Green OA will be with us "fairly soon." So how can a library cancel a journal while only an unknown percentage of articles from an unknown number of mandating universities are being self-archived?

I can only repeat: It would do a lot more good if we self-archived (and supported self-archiving mandates) more and speculated about the future of publishing (or libraries) less...

TC: "[Publishers currently endorse OA self-archiving] under licence which they remain free to withdraw, if that should be in their interests. Don't fool yourself that they couldn't if need be. At present it doesn't serve publishers to do so, so they don't. This is no basis on which to plan."

I cannot fathom, Talat, why you would prefer to keep speculating about whether and when publishers might withdraw their Green light to self-archive instead of pressing on with self-archiving and self-archiving mandates while the going is Green. This is one of the 34 familiar symptoms of Zeno's Paralysis, and it's been with us for years now:

#32. Poisoned Apple
I worry about self-archiving even if the journal gives me the green light to do so, because if I do, the light may change to red."

The answer to your question is that as Green OA grows, the risk to publishers is less that of losing subscribers, but that of losing authors. And losing authors would certainly accelerate cancellations a lot faster than the anarchic growth of Green OA self-archiving will. (Losing Harvard authors today would be bad enough, but it would only get worse, if the publishers' response to OA mandates were to try to revert to Gray instead of Green.)

The only thing you need to "plan" today is how to facilitate the provision of Green OA.

TC: "I happen to believe that nobody wants Gold OA in the future, as they don't appear to want it now."

Most researchers don't want Gold OA now because the money to pay for it is tied up in subscriptions and the asking price is way too high. What they want now is OA, and Green OA mandates will see to it that they get (and give) it.

If and when the subscription funds are released, the price drops, and there is no other way to publish, researchers will want Gold OA.

But why keep speculating about if and when? Green OA is within reach, and all it needs is more and more Green OA mandates.

TC: "Arts departments have not co-operated with the Green OA revolution, as has recently been brought home to me here by our English Department."

No? It seems to me that the Harvard Faculty of Arts and Science voted unanimously for Harvard's Green OA mandate. All the other universities that have mandates have English Departments too.

TC: "This is because we haven't understood their needs and continue to talk only about the most recent cutting edge science departments. Arts subjects are much more concerned with what you dismiss as "legacy" literature, preservation, book publishing, without which OA means little to them. We have sought no answers for any of these areas and so have no solutions for these academics."

OA's primary targets are journal articles; Green OA mandates only mandate the self-archiving of journal articles. The Arts and Humanities disciplines are more book-intensive than journal-intensive compared to the Physical, Biological and Social Sciences. But inasmuch as they publish in journals at all, no discipline is indifferent to the usage and impact of its journal articles. So:

(1) Preservation is important (but no more relevant to OA than non-OA).

(2) Book-publishing, inasmuch as it is not an author give-away literature, the way refereed journal articles are, is (so far) irrelevant.

(3) The legacy literature will have its turn once the current and forward-going literature is OA. (World hunger, the cure of all diseases and the righting of all wrongs and injustice will have to wait too, I am afraid: We're just talking about righting one wrong here: needless and obsolete access-denial for refereed research journal articles in the online age.)

TC: "...repository managers, their libraries and therefore their institutions... may not be so eager to follow your predictions as you hope, given that you have such a poor view of their "legacy" holdings and given the comments I have made on the failure to address the needs of all disciplines. I'm not sure we even have a solution for sciences and social sciences."

OA IRs are not preservation archives, they are access-provision archives. And the access is to their own institutional research output, not to the licensed subscription content their libraries have bought in from other institutions. (There is some fundamental underlying confusion here, or a conflation of two agendas, only one of which is OA. And none of this has anything to do with discipline differences.To my knowledge, no significant discipline differences have been reported across all the disciplines tested, either for the size of the OA impact advantage, or for the willingness to comply with OA mandates.)

TC: "as I have agreed, we are stuck with the necessity for mandates asap. [But] publishers and universities alike need to find different funding models now, ahead of time, before conflict arises with the publishers, as it must inevitably do if we simply eyeball them from the trenches waving our mandates."

I'm glad we agree on the necessity of mandates, Talat (although only one of us regrets that we are "stuck" with them).

But we will have to agree to disagree on the advance need to find different funding models. Or rather, I would say we have found already a different funding model (Gold: author-institutions pays for publication output instead of user-institution paying for publication input), but its time has not yet come: Universally mandated OA is needed first, to pave the way.

TC: "The relationship between library (i.e. those who acquire both resource and locus of deposit) and researcher is key to the solution, as any good subject librarian will tell you. I fear that you don't understand how libraries form a key part of the way in which the institutions who they serve, and who you mention as players in this, actually change policy in the interests of the researchers. This is the main point of contact in the institution."

Talat, is the role of libraries and librarians in the transition to free online access to research journal articles really that apparent? Did good subject librarians know where we're all heading a-priori, even before the online era?

OA is largely a matter for the research community: They are the providers as well as the users. But unlike with books, which need to be bought in and collected by their libraries, it is not at all obvious that OA requires library mediation.

It is fine if the library is made the manager of the IR, but then let the task be taken on as the radically new task it really is, rather than forced into the Procrustean Bed of what the library's traditional expertise and functions have been. OA is new territory, requiring new, Open mind-sets, not "what any good subject librarian already knows"...

TC: "I would instead hope to hear direct answers to the points raised, as well as a reasoned argument against "consortia" journals rather than merely waving them aside as a foolish repository manager's fancy."

You've had direct answers, Talat: Self-archiving mandates have already been tested and demonstrated to generate Green OA, they are feasible, they are growing, and they scale (to all universities and all funders).

What has been tested and demonstrated with (1) generating peer-review consortia and (2) getting journal titles and authors to migrate to them?

It does not help to repeat an N of 1 (the Board of Celtic Studies). There are over 3000 Gold OA journals, and you yourself doubt they will scale...

TC: "Sadly it appears this point has been side-stepped deliberately, as I confess that I anticipated. Core Green OA forecasts, however speculative, are to be supported. Others are to be rejected as mere speculations, a double standard."

No double standard. I have left no substantive point unanswered.

TC: "from the academics' expressed point of view... [self-archiving] is a new obligation that impinges on, as they see it, what they do with their copyright. Hence it looks like coercion. Taking heed of this reaction is the only way to get true co-operation."

Arthur's Sale's studies (and the continuing evidence since) keep confirming that authors willingly comply with self-archiving mandates. The latest two mandates (from Harvard) have been unanimously voted in by the academics themselves.

TC: "I'm not in a position to [university peer-review consortia]. But I suggest that someone whose advocacy on the subject will be heard, such as yourself, might be in a position to popularise the idea speedily, if you wished to put your efforts into it."

I did put my efforts behind alternative publishing models, a decade and a half ago, when I thought the problem was in the publishing model. Crashing failure made me realize that the problem was not in the publishing model but in academics' heads (Zeno's Paralysis). And the tried and tested cure is Green OA mandates, and they are happening. So why should we go back to old, far-fetched, and discarded hypotheses?

TC: "Simply, what we don't have is an answer to how peer review, copy editing and so forth will actually be provided after the Green OA revolution."

What we need now is not an answer to that question! What we need now is universal Green OA!

TC: "If there is no way forward, the revolution cannot happen."

There is a way forward: Green OA self-archiving mandates, by universities and funders, and they are happening.

TC: "I support Green OA but I do not believe at all that it will, or should, lead to Gold OA."

Fine. Let it just lead to Green OA!

TC: "There is no natural progression in this whatsoever, as nobody wants Gold OA anyway."

Fine. It is OA that research needs, not necessarily Gold OA.

TC: "If you destroy the publishers, as you suggest, who will then do the peer review?"

Where did I ever suggest "destroying the publishers"? (The talk of impending destruction, catastrophe and doom has come from speculators (and mostly by those with vested interests in the status quo, such as publishers, but sometimes also, for different reasons, librarians).

And peer review will continue to be done (for free) by the peers who review, regardless of who is paying to implement the process, or how.

TC: "All this talk about costs is a whitewash: they are relatively insignificant anyway compared to the research process. Universities could easily shoulder them, especially given savings from subscriptions, which are exorbitant."

You sound like you are saying the same thing I am now. So why are we talking about costs, and who will pay them, and when, when the urgent issue is Green OA, and getting it mandated so that we have it, at long last, now?

TC: "Find a solution to the future source of peer review (that isn't merely Gold OA) and you solve the whole thing. This peer review problem is all that is holding back Green OA. Forget Gold OA, it simply isn't part of the solution."

No, it is Zeno's Paralysis that is holding back Green OA, and one of the symptoms of Zeno's Paralysis is fretting needlessly about who is going to pay for peer review if and when it is no longer being paid by subscriptions.

But the OA problem is not "who is going to pay for peer review if and when it is no longer being paid by subscriptions." It is the research access and impact that continues to be lost daily, as we sit counterfactually speculating, day in and day out, about the future of publishing instead of the present of research.

And the solution is at researcher' fingertips: They just need to do the keystrokes that will get their articles into their Green OA IRs.

And the cure for what is holding back researchers' fingers is Green OA mandates.

You have created an IR, Talat, but we know that's not enough. You now have to help fill it, and your scepticism about university mandates and preference for conjectures about university peer-review consortia certainly is not helping to fill it.

Stevan Harnad
American Scientist Open Access Forum

There is no need to keep waiting for governmental OA mandates.

Harnad, Stevan (2005) The OA Policy of Southampton University (ECS), UK: the "Keystroke" Strategy [Putting the Berlin Principle into Practice: the Southampton Keystroke Policy] . Delivered at Berlin 3 Open Access: Progress in Implementing the Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities, University of Southampton (UK).

University OA mandates are natural extensions of universities' existing record-keeping, asset management, and performance-assessment policies. They complement research-funder OA mandates, and are the most efficient and productive way to monitor and credit compliance and fulfillment for both. Australia's Arthur Sale has done the most work on this. Please read what he has to say:

---

Arthur Sale wrote in the American Scientist Open Access Forum:

The evidence is quite clear that advocacy does not work by itself, and never has worked anywhere. To repeat the bleeding obvious once again: depositing in repositories is avoidable work under a voluntary regime, and like all avoidable work it will be avoided by most academics, even if perceived to be in their best interests, and even if the work is minor. The work needs to be (a) required and (b) integrated into the work pattern of researchers, so it becomes the norm. This is the purpose of mandates - to make it clear to researchers that they are expected to do this work.

My research and published papers show that mandates do work, and they take a couple of years for the message to sink in. Enforcement need only be a light touch - reporting to heads of departments for example. [See references below.]

At the risk of boring some, may I point to a similar case in Australia. All universities are required to produce an annual return to the Australian Government of publications in the previous year in the categories of refereed journal articles, refereed conference papers, books, and book chapters. The universities make this known to their staff (a mandate), and they all fill out forms and provide photocopies of the works. The workload is considerably more than depositing a paper in a repository. The scheme has been going for many years and is regarded as part of the academic routine. The data is used by Government to determine part of the university block grant. The result is near 100% compliance.

What I am doing in Australia is pressing for this already existing mandate to be extended to the repositories. If the researcher deposits in the repository, and the annual return is automatically derived from the repository, then (a) the researcher wins because it takes him/her less time, (b) it takes the administrators less time as the process is automated and only needs to be audited, and (c) the repository delivers its usual benefits for those with eyes to see. All we need is for the research office to promulgate such a policy in each university. It is in their own interests as well as the university's.

Arthur Sale
University of Tasmania

Swan, A. and Brown, S. (2005) Open access self-archiving: An author study. JISC Technical Report, Key Perspectives Inc. http://eprints.ecs.soton.ac.uk/10999/

Sale, Arthur (2006) Researchers and institutional repositories, in Jacobs, Neil, Eds. Open Access: Key Strategic, Technical and Economic Aspects, chapter 9, pages 87-100. Chandos Publishing (Oxford) Limited. http://eprints.utas.edu.au/257/

Sale, A. The Impact of Mandatory Policies on ETD Acquisition. D-Lib Magazine April 2006, 12(4). http://dx.doi.org/10.1045/april2006-sale

Sale, A. Comparison of content policies for institutional repositories in Australia. First Monday, 11(4), April 2006. http://firstmonday.org/issues/issue11_4/sale/index.html

Sale, A. The acquisition of open access research articles. First Monday, 11(9), October 2006. http://www.firstmonday.org/issues/issue11_10/sale/index.html

Sale, A. (2007) The Patchwork Mandate D-Lib Magazine 13 1/2 January/February http://www.dlib.org/dlib/january07/sale/01sale.html

Posted by Arthur Sale in the American Scientist Open Access Forum:

Yesterday, Australia held a Federal Election. The Australian Labor Party (the previous opposition) have clearly won, with Kevin Rudd becoming the Prime-Minister-elect.

What has this to do with the American Scientist Open Access Forum? Well the policy of the ALP is that the plans for the Research Quality Framework (the RQF - our research assessment exercise) will be immediately scrapped, and it will be replaced by a cheaper and metrics-based assessment, presumably a year or two later.

At first sight this is a setback for open access in Australia, because institutional repositories are not essential for a metrics-based research assessment. They just help improve the metrics. However, the situation may be turned to advantage, and there are several major pluses.

(1) Previous RQF grants should have ensured that every university in Australia now has a repository. Just mostly empty, or mostly dark, or both.
(2) The advisers in the Department of Education, Science & Technology (DEST) haven’t changed. The Accessibility Framework (ie open access) is still in place as a goal.
(3) A new metric-based evaluation could and should be steered to be a multi-metric based one. The ALP has already stated that it will be discipline-dependent.
(4) If the Rudd government is serious about efficiency in higher education, they could simply instruct DEST to require universities to put all their currently reported publications in a repository (ID/OA policy), from which the annual reports would be automatically derived. In addition all the desired publication metrics would also be derived, at any time. The Accessibility Framework would be achieved.

It should now be crystal clear to every university in Australia that citations and other measures will be key in the future. It should be equally clear that they should do everything possible to increase their performance on these measures. Any university that fails to immediately implement an ID/OA mandate (Immediate Deposit, Open Access when possible) in its institutional repository is simply deciding to opt out of research competition, or mistakenly thinks that it knows better. Although I suppose there is still the weak excuse that it is all too hard to understand or think about.

Here is the edited text of a press release by the shadow minister before the election. The boldface over some paragraphs is mine.

Arthur Sale
Professor of Computer Science
University of Tasmania

[BEGINS]
Senator Kim Carr
Labor Senator for Victoria
Shadow Minister for Industry, Innovation, Science and Research

Thursday, 15 November 2007 (58/07)
Building a strong future for Australian research

Federal Labor’s key research initiatives, announced during yesterday’s Campaign Launch, highlight our commitment to a research revolution.

[snip]

A Rudd Labor Government will be committed to rebuilding the national innovation system and, over time, doubling the amount invested in R&D in Australia.

· Labor will bring responsibility for innovation, industry, science and research into a single Commonwealth Department.

· Labor will develop a set of national innovation priorities to sit over the national research priorities. Together, these will provide a framework for a national innovation system, ensuring that the objectives of research programs and other innovation initiatives are complementary.

· Labor will abolish the Howard Government’s flawed Research Quality Framework, and replace it with a new, streamlined, transparent, internationally verifiable system of research quality assessment, based on quality measures appropriate to each discipline. These measures will be developed in close consultation with the research community. Labor will also address the inadequacies in current and proposed models of research citation. Labor’s model will recognise the contribution of Australian researchers to Australia and the world.

[snip]

· Labor recognises the importance of basic research in the creation of new knowledge, and also the value and breadth of Australian research effort across the humanities, creative arts and social sciences as well as scientific and technological disciplines.

The Howard Government has allocated $87 million for the implementation of the RQF. Labor will seek to redirect the residual funds to encourage genuine industry collaboration in research.

[snip]

---

SUMMARY: Three things need to be remedied in the UK's proposed HEFCE/RAE Research Evaluation Framework:
   (1) Ensure as broad, rich, diverse and forward-looking a battery of candidate metrics as possible -- especially online metrics -- in all disciplines.
   (2) Make sure to cross-validate them against the panel rankings in the last parallel panel/metric RAE in 2008, discipline by discipline. The initialized weights can then be fine-tuned and optimized by peer panels in ensuing years.
   (3) Stress that it is important -- indeed imperative -- that all University Institutional Repositories (IRs) now get serious about systematically archiving all their research output assets (especially publications) so they can be counted and assessed (as well as accessed!), along with their IR metrics (downloads, links, growth/decay rates, harvested citation counts, etc.).
   If these three things are systematically done -- (1) comprehensive metrics, (2) cross-validation and calibration of weightings, and (3) a systematic distributed IR database from which to harvest them -- continuous scientometric assessment of research will be well on its way worldwide, making research progress and impact more measurable and creditable, while at the same time accelerating and enhancing it.

---

Once one sees the whole report, it turns out that the HEFCE/RAE Research Evaluation Framework is far better, far more flexible, and far more comprehensive than is reflected in either the press release or the Executive Summary.

It appears that there is indeed the intention to use many more metrics than the three named in the executive summary (citations, funding, students), that the metrics will be weighted field by field, and that there is considerable open-mindedness about further metrics and about corrections and fine-tuning with time. Even for the humanities and social sciences, where "light touch" panel review will be retained for the time being, metrics too will be tried and tested.

This is all very good, and an excellent example for other nations, such as Australia (also considering national research assessment with its Research Quality Framework), the US (not very advanced yet, but no doubt listening) and the rest of Europe (also listening, and planning measures of its own, such as EurOpenScholar).

There is still one prominent omission, however, and it is a crucial one:

The UK is conducting one last parallel metrics/panel RAE in 2008. That is the last and best chance to test and validate the candidate metrics -- as rich and diverse a battery of them as possible -- against the panel rankings. In all other fields of metrics -- biometrics, psychometrics, even weather forecasting metrics – before deployment the metric predictors first need to be tested and shown to be valid, which means showing that they do indeed predict what they were intended to predict. That means they must correlate with a "criterion" metric that has already been validated, or that has "face-validity" of some kind.

The RAE has been using the panel rankings for two decades now (at a great cost in wasted time and effort to the entire UK research community -- time and effort that could instead have been used to conduct the research that the RAE was evaluating: this is what the metric RAE is primarily intended to remedy).

But if the panel rankings have been unquestioningly relied upon for 2 decades already, then they are a natural criterion against which the new battery of metrics can be validated, initializing the weights of each metric within a joint battery, as a function of what percentage of the variation in the panel rankings each metric can predict.

This is called "multiple regression" analysis: N "predictors" are jointly correlated with one (or more) "criterion" (in this case the panel rankings, but other validated or face-valid criteria could also be added, if there were any). The result is a set of "beta" weights on each of the metrics, reflecting their individual predictive power, in predicting the criterion (panel rankings). The weights will of course differ from discipline by discipline.

Now these beta weights can be taken as an initialization of the metric battery. With time, "super-light" panel oversight can be used to fine-tune and optimize those weightings (and new metrics can always be added too), to correct errors and anomalies and make them reflect the values of each discipline.

(The weights can also be systematically varied to use the metrics to re-rank in terms of different blends of criteria that might be relevant for different decisions: RAE top-sliced funding is one sort of decision, but one might sometimes want to rank in terms of contributions to education, to industry, to internationality, to interdisciplinarity. Metrics can be calibrated continuously and can generate different "views" depending on what is being evaluated. But, unlike the much abused "university league table," which ranks on one metric at a time (and often a subjective opinion-based rather than an objectiveone), the RAE metrics could generate different views simply by changing the weights on some selected metrics, while retaining the other metrics as the baseline context and frame of reference.)

To accomplish all that, however, the metric battery needs to be rich and diverse, and the weight of each metric in the battery has to be initialised in a joint multiple regression on the panel rankings. It is very much to be hoped that HEFCE will commission this all-important validation exercise on the invaluable and unprecedented database they will have with the unique, one-time parallel panel/ranking RAE in 2008.

That is the main point. There are also some less central points:

The report says -- a priori -- that REF will not consider journal impact factors (average citations per journal), nor author impact (average citations per author): only average citations per paper, per department. This is a mistake. In a metric battery, these other metrics can be included, to test whether they make any independent contribution to the predictivity of the battery. The same applies to author publication counts, number of publishing years, number of co-authors -- even to impact before the evaluation period. (Possibly included vs. non-included staff research output could be treated in a similar way, with number and proportion of staff included also being metrics.)

The large battery of jointly validated and weighted metrics will make it possible to correct the potential bias from relying too heavily on prior funding, even if it is highly correlated with the panel rankings, in order to avoid a self-fulfilling prophecy which would simply collapse the Dual RAE/RCUK funding system into just a multiplier on prior RCUK funding.

Self-citations should not be simply excluded: they should be included independently in the metric battery, for validation. So should measures of the size of the citation circle (endogamy) and degree of interdisciplinarity.

Nor should the metric battery omit the newest and some of the most important metrics of all, the online, web-based ones: downloads of papers, links, growth rates, decay rates, hub/authority scores. All of these will be provided by the UK's growing network of UK Institutional Repositories. These will be the record-keepers -- for both the papers and their usage metrics -- and the access-providers, thereby maximizing their usage metrics.

REF should put much, much more emphasis on ensuring that the UK network of Institutional Repositories systematically and comprehensively records its research output and its metric performance indicators.

But overall, thumbs up for a promising initiative that is likely to serve as a useful model for the rest of the research world in the online era.

References

Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated online RAE CVs Linked to University Eprint Archives: Improving the UK Research Assessment Exercise whilst making it cheaper and easier. Ariadne 35.

Brody, T., Kampa, S., Harnad, S., Carr, L. and Hitchcock, S. (2003) Digitometric Services for Open Archives Environments. Proceedings of European Conference on Digital Libraries 2003, pp. 207-220, Trondheim, Norway.

Harnad, S. (2006) Online, Continuous, Metrics-Based Research Assessment. Technical Report, ECS, University of Southampton.

Harnad, S. (2007) Open Access Scientometrics and the UK Research Assessment Exercise. Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics 11(1), pp. 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds.

Brody, T., Carr, L., Harnad, S. and Swan, A. (2007) Time to Convert to Metrics. Research Fortnight pp. 17-18.

Brody, T., Carr, L., Gingras, Y., Hajjem, C., Harnad, S. and Swan, A. (2007) Incentivizing the Open Access Research Web: Publication-Archiving, Data-Archiving and Scientometrics. CTWatch Quarterly 3(3).

See also: Prior Open Access Archivangelism postings on RAE and metrics

---

The following had been based on the earlier press alone, before seeing the full report. The press release had said:

"For the science-based disciplines, a new bibliometric indicator of research quality is proposed, based on the extent to which research papers are cited by other publications. This new indicator will be combined with research income and research student data, to drive the allocation of HEFCE research funding in these disciplines"

"For the arts, humanities and social sciences (where quantitative approaches are less developed) we will develop a light touch form of peer review, though we are not consulting on this aspect at this stage."

Comments:

(1) Citations, prior funding and student counts are fine, as candidate metrics. (Citations, suitably processed and contextually weighted, are a strong candidate, but it is important not to give prior funding too high a weight, or else it collapses the UK's Dual RAE/RCUK funding system into just a multiplier effect on prior RCUK funding.

(2) But where are the rest of the candidate metrics? article counts, book counts, years publishing, co-author counts, co-citations, download counts, link counts, download/citation growth rates, decay rates, hub/authority index, endogamy index... There is such a rich and diverse set of candidate metrics. It is completely arbitrary, and unnecessary, to restrict consideration to just an a-priori three.

(3) And how and against what are the metrics to be validated? The obvious first choice is the RAE panel rankings themselves: RAE has been relying on them unquestioningly for two decades now, and the 2008 RAE will be a (last) parallel panel/metric exercise.

(4) The weights on a rich, diverse battery of metrics should be jointly validated (using multiple regression analysis) against the panel rankings to initialize the weight of each metric, and then the weights should be calibrated and adjusted field by field, with the help of peer panels.

(5) It is not at all obvious that the humanities and social sciences do not have valid, predictive metrics too: They too write and read and use and cite articles and books, receive research funding, train postgraduate students, etc.

The metric RAE should be open-minded, open-ended forward-looking, in the digital online era, putting as many candidates as possible into the metric equation, testing the outcome against panel rankings, and then calibrating them, continuously, field by field, so as to optimise them. This is not the time to commit, a priori, to just 3 metrics, not yet validated and weighted, in science fields, nor to stick to panel review in other fields, again because of a priori assumptions about its metrics.

Let 1000 metric flowers bloom, initialize their weights against the parallel panel rankings (and any other validated or face-valid criteria) and then set to work optimising those initial weights, assessing continuously, field by field, across time. The metrics should be stored and harvestable from each institution's Open Access Institutional Repository, along with the research publications and data on which they are based.

---

Stevan Harnad
American Scientist Open Access Forum