Open Access Archivangelism

SM: "Stevan asserts that researchers who cannot afford access to the published version of articles are perfectly happy with the self-archived author's final version.

"Interestingly, in our survey of learned society members Sue Thorn and I found that most of our 1368 respondents did not, in fact, use authors' self-archived versions even when they had no access to the published version - 53% never did so, and only 16% did so whenever possible."

Sally does not always put her survey questions in the most transparent way.

If you really want to find out whether or not researchers are "happy" with the author's refereed, accepted final draft when they lack access to the published version you have to ask them that:

(1) "How often do you encounter online, in a search or otherwise, the author's free refereed, accepted final draft of a potentially relevant article to which you (or your institution) cannot afford paid full-text access?"

(2) "If you lack access to the published version of such a potentially relevant article, would you prefer to have no access at all, or access to the author's free refereed, accepted final draft?"

(3) "If you would prefer access to the author draft over no access at all, how strongly would you prefer it over no access at all?

That's the forthright, transparent way to put the exact contingencies we are addressing. No equivocation or ambiguity.

In contrast, I am sure that Sally's question about "How often do you use author drafts?" was just that: "How often do you use author drafts?" Not "How often do you encounter a potentially relevant article, but decline to use it because you only have access to the author draft and not the published version?"

Sally's responses -- which seem to say that 47% do use the author draft and 53% do not use the author draft -- fail to reveal whether the 53% who fail to use the author draft indeed fail to do so because, even though they have found a potentially relevant author draft free online, and lack access to the publisher draft, they prefer to ignore the potentially relevant author draft (this would be very interesting and relevant news if it were indeed true), or simply because they happen to be among the 53% who had never encountered a potentially relevant author draft free online when they had no access to the publisher version. (And could the 16% who did use the author draft "wherever possible" perhaps correspond to the well-known datum that only about 15% of all articles have freely accessible author drafts online)?

Surveys that obscure these fundamental details under a cloud of ambiguity are not revealing researchers' preferences but their own.

Stevan Harnad
American Scientist Open Access Forum

Professor Henk Schmidt, Rector of Erasmus University, Rotterdam, in an interview about Open Access conducted by Leo Waaijers, has announced that he proposes to adopt a Green Open Access self-archiving mandate for Erasmus University's Institutional Repository, RePub:

HS: "I intend obliging our researchers to circulate their articles publicly, for example no more than six months after publication... if possible in collaboration with publishers via the 'Golden Road' and otherwise without the publishers via the 'Green Road'... [We] can’t just oblige researchers to publish in Open Access journals. It has not yet been established that there are enough prestigious Open Access journals, but – until there are – prescribing the 'Green Road' seems to me an excellent idea... even though it’s a bit of a problem that this will lead to two versions of the article being circulated."

This is excellent news, but let me dispel the misapprehension that it will entail even a "bit of a problem":

Professor Schmidt states, quite rightly, that since most journals are not Gold OA (and especially few of the top journals are Gold OA), universities (and funders) cannot achieve OA by obliging their authors to publish in Gold OA journals. 

However, as Professor Schmidt notes, universities (and funders) can require (mandate) that their authors make their articles Green OA by depositing them in their institutional OA repositories (of which every Dutch university now has one) immediately upon publication -- allowing an embargo on setting access to the deposit for a maximal permissible interval (say, 6 months) for those journals that do not yet already endorse immediate OA. (63% of journals already do endorse immediate OA, and that includes virtually all the top journals. And 79 institutions, 18 departments and 42 research funders worldwide already mandate Green OA).

All of this is extremely welcome, and spot-on. I would add only that the difference between the author's peer-reviewed, revised, and accepted final draft (the postprint) and the publisher's version-of-record (PDF) is negligible for active researchers (especially those for whom OA is really intended, namely, the many would-be users whose institutions cannot afford subscription access to the journal in which an article happens to be published); moreover, most researchers are already quite accustomed to receiving and using prepublication hard copies (and, lately, email versions) of final drafts rather than waiting for the journal to appear.

Professor Schmidt adds:

HS: "It may well take a year before your article appears in a journal. But I do expect the time pressure to increase. In that case, circulating your work by uploading it to a repository could speed things up." 

As noted, OA is not merely for the sake of earlier access during the publication lag (most journals now offer access to the online version immediately, and even to the author's final draft -- but to subscribers only). The primary motivation for OA is the need for access to journals to which the would-be user's institution cannot afford to subscribe.

HS: "I don’t... upload [my articles to] the university’s repository... I had never even consulted the repository. I did try it once a few weeks ago and realised that none of my publications are in there. It was just too awkward, and I’ll now probably wait quite a long time before I try it again. I’m just too busy for this kind of experimentation. It really does need to be made a lot simpler... it would make a difference if it were... easy to deposit your PDF... Either that or somebody has to do it for you. [Our researchers] are of course used to registering the metadata in Metis. But it would make a difference if it were then easy to deposit your PDF..."

This passage is a bit ambiguous as to whether Professor Schmidt is referring here to (1) consulting the repository, in search of an article, as a user, or to (2) depositing one's own articles in the repository, as an author.

(1) Consultation: Institutional repositories (IRs) can be consulted directly (for institution-internal record-keeping, monitoring or showcasing purposes) but that is certainly not the primary purpose of either IRs or OA. The way most OA IR deposits are consulted by potential users is not by going to each individual IR to search! The IRs are OAI-compliant, hence interoperable, and hence they are harvested by central search services (such as OAIster, Base, Scirus, Scopus, PubMed, Citeseer, Celestial, and even Google Scholar) so they become jointly searchable by users as if they were all in one and the same global repository.

(2) Deposit: To find out how quick and easy deposit really is, one must actually have deposited an article in an IR. It is certainly as simple as depositing the metadata in Metis -- moreover, software can easily import/export directly from one to the other (Metis to IR or IR to Metis), automatically. So the (few) keystrokes only ever need to be done once. 

Carr, L. and Harnad, S. (2005) "Keystroke Economy: A Study of the Time and Effort Involved in Self-Archiving." 

(It's fine to have the keystrokes done by proxy -- by an assistant, a student, a librarian -- if an institution wishes, but it is not clear that there is even the need to do so: Do researchers need proxies to deposit in Metis? It's virtually the same thing.)

Nor is the publisher's PDF needed. The author's final draft is what needs to be deposited, and the author has that at his fingertips as soon as a final draft is accepted for publication (i.e., when no more revisions are required).

Metadata are metadata, and the same metadata are needed for OA IR deposit as for Metis (author, title, date, journal, etc.) registration. The publisher's PDF is both unnecessary and undesirable (because it has more access restrictions than the author's refereed. accepted final draft.)

Moreover, the most successful university deposit mandates (such as the mandate at University of Liège) have combined the functions of the OA IR and (their equivalent of) Metis: The form that the deposit mandate takes is that it is in the IR that the researcher must deposit for performance review!

Here is how the Rector of U Liege, Professor Bernard Rentier, worded the Liège mandate:

-- deposit in ORBi will be mandatory as soon as the article is accepted by the journal

-- starting October 1st, 2009, only those references introduced in ORBi will be taken into consideration as the official list of publications accompanying any curriculum vitae for all evaluation procedures 'in house' (designations, promotions, grant applications, etc.) 

-- Wherever publisher agreement conditions are fulfilled, the author will authorize setting access to the deposit as open access 

-- For closed access deposits, the institutional repository will have an EMAIL EPRINT REQUEST BUTTON which allows the author to fulfill individual eprint requests. 

In response to the question "If uploading material to a repository were actually made a lot simpler, would they all do it, or would something else have to happen?" Professor Schmidt replied:

HS: "I think it will be necessary to impose an obligation so as to get them used to it. But if it were really simple and it took only a single action to upload the publication to the repository and register it in Metis for the annual report, then they’d come on board."

This reply is spot-on, on all counts: Researchers will not deposit unless it is mandated, but if it is mandated, they will indeed deposit (95%), and the vast majority will do so willingly (81%).

What Professor Schmidt may not have realized is that deposit is already easy, just a few minutes worth of keystrokes, and virtually identical to the keystrokes for registering in Metis. So all that needs to be done is to mandate deposit in the Erasmus IR, as the prerequisite for performance evaluation, and automatically export the metadata from the IR to Metis!

All universities considering the adoption of a Green Open Access mandate are urged to join EOS (Enabling Open Scholarship). The chairman of the EOS Board is Professor Bernard Rentier (Rector of the University of Liège), and the Coordinator is Dr. Alma Swan (of Southampton and Key Perspectives Inc). These are the two most far-sighted and dynamic leaders in the international OA mandate movement, and with their help university IRs and mandates will be the most effective they can be: 

EnablingOpenScholarship (EOS) is an organisation for universities and research institutions worldwide. The organisation is both an information service and a forum for raising and discussing issues around the mission of modern universities and research institutions, particularly with regard to the creation, dissemination and preservation of research findings
 
The aim of EOS is to further the opening up of scholarship and research that we are now seeing through the growing open access, open education, open science and open innovation movements.

 
Stevan Harnad
American Scientist Open Access Forum

---

SUMMARY: Leo Waaijers suggests in Ariadne that funder OA mandates impose "unfair conditions" on authors because there are not enough Gold OA journals to publish in. So instead, funders should fund research on alternative "non-proprietary peer review" services. This is based on a misunderstanding of OA mandates (which are for author self-archiving of the final peer-reviewed drafts of articles published in peer-reviewed non-OA journals, i.e., Green OA, not Gold OA) as well as a misunderstanding of peer review, which needs to be neither reformed, replaced nor redirected: existing peer-reviewed articles need merely to be self-archived.

---

Leo Waaijers wrote in Ariadne, "Publish and Cherish with Non-proprietary Peer Review Systems":

LW: "More and more research funders require open access to the publications that result from research they have financed... Although there is a steadily growing number of peer-reviewed Open Access journals... the supply fails to keep pace with the demand... [A]s authors cannot all publish in Open Access journals... Open Access-mandating funders impose unfair conditions on authors."

There is a profound misunderstanding here. Funders who mandate Open Access (OA) impose no "unfair conditions." What they mandate is the self-archiving of all published articles ("Green OA"), not the publishing of all articles in Open Access (OA) journals ("Gold OA").

It cannot be pointed out often enough that Gold OA is not the sole or primary way to provide OA: The incomparably faster, easier, cheaper and surer way to provide OA is for authors to self-archive articles published in non-OA journals by depositing them in the author's Institutional Repository (IR) (Green OA). And that is exactly what funders and institutions are mandating. No need to wait for all publishers to convert to Gold OA. Hence no "unfairness."

Most of the top institutions already have IRs. All the rest can create them with free (and extremely powerful) software; moreover, the DEPOT repository is available (now internationally) for self-archiving by author's whose institutions do not yet have an IR or for authors who do not have an institution.

Hence Leo's point about "unfair conditions" by funder mandates is either misinformed or misinforming.

LW: "[A] conversion... from proprietary to non-proprietary systems of peer review... can be speeded up if disciplinary communities, universities, and research funders actively enter the market of the peer review organisers by calling for tenders and inviting publishers to submit proposals for a non-proprietary design of the peer review process"

This is a non sequitur. What is needed is Open Access to peer-reviewed articles (2.5 million articles per year, published in 25,000 peer-reviewed journals). Most of those articles are available today only via toll-access (institutional subscriptions). The solution is neither to keep waiting for those journals to convert to Gold OA, nor to try to invent alternative forms of peer review. That would be like thinking that the way to solve the problem of public smoking is not to mandate no-smoke zones but to invent an alternative form of cigarette, or instead of mandating medicare to invent alternative forms of medicine, or instead of mandating recycling to invent an alternative form of garbage.

Not only is there no need to try to replace existing journals and their peer review system, but the problem with peer review is not that it is a proprietary service (for which the service-provider -- the journal -- needs to be paid) but that the byproducts of the service -- the peer-reviewed articles -- are not openly accessible to all would-be users.

And the solution is for authors to self-archive (the final, peer-reviewed drafts of) their peer-reviewed articles (Green OA) -- and for authors' institutions and funders to mandate it (submit submit tenders soliciting research proposals for alternatives to peer review!).

There is no need (and certainly no time) to wait to re-invent peer review in new hands and try to persuade (mandate?) authors to publish in these new "non-proprietary systems of peer review" instead of their existing peer-reviewed journals; nor is there the need or time to persuade publishers (already sluggish about converting to Gold OA) or others to turn to "tenders" inviting them to design a new system of peer review.

What is needed is for authors' institutions and funders to mandate Green OA self-archiving, a non-hypothetical solution that has already been tested, works, and can scale to all 2.5 million articles published annually in the planet's 25,000 peer-reviewed journals as quickly and surely as it can be mandated.

LW: "The [funder]... requires that... published research appears as openly accessible peer-reviewed articles."

The way for the funder to require that published research should appear as openly accessible peer-reviewed articles is to mandate that the author's final peer-reviewed draft (not the publisher's proprietary PDF) must be self-archived in the author's OA IR immediately upon acceptance for publication. That's all. No need for "tenders" for "non-proprietary peer review."

And this is exactly what most of the existing and proposed OA mandates (by funders as well as institutions) require -- not the "unfair condition" of having to find and publish in a suitable Gold OA journal. Hence there is no need at all to create new Gold OA journals, let alone new forms of peer review, in order to provide universal OA, today. All that is needed is Green OA mandates.

But Leo instead recommends a highly speculative alternative for reforming peer review that is not only untested and unnecessary, but contains within the proposal itself the signs that it misunderstands how peer review itself works, and why it is needed for research and researchers:

LW: "In order to have appropriate review procedures in place to process these articles... The reviewing process must be independent, rigorous and swift..."

So far, so good (except Leo does not say how peer review should be speeded up, given the number of papers submitted daily for peer review, the number of qualified peer reviewers available, and the number of their waking hours that researchers can devote to peer reviewing. Let us agree, however, that there are indeed ways to make this process faster and more efficient in the online era.)

But now, the proposed system (for which, Leo recommends, funders should solicit proposals, instead of mandating Green OA):

LW: "As a result of the reviewing process, articles will be marked 1 [low] to 5 [high]... In review procedures the [funder] will weigh articles with marks 3, 4 and 5 as if they were published in journals with impact factors 1-3, 4-8 and 9-15 respectively... For articles marked 3 to 5 adequate Open Access publication platforms must be available (e.g. new Open Access journals). Alternatively, authors may publish their articles in any existing OA journal. Upon publication all articles will be deposited in a certified (institutional) repository."

This speculative notion of peer review imagines that peer review consists of giving papers marks.

(It does not. It involves assessing their contents and making concrete recommendations as to what needs to be done by way of revision -- if they are potentially acceptable -- and reasons for rejection if not. What users expect and need from journals is an all-or-none indication of whether an article has met that journal's established quality-standards for acceptance. Any internal ratings the referees might have used in the process of coming to a recommendation on acceptance or rejection are not for the user but the editor. The real ranking for the user -- and author -- is in the quality hierarchy among journals. Their quality standards -- meaning what percentage of articles meet their acceptance criteria -- are reflected in their track-records and known to users. They are also (sometimes) reflected in the journals' impact factor. But that impact factor -- which is objectively determined by journals' average citation counts -- is certainly not the same thing as referees' internal ratings.)

Leo suggests that these "marks" should be given (by someone), with marks 3-5 standing in for having been published in peer-reviewed journals with corresponding "impact factors," for which there must be a corresponding Gold OA journal (either new or existing) for them to appear in (created to ensure that "unfair conditions" are not imposed by the deposit requirement).

Then the paper can be deposited in a "certified" IR. (One wonders why? Since all articles, in Leo's hypothetical scenario, would be published in Gold OA journals that this peer-review reform proposal had miraculously generated, why do they need to be deposited in IRs at all -- "certified" or otherwise -- since they are all already OA?)

In other words, Leo has invented an imaginary problem with deposit mandates (viz, "there aren't enough Gold OA journals") -- whereas the mandates are not to publish in Gold OA journals but to deposit in Green OA IRs. And then he has invented an imaginary solution to the problem (viz, create new "non-proprietary peer-review services" and then publish the outcome in new Gold OA journals that no longer need to bother to implement peer review). All in order to be able to "fairly" deposit them in "certified" IRs?

My guess is that this rather complicated conjectural solution ("non-proprietary peer-review services") to an imagined problem ("unfair Gold OA mandates") was inspired by Leo's incorrect six assumptions:

(1) that what needs to be deposited in an IR in order to provide OA is the publisher's proprietary PDF, i.e., the canonical version of record (whereas what needs to be deposited for OA is just the peer-reviewed final draft ("postprint" which is merely a supplement to -- not a substitute for -- the canonical version of record);

(2) that depositing in Closed Access during any publisher embargo on OA would not fulfill the deposit mandate (whereas in fact it would, and especially with the help of the IR's semi-automatic "email eprint request" button to tide over research usage needs during any embargo on OA);

(3) that peer review is largely just a rank-assignment (whereas in reality peer review is a dynamic, interactive, answerable system of detecting and correcting errors so as to meet a given journal's established quality standards);

(4) that OA means just Gold OA publishing (whereas Green OA self-archiving too is OA);

(5) that what research and researchers really need is not just Green OA, but Gold OA (whereas what they need is OA) and hence that

(6) IRs are just for digital preservation (whereas OA IRs are for OA-provision today, and digital preservation of the canonical version of record is an entirely different matter, unrelated to OA until and unless subscriptions become unsustainable, thus making the author's peer-reviewed final draft the canonical version of record!).

Stevan Harnad
American Scientist Open Access Forum

The Undergraduate Science Librarian wrote:

"For a small institution like mine, having our own institutional repository might not make sense. We probably don’t have the library staff to run it well... [F]or many of our faculty, their only way of archiving their papers may be to post them on their own personal website, where they might not be as easy to find..."

REPLY:

(1) If a campus has the infrastructure to host a website at all (as SUNY GENESEO clearly does), it has the infrastructure to host its own institutional repository (IR) for its own research article output. (If not, it probably does not have the insfrastructure to conduct research at all.)

(2) Library staff are not needed to host an IR.

(3) All that's needed is some disk space on one of the institution's webservers, plus the installation of (free, open-source) IR software.

(4) Even individuals can install and host the IR software on their own PCs or their personal websites. (There is even a (free) Microsoft Windows version of EPrints.)

(5) All EPrints IR installations are OAI-compliant, hence harvested for searchability by all of the major search engines: scirus, scopus, citeseerx, citebase, oaister, base, etc., as well as google and google scholar (the major ports of entry for all IRs). (Worries about IR deposits not being "easy to find" are based on a profound misunderstanding of search over distributed OAI-harvestable contents.)

(5) CalTech alone hosts 26 EPrints IRs.

(6) The importance of institutional IR installations is as the convergent locus of deposit for Open Access self-archiving mandates (without which all IRs, personal or institutional, are doomed to lie fallow).


Stevan Harnad
American Scientist Open Access Forum

The ROAR Registry of Open Access Repositories has just been upgraded to the full power of the EPrints software's remarkable functionality.

Please come and explore the power of ROAR to display and track repository size, contents and growth across time, by country, repository type, and many performance parameters.

This will make it possible to monitor and analyze repository growth worldwide, and to encourage institutions to create their own repositories and adopt deposit mandates for filling them.

Please also register your repository if it is not yet in ROAR. The new functionality means you will own your record and will be able to update it as it progresses.

And register your institution's Open Access Mandate in ROARMAP, ROAR's companion service, tracking mandate growth worldwide.

---

Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

Phil Davis: "An interesting bit of research, although I have some methodological concerns about how you treat the data, which may explain some inconsistent and counter-intuitive results, see: http://j.mp/8LK57u A technical response addressing the methodology is welcome."

Thanks for the feedback. We reply to the three points of substance, in order of importance:

(1) LOG RATIOS: We analyzed log citation ratios to adjust for departures from normality. Logs were used to normalize the citations and attenuate distortion from high values. Moed's (2007) point was about (non-log) ratios that were not used in this study. We used log citation ratios. This approach loses some values when the log tranformation makes the denominator zero, but despite these lost data, the t-test results were significant, and were further confirmed by our second, logistic regression analysis. It is highly unlikely that any of this would introduce a systematic bias in favor of OA, but if the referees of the paper should call for a "simpler and more elegant" analysis to make sure, we will be glad to perform it.

(2) EFFECT SIZE: The size of the OA Advantage varies greatly from year to year and field to field. We reported this in Hajjem et al (2005), stressing that the important point is that there is virtually always a positive OA Advantage, absent only when the sample is too small or the effect is measured too early (as in Davis et al's 2008 study). The consistently bigger OA Advantage in physics (Brody & Harnad 2004) is almost certainly an effect of the Early Access factor, because in physics, unlike in most other disciplines (apart from computer science and economics), authors tend to make their unrefereed preprints OA well before publication. (This too might be a good practice to emulate, for authors desirous of greater research impact.)

(3) MANDATED OA ADVANTAGE? Yes, the fact that the citation advantage of mandated OA was slightly greater than that of self-selected OA is surprising, and if it proves reliable, it is interesting and worthy of interpretation. We did not interpret it in our paper, because it was the smallest effect, and our focus was on testing the Self-Selection/Quality-Bias hypothesis, according to which mandated OA should have little or no citation advantage at all, if self-selection is a major contributor to the OA citation advantage.

Our sample was 2002-2006. We are now analyzing 2007-2008. If there is still a statistically significant OA advantage for mandated OA over self-selected OA in this more recent sample too, a potential explanation is the inverse of the Self-Selection/Quality-Bias hypothesis (which, by the way, we do think is one of the several factors that contribute to the OA Advantage, alongside the other contributors: Early Advantage, Quality Advantage, Competitive Advantage, Download Advantage, Arxiv Advantage, and probably others).

The Self-Selection/Quality-Bias (SSQB) consists of better authors being more likely to make their papers OA, and/or authors being more likely to make their better papers OA, because they are better, hence more citeable. The hypothesis we tested was that all or most of the widely reported OA Advantage across all fields and years is just due to SSQB. Our data show that it is not, because the OA Advantage is no smaller when it is mandated. If it turns out to be reliably bigger, the most likely explanation is a variant of the "Sitting Pretty" (SP) effect, whereby some of the more comfortable authors have said that the reason they do not make their articles OA is that they think they have enough access and impact already. Such authors do not self-archive spontaneously. But when OA is mandated, their papers reap the extra benefit of OA, with its Quality Advantage (for the better, more citeable papers). In other words, if SSQB is a bias in favor of OA on the part of some of the better authors, mandates reverse an SP bias against OA on the part of others of the better authors. Spontaneous, unmandated OA would be missing the papers of these SP authors.

There may be other explanations too. But we think any explanation at all is premature until it is confirmed that this new mandated OA advantage is indeed reliable and replicable. Phil further singles out the fact that the mandate advantage is present in the middle citation ranges and not the top and bottom. Again, it seems premature to interpret these minor effects whose unreliability is unknown, but if forced to pick an interpretation now, we would say it was because the "Sitting Pretty" authors may be the middle-range authors rather than the top ones...

Yassine Gargouri, Chawki Hajjem, Vincent Lariviere, Yves Gingras, Les Carr, Tim Brody, Stevan Harnad

Brody, T. and Harnad, S. (2004) Comparing the Impact of Open Access (OA) vs. Non-OA Articles in the Same Journals. D-Lib Magazine 10(6).

Davis, P.M., Lewenstein, B.V., Simon, D.H., Booth, J.G., Connolly, M.J.L. (2008) Open access publishing, article downloads, and citations: randomised controlled trial British Medical Journal 337:a568

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) 39-47.

Moed, H. F. (2006) The effect of 'Open Access' upon citation impact: An analysis of ArXiv's Condensed Matter Section Journal of the American Society for Information Science and Technology 58(13) 2145-2156

Universities UK recommends making all the research outputs submitted to the UK's new Research Excellence Framework (REF) Open Access (OA).

The UUK's recommendation is of course very welcome and timely.

All research funded by the RCUK research councils is already covered by the fact that all the UK councils already mandate OA. It is this policy, already adopted by the UK, that the US is now also contemplating adopting, in the form of the proposed Federal Research Public Access Act (FRPAA), as well as the discussion in President Obama's ongoing OSTP Public Access Policy Forum.

But if HEFCE were to follow the UUK's recommendation, it would help to ensure Open Access to UK research funded by the EU (for which OA is only partially mandated thus far) and other funders, as well as to unfunded research -- for which OA is mandated by a still small but growing number of universities in the UK and worldwide. (The same UUK proposal could of course be taken up by UK's universities, for once they mandate OA for all their research output, all UK research, funded and unfunded, becomes OA!)

There is an arbitrary constraint on REF submissions, however, which would greatly limit the scope of an OA requirement (as well as the scope of REF itself): Only four research outputs per researcher may be submitted, for a span covering at least four years, rather than all research output in that span.

This limitation arises because the REF retains the costly and time-consuming process of re-reviewing, by the REF peer panels, of all the already peer-reviewed research outputssubmitted. This was precisely what it had earlier been proposed to replace by metrics, if they prove sufficiently correlated with -- and hence predictive of -- the peer panel ranklings. Now it will only be partially supplemented by a few metrics.

This is a pity, and an opportunity lost, both for OA and for testing and validating a rich and diverse new battery of metrics and initializing their respective weights, discipline by discipline. Instead, UUK has endorsed a simplistic (and likewise untested and arbitrary) a-priori weighting ("60/20/20 for outputs, impact and environment").

Harnad, S. (2009) Open Access Scientometrics and the UK Research Assessment Exercise. Scientometrics 79 (1) Also in Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics 11(1), pp. 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds. (2007)

---

Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research

Authors: Yassine Gargouri, Chawki Hajjem, Vincent Larivière, Yves Gingras, Les Carr, Tim Brody, Stevan Harnad
Click here for preprint
Abstract: Articles whose authors make them Open Access (OA) by self-archiving them online are cited significantly more than articles accessible only to subscribers. Some have suggested that this "OA Advantage" may not be causal but just a self-selection bias, because authors preferentially make higher-quality articles OA. To test this we compared self-selective self-archiving with mandatory self-archiving for a sample of 27,197 articles published 2002-2006 in 1,984 journals. The OA Advantage proved just as high for both. Logistic regression showed that the advantage is independent of other correlates of citations (article age; journal impact factor; number of co-authors, references or pages; field; article type; or country) and greatest for the most highly cited articles. The OA Advantage is real, independent and causal, but skewed. Its size is indeed correlated with quality, just as citations themselves are (the top 20% of articles receive about 80% of all citations). The advantage is greater for the more citeable articles, not because of a quality bias from authors self-selecting what to make OA, but because of a quality advantage, from users self-selecting what to use and cite, freed by OA from the constraints of selective accessibility to subscribers only.