Open Access Archivangelism

---

SUMMARY: The loophole in the proposed FRPAA, RCUK and European Commission self-archiving mandates is that publisher embargo policies can open-endedly determine when the research is deposited. Two small but crucial changes can correct this: (1) Mandate immediate deposit for all articles immediately upon acceptance for publication (rather than mandating deposit only after an interval determined by the publishers, or after a fixed 6-month embargo). (2) Only recommend (rather than mandate) immediately setting the access to each deposit as Open Access (OA): allow the option of setting access instead as Closed Access (CA) where deemed necessary. This separate "Dual Deposit/Release" (DD/R) policy requires that the full text and metadata be deposited immediately, not after a delay; and it requires that the metadata (only) be immediately set to Open Access, so they are visible and accessible worldwide immediately upon deposit; but it allows the option of not setting access to the full-text itself as OA immediately. It thereby transfers the force of any embargo/delay onto OA access-setting instead of leaving it on the deposit itself, which must be immediate. Nothing at all is lost, relative to the delayed-deposit mandates currently being proposed, but what is gained is systematic immediate deposit of the full texts -- plus the capability of providing individual access via user email-eprint requests to the author, semi-automatized by the Institutional Repository software.

---

ANON: " My primary aim is to develop a model OA legislation... I am consulting the FRPAA 2006 bill and other texts. In formal legislation, we generally avoid any loopholes or pitfalls that may have a counter-effect on the very objectives of the legislation."

Pre-emptively avoiding loopholes and pitfalls is indeed the optimal strategy. And it is precisely for that reason that instead of recommending that you emulate exactly the proposals of the FRPAA, the RCUK, or the European Commission, which all still have some needless weaknesses as well as loopholes, I recommend that you change two of their parameters to make your own policy recommendation the optimal one:

The two changes are small but absolutely crucial:

(1) Mandate immediate deposit for all accepted papers (rather than mandating deposit only after an interval determined by the publishers, or after a fixed 6-month embargo).

(2) Only recommend (rather than mandate) immediately setting the access to each deposit as Open Access (OA): allow the option of setting access instead as Closed Access (CA) where deemed necessary (i.e., where it is thought that immediately setting access as OA would contravene the author's copyright agreement with the publisher).

This separate Dual Deposit/Release (DD/R) policy requires that the full text and metadata be deposited immediately, not after a delay; and it requires that the metadata (only) be immediately set to Open Access, so they are visible and accessible worldwide immediately upon deposit; but it allows the option of not setting access to the full-text itself as OA immediately. In effect, it thereby transfers the force of any embargo/delay onto OA access-setting instead of onto the deposit itself, which (to repeat) must be immediate.

The US is probably the only country in the world that has enough collective weight to go even further, because it represents such a large proportion of the authorship of so many journals, and of the funding of so much published research: The US can, I think, with impunity put a cap (of six months, or even less) on the maximal allowable embargo period. The Wellcome Trust has already done this, telling their authors: "If your publisher does not agree to a cap of 6 months on the embargo, choose another publisher!" Wellcome, however, has the advantage, for this admirable and bold move, that they are a private funder. So all an author can do, if he does not like the Wellcome's terms, is to choose another funder.

But the US Federal funding agencies are governmental, so they can be lobbied, not only by publishers but also by researchers and their institutions, if they object to the terms: And researchers may well object to being told that they cannot publish in the journal of their choice if it does not agree to a 6-month embargo cap!

But the Dual Deposit/Release (DD/R) mandate removes even this possible obstacle to the adoption of the policy; for it does not require either switching publishers or contravening the terms of their copyright agreement. It allows embargoed access-setting but mandates immediate deposit (and then semi-automatic email-eprint requests in the repository software will take care of any gap-period in which the metadata are visible and accessible but the full-text is Closed Access).

I think this slight parametric change answers your remaining concerns.

ANON: "In the case of FRPAA 2006, one has to be dependent on the mercy of the journal publishers, presuming that 93% of them will immediately allow the researcher to deposit his research in OA. Please let us know how you have reached that figure."

That's 93% of journals, not publishers (as some publishers publish more journals than others). I have reached the figure on the basis of each journal's own official author self-archiving policy, as indexed, by journal as well as by policy, in the Romeo directory, for over 9000 journals, including all the principal ones.

But note that the Dual Deposit/Release (DD/R) policy would be immune to publisher objections and policies even if 100% of journals had had an embargo, because it does not mandate Open Access! It simply mandates immediate repository deposit, which is an internal record-keeping matter, and that is in no way the business of publishers or anyone else.

What gives the Dual Deposit/Release (DD/R) policy its power is that (a) it also mandates the immediate deposit of the bibliographic metadata (author, title, journal name, date, etc.), which no one can embargo, and (b) the webwide, uniform visibility and accessibility of the metadata immediately makes the existence of the article visible as well, and allows each individual would-be user who finds that the article is Closed Access to use the semi-automatic eprint-request button we have designed for the GNU Eprints Institutional Repositories (since implemented for the Dspace repository software as well, and easily implementable by all other repository softwares) to request that the author email him the full-text.

So whereas the dual mandatory-immediate-deposit plus optional-delayed-release policy does not mandate OA per se, it instead -- in exchange for immunity from all possible objections on the grounds of either copyright or author choice of journal -- requires immediate deposit in all cases, no exceptions or delays, and relies on (trivial) technology to fill any access-needs during any embargo-gap. (It is unlikely, by the way, that once the Dual Deposit/Release mandate is widely adopted, embargoes will survive for very long -- and in the meanwhile, all usage needs will be filled by the software technology, using the unique capabilities of the new medium).

If the adoption of the proposed mandates instead keeps being delayed and delayed, as it has been now for the past 3 years, for the very reasons you are here raising, everywhere (except the Wellcome Trust plus the six institutions that have already mandated self-archiving) -- and even if the mandates are adopted as delayed deposit mandates (as the Wellcome policy is, with a cap of 6 months on the permissible delay) -- the result will be that embargoes will only become more deeply entrenched instead of being defused, as they are by the mandatory-immediate-deposit plus optional-delayed-release policy, which fixes both the date of deposit and the practise of depositing for all authors and papers uniformly and optimally, allowing no exceptions.

It may be useful for you to bear in mind -- and also to inform your policy advisees -- that the real thing that has been holding back 100% OA for a dozen years now has not been copyright policy but keystrokes: Authors have not been depositing their articles in their institutional (or central) repositories. Two international, cross-disciplinary surveys for JISC by Key Perspectives have shown that most authors don't and won't self-archive spontaneously, but 95% of them will do see if/when mandated to do so by their funders and/or their institutions. And the actual experience of the (few) institutions that have already gone ahead and mandated self-archiving confirms the predictions of these JISC surveys.

The self-archiving mandate should be thought of as a mandate to perform those all-important keystrokes -- for record-keeping as well as metadata-exposing purposes -- immediately upon acceptance for publication, which is the natural date, the date when usage of the research can and should begin, and the date that provides a fixed, objective landmark for all papers.

(Swan et al. have written an excellent strategic analysis of the question of institutional versus central repositories: There is also excellent further policy guidance here [1] and [2] )

Once a "Keystroke Mandate" becomes widely implemented, 100% OA will not be far behind; but keep delaying the keystrokes because of other worries, or over-reaching, and you only keep delaying 100% OA.

The timing of the deposit is now independent of any publisher embargo periods, which instead apply only to the timing of the Open Access-setting for the (Closed Access) full text. The policy need make no mention of embargo periods, except to recommend setting access OA immediately or as soon as possible.

ANON: "Nevertheless, a public law can not be dependent on the assumptions, if this 93% of publishers tomorrow decide not to allow researchers to deposit in OA, the law is then helpless."

Not the law I have just described! It is completely unaffected. Please also see also the self-archiving FAQ on this very question.

ANON: "In summary, there is a difficulty in constructing a law that guarantees 100% of OA even if the researcher/research-institute is willing to do so."

What is needed right now is not a law that guarantees 100% OA; what is needed is a mandate that guarantees 100% immediate deposit of the full-texts and 100% immediate visibility of the metadata. Nature will take care of the rest.

But delaying and delaying the adoption of the policy (or weakening it to ineffectuality by allowing fixed or open-ended deposit-delays in order to comply with every possible embargo) is a way of guaranteeing that 100% OA will remain a long, long way off!

The obstacle is keystrokes, and what is needed is a keystroke mandate, i.e., an immediate-deposit mandate, for internal record-keeping and external visibility purposes only. That is not an "OA law" but a law specifying conditions on the fulfilment conditions for receiving public funding for research: The resulting publications must be immediately deposited in an OAI Repository and must immediately make their metadata visible.

Once that law is in place, 100% OA will quickly ensue of its own accord. But keep delaying the law by agonising instead over what will allow immediate OA for the full text in all cases -- or what interval can be agreed upon for delayed deposit -- and 100% OA will be delayed for yet another needless decade,

ANON: "Moreover, the semi-automatic email is not open access as the term is being used in the legal texts or definition of OA. It depends on the will of the author/publishers et al and access is platform/ technology depended. A legislation has to be platform/ technology neutral/ independent."

You are quite right that immediate deposit, immediate access to the metadata, plus the email-eprint feature is not OA! But (1) it is almost OA, (2) it will rapidly usher in OA, and (3) it is infinitely more useful to research and researchers, now -- and for the very reasons that OA is so important and needed -- than continuing to delay and agonise over legislation that will somehow manage to formally accommodate publisher copyright agreements, and lobbies, and author choice, all at once, and in advance.

However, it is not true at all that the Dual Deposit/Release "keystroke" mandate depends on the will of the publishers, nor that access is platform/technology dependent. All of OA, and indeed the definition of OA, is dependent on some technology (the Internet and the Web) and in particular, on the OAI protocol for metadata-harvesting and interoperability. Hence all Open Access Repositories are OAI-compliant. That's easy, free, and all that's needed.

And the mandate is platform-neutral too: It just requires immediate deposit of the full-text and immediate webwide visibility of the metadata. The rest will take care of itself.

As to dependence on the author's will: All authors of all 2.5 million annual articles in the world's 24,000 peer-reviewed journals want to have as many users and citers of their research as they possibly can. That is why, in paper days, they would take the time, trouble and expense to mail paper reprints to individual reprint requesters. With the DD/R mandate, this is all reduced to one author keystroke, upon receiving the semi-automated eprint-request generated by the repository software. The only thing now standing in the way of that option today is the other N-1 keystrokes needed to deposit the full-text and the metadata into the repository. And that is what the DD/R "keystroke" mandate is for!

Researchers never sought royalty revenue for their articles, and they never sought to deny access. Mandate the requisite N-1 keystrokes and they will do the Nth one happily for each individual eprint request (and to re-set access from Closed Access to Open Access when the embargo expires or they tire of doing the individual keystrokes -- whichever comes first!).

ANON: "In some cases, especially in social sciences, researchers build their papers on their research experiences from various sources, not necessarily with that of current research funding. So, can government law bind them in a strict legal sense to make their publications OA?"

No. A law based on conditions of funding (whether funding of research grants or funding of research or academic institutions) can have no power over what it does not fund. But it need not. The effects of a self-archiving mandate on funded research will propagate to non-funded research rapidly as the beneficial effects of Open Access on research accessibility, impact and progress are increasingly felt across the entire world research community.

Right now, to repeat: the only thing holding us back from the feeling of those beneficial effects, and from their propagation across disciplines and around the world, is keystrokes -- not copyright: keystrokes. Hence an immediate-keystroke mandate is all that is needed.

The (proposed) law does not tell the fundee where to publish. (It is already required that funded findings be published ["publish or perish"]). The discipline and research community dictates where to publish (in the highest quality peer-reviewed journal whose content and quality standards it successfully meets). The DD/R mandate would add only that the fundee must deposit the publication (if it is an article rather than a book) in an OAI-compliant repository (preferably his own institutional IR) immediately upon acceptance for publication.

ANON: "If the journal publisher allows... the researcher to deposit the paper in [immediate] OA it is fine; otherwise the researcher is in danger of violating the copyright law of illegally distributing, reproducing, copying, extracting copyright material over the internet."

You are not talking above merely about depositing but about OA-depositing (i.e., depositing and setting access to the full-text immediately to OA rather than CA). So if thus making the full-text freely available online would violate the copyright agreement with the publisher, the only thing the FRPAA can do, if it wishes to mandate immediate OA-deposit, is to require the fundee not to sign such an agreement (as the Wellcome Trust requires, for agreements with embargoes exceeding 6 months).

But such a requirement would in turn be open to the author/institutions objection that it constrains the author's free choice as to where to publish.

So the solution is to require deposit only (not OA-deposit), and merely to encourage OA-access setting whenever and as soon as possible.

There is no other current solution: only the endless pre-emptive debating about what the publisher would/should/could require and what the author would/should/could do -- in other words, the effective pre-emptive "embargo" on the very adoption of a self-archiving mandate that we have been stalled in now for three years! Continuing that debate simply invites more years of lobbying, delay, and accumulating access/impact loss.

What is needed is an immediate keystroke mandate, immune to debate or delay. (Then continue the debate while the keystroke mandate is having its natural effect.)

This ground has already been covered, over and over, many times before, in the past three years. There is no resolution, only inaction. It is time for action, and the Keystroke Mandate is the requisite action.

ANON: "So the solution to this problem cannot be within the technological systems of OA, [e.g., the email-eprint button of the IR software], but the OA Act would have to make exceptions for such cases. The FRPAA does not mention such exceptions - hence it needs to be rectified from this pitfall. A legal expert would be able to throw some light on it, I believe."

No, a (lawyer-dictated) inventory of provisos and exceptions will only make the mandate, confused, confusing and ineffectual: Immediate deposit of full text and metadata. No exceptions. No delays. And as access-setting is not at issue, publishers, hence author journal-choice, are both out of the loop.

Words stand in our way: The purpose of a Keystroke Mandate is of course Open Access. But call it an "Open Access Mandate" instead and you are up against the inventory of provisos and exceptions (that will in any case not add up to 100% OA). It is the tail wagging the dog. Mandate the immediate keystrokes, now, and then go back to debating the provisos and exceptions.

Nor has technology anything to do with it, other than the online medium itself, that new medium on which the very concept and possibility of OA are predicated (Open Online Access). Read the opening words of the BOAI statement: "An old tradition and a new technology have converged to make possible an unprecedented public good:

OAI-compliance is highly desirable, and easily feasible too, but need not be mentioned in any law, if you think it will produce a paralysing technology-dependence debate. Same for the trivial eprint-request button, implemented in both of the major free softwares for OAI IR creation (Eprints and Dspace)...

Just mandate the immediate keystrokes and nature will take care of all the rest...

Stevan Harnad
American Scientist Open Access Forum

---

Update Jan 1, 2010: See Gargouri, Y; C Hajjem, V Larivière, Y Gingras, L Carr,T Brody & S Harnad (2010) “Open Access, Whether Self-Selected or Mandated, Increases Citation Impact, Especially for Higher Quality Research”
Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

As Eysenbach’s very long and remarkably intemperate response to my prior response mainly repeats prior (answered) points, I will respond only to his very few substantive points:

I had asked: “Does [Eysenbach] seriously think that partialling out the variance in the number of authors would make a dent in that huge, consistent effect [the within-journal citation advantage for self-archived articles]?”

GE: “the answer is ‘absolutely’… If high-author papers are overrepresented in self-archived papers, then this confounder alone will contribute to having a greater number of citations… Only if one statistically controls for all these confounders (there are several of them - see PLoS paper), and one STILL sees an open access citation advantage, then (and only then) one has a SOLID, defendable study. ”

Here is Eysenbach’s list of confounders :

(1) number of authors: As Eysenbach says he is serious, we will now test this. We have the data. Eysenbach’s prediction is that partialling out the effect of the number of authors will make a dent in our huge, consistent citation advantage. Stay tuned...

(2) number of days since publication: This is relevant and feasible in a 1-year, 1-journal study like Eysenbach’s but neither relevant nor feasible for a sample of over a million articles ranging over 12 years, 12 disciplines, and hundreds of journals -- all showing exactly the same citation advantage for self-archived articles in every year and every discipline.

(3) article type: We are able to test this separately too (because we have ISI data on article type) but first let’s see whether partialling out author numbers makes a dent in our basic effect.

(4) country of the corresponding author: This is testable too, but first let’s see how the author-number ‘confounder’ pans out (we could look at the first-author's birth-sign too...).

(5) funding type: Data not available, and extremely far-fetched.

(6) subject area: Already tested and reported in our data, separately for 12 different disciplines : the self-archiving advantage is consistently present in all of them.

(7) submission track (PNAS has three different ways that authors can submit a paper): Not relevant to the journals we tested, which were all non-OA and pre-dated Open Choice.

(8) previous citation record of the first and last authors: This, as I noted, is -- along with the demonstration of how early the OA advantage emerges in PNAS – a potentially interesting variable in the fine-tuning of the OA advantage, but our own studies are concerned with estimating the generality and size of the OA advantage, not with its fine tuning.

(9) whether authors choosing the OA option in PNAS chose to do so for only their most important research (“they didn't”): Neither Eysenbach’s study nor ours can confirm causality or eliminate the possibility of self-selection bias.

GE: "the fact that we look at a immediate (gold-)OA article population in a longitudinal cohort study design takes care of the “arrow of causation” problem, because it makes sure that open access status comes first, then the citations are coming, not the other way round.

I'm afraid it's not quite that easy to take care of the "arrow of causation" problem, which is confounded (sic) with the problem of self-selection bias: For if authors are (contrary to their subjective reports) indeed self-selecting their better papers (or themselves!) for OA-gold (or for self-archiving) then that, and not the OA, could explain why their papers get more citations.

GE: “it is entirely possible that the articles in his sample (which he refers to as green-OA articles) were not “immediately” self-archived after publication, but 1 month, 6 months, or 12 months after original publication, therefore not really what Harnad refers to as green-OA, implying “immediate” deposition.”

This is actually a valid point of definition: OA should be defined as ‘immediate’ in order to rule out claims that delayed/embargoed access is Open Access. The point at which refereed research can and should begin to be used is when the final refereed draft is accepted for publication, and that is the point when it should be made freely accessible online. So a portion of the citation advantage for self-archived articles could well have come from self-archiving later than the publication date; technically speaking this should be called a ‘free access’ advantage, if we reserve the term "OA" for access that is free immediately. But surely nothing of substance rides on this: If there is a self-archiving advantage even for tardy self-archiving, that confirms, a fortiori, the self-archiving advantage of prompt (OA) self archiving too!

GE: “I… made a conscientious decision to submit my paper to a gold-OA journal (PLoS) rather than publishing the study in an obscure scientrometrics journal and then self-archived [sic] it”

Actually, unless I am mistaken, I seem to recall corresondence from GE to the effect that it was first declined by Science (or was it Nature?) – not a gold-OA journal – before being submitted to PloS Biology)…

GE: “The visibility of an article published in a properly promoted OA journal site will always be better than a paper that is published in a toll-access journal site, even if it is self-archived. This is exactly why my study shows an advantage of gold-OA over green-OA, this is also why I personally chose the gold route to publish this paper in PLoS, and not the green route”

Let us not confound a journal's profile/impact level with its OA/non-OA status.

The visibility (and no doubt also the citation impact) of an article will always be better when it is published in a high-profile, high-impact journal, whether it is OA (like PLoS) or non-OA (like Science or Nature) rather than an obscure scientometrics journal (or an obscure OA journal). Its visibility and impact will be higher if self-archived in either case (except perhaps if the journal is both high-profile and optional-OA, which is partly what Eysenbach’s study has shown).

GE: “the PLoS paper is the first study which contains an analysis of both gold and green (thus focuses on “OA itself”), whereas the rest of the studies is actually focused on ‘green’”.

Because most of the existing data for within-journal OA/non-OA comparisons comes from the millions of articles published in the thousands of non-gold journals indexed by ISI and not just the thousands of articles published in the few journals that are as yet (like PNAS) optional-gold...

Stevan Harnad
American Scientist Open Access Forum

---

Update Jan 1, 2010: See Gargouri, Y; C Hajjem, V Larivière, Y Gingras, L Carr,T Brody & S Harnad (2010) “Open Access, Whether Self-Selected or Mandated, Increases Citation Impact, Especially for Higher Quality Research”
Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

PLoS seems to have concluded that it is not in their interest to host further public contributions to the Eysenbach debate from me -- and perhaps they are right (that it is not in their interest)...

From: plos AT plos.org
Date: May 19, 2006 3:13:07 PM EDT (CA)
To: Stevan Harnad harnad AT ecs.soton.ac.uk
Subject: e-Letter: Decision

Unfortunately we decided not to accept your e-Letter. Letters are published at the editors' discretion, and we publish only those that we believe will contribute substantially to the debate. Our editorial decisions about publishing letters are final, and are not open to appeal.

Below is appended the mercilessly compressed fragment that I had submitted to PLoS as a follow-up letter (responding to Eysenbach's PLoS letter responding to my PLoS letter responding to his PLoS article).

The full version of my reply of course appeared on AmSci and in my Archivangelism blog -- but, as Eysenbach's study showed, one gets still further visibility from appearing on the website of a high-profile, high-impact journal! My (valid) rebuttal to Eysenbach's suggestion that self-archiving is to OA publishing as handing out leaflets is to publishing in a newspaper was that we are talking about publications in the case of OA, not unpublished materials! But with a letter, it's more like handing out leaflets when it just appears in a Forum or a Blog, versus the website of a high-impact journal...

Never mind. The content is what matters, and time is on OA's side (even if it is much too dilatory!), because OA is (you've heard the song!): Optimal and Inevitable.

---

Confirming the Within-Journal OA Impact Advantage

(Click for Fuller version)

Given the large within-journal OA citation impact advantages repeatedly found across all journals, disciplines and years in samples four orders of magnitude larger than Eysenbach's, it is not clear that controls for "multiple confounders" are needed to demonstrate the reality, magnitude and universality of the OA advantage. (This does not mean Eysenbach’s controls are not useful, just that they are not yet telling us much that we don't already know.)

Eysenbach (and PLoS) are focussed on gold-OA journals; most other OA impact studies are focused on OA itself. Only ~10% of journals are gold today. Few as yet offer authors "Open Choice" (allowing gold within-journal OA/NOA comparisons) and few authors are as yet choosing paid OA.

Regarding the “arrow of causation: yes, “longtitudinal cohort” data would demonstrate causation (for skeptics who think the OA advantage might be a self-selection bias) but Eysenbach's author self-reports certainly aren’t such data! Meanwhile: (a) the OA advantage does not diminish for younger articles; (b) OA increases downloads; (c) increased downloads in the first 6 months correlate with increased citations later; (d) unaffordability reduces access; (e) access is a necessary condition for citation.

About OA being a “continuum” or “spectrum”: Time is certainly a continuum, and access certainly admits of degrees (access may be easier/harder, narrower/wider, cheaper/dearer, longer/shorter, earlier/later, partial/full) -- but Open Access does not admit of degrees (any more than pregnancy does). OA is defined as: full-text online access, free for all.

Eysenbach likens self-archiving to “printing something on a flyer and handing it out to pedestrians on the street [instead of] publishing an article in a national newspaper." But it is published articles that are being self-archived.

NOA (Not OA): 1159 articles (86.2% cited at least once)
POA (Payed OA only): 176 (94.3%)
SOA (Self-Archived OA only): 121 (90.1%)
BOA (POA and SOA): 36 (97.2%)

In this PNAS sample, POA, SOA and BOA together, and POA alone, all have significantly more citations than NOA, but SOA alone ("stratified") does not; also, both POA and SOA increase citations, but POA does it more.

Three possible hypotheses explaining the BOA>POA>>SOA>NOA outcome:

H1: The POA advantage might be a multiple-archiving effect, maximal for high-profile , 3-option (POA, SOA, NOA) journals like PNAS because POA articles are more visible than SOA. (POA + SOA = BOA highest of all: redundancy helps!) As Institutional Repositories fill, this extra advantage will disappear.

H2: The POA advantage might arise in part from self-selection because the decision to pay for POA is influenced by the author's sense of the potential importance (hence impact) of his article. (But I think self-selection quality-bias is just one of many contributors to the OA advantage itself, not the only one or the biggest.)

H3: The POA advantage might be either a small-sample chance result or a temporary side-effect of the 3-option journals in early days: a one-stop shopping advantage for PNAS articles, in a high-profile store, today. It needs to be tested for replicability and representativeness in larger samples of articles, journals, and time-bases.

The true measure of the SOA advantage today is not found in PNAS but in the far more populous and representative full spectrum of journals not yet offering POA. (I’d be delighted if those journals took Eysenbach’s findings as a reason for offering a POA option! But not at the expense of authors wrongly inferring that for the journals they currently publish in, SOA alone would not confer citation advantages at least as big as the ones we have been reporting.)

Harnad & Brody (2004)
Brody et al (2005)
Hajjem et al (2005)

Stevan Harnad
American Scientist Open Access Forum

---

Update Jan 1, 2010: See Gargouri, Y; C Hajjem, V Larivière, Y Gingras, L Carr,T Brody & S Harnad (2010) “Open Access, Whether Self-Selected or Mandated, Increases Citation Impact, Especially for Higher Quality Research”
Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

Gunther Eysenbach (GE) (in a letter in letter PLoS Biology today) wrote:

GE: "The introduction of the article and two accompanying editorials [1, 2, 3] already answer Harnad's questions why author, editors, and reviewers were critical of the methodology employed in previous studies, which all only looked at "green OA" (self-archived/online-accessible papers)"

I didn't ask why the author and editors were critical of prior self-archiving (green OA) studies; I asked why they said such studies were "surprisingly hard to find" and why the two biggest and latest of them were not even taken into account:

Brody, T., Harnad, S. and Carr, L. (2005) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 56.

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47.

And the reason all prior within-journal studies look only at "green OA" is that the majority of OA today is green; hence almost all OA/NOA impact comparisons are based on green OA (self-archiving) rather than on paid-OA (gold). To compare OA and NOA between rather than within journals would be to compare apples and oranges: See critique of ISI's between-journal OA/NOA comparisons in:

Brody, T. and Harnad, S. (2004) Comparing the Impact of Open Access (OA) vs. Non-OA Articles in the Same Journals. D-Lib Magazine 10(6).

GE: " (hint 1: "confounding") (hint 2: arrow of causation: are papers online because they are highly cited, or the other way round?)."

I am afraid I don't see Eysenbach's point here at all: What exactly does he think is being confounded in within-journal comparisons of self-archived versus non-self-archived articles? The paid-OA effect? But among OA articles today there is almost zero within-journal paid-OA, because so few journals offer it! (Hajjem et al.'s within-journal comparisons were based on over a million articles, across 12 years and hundreds of journals, in 12 disciplines! Eysenbach's were based on 1492 articles, in 6 months, in one journal.)

And is Eysenbach suggesting that his failure to find any significant difference among author self-reports -- about their own article's quality and its causal role in their decision about whether or not to pay for OA (or to self-archive) in his sample of 237 authors -- is an objective test of the arrow of causation? (I agree that Eysenbach's failure to find a difference fails to support the hypothesis of a self-selection bias, but surely that won't convince those who are minded to hold that hypothesis! I would welcome rigorous causal evidence against the self-selection hypothesis as much as Eysenbach would, but author self-reports are alas not that evidence!)

GE: " The statement in the PLoS editorial has to be seen against this background. None of the previous papers in the bibliography mentioned by Harnad employed a similar methodology, working with data from a "gold-OA" journal."

Yes, almost all prior studies on the OA impact advantage are based on green OA, not gold, but so what? It is Eysenbach (and PLoS) who are focussed on gold-OA journals; the rest of the studies are focussed on OA itself. Only about 10% of the planet's peer-reviewed journals are gold today, and most of those are 100% gold, hence allow no within-journal comparisons. Very few journals as yet offer authors the "Open Choice" (optional paid gold) that would allow gold within-journal OA/NOA comparisons; and few authors are as yet taking those journals up on it (about 15% in this PNAS sample), compared to the far larger number that are self-archiving (also 15%, as it happens, though that percentage too is still far too small!). The difference in article sample sizes is about four orders of magnitude (c. 1500 articles in Eysenbach's study to 1.5 million in Hajjem et al's).

GE: " The correct method to control for problem 1 (multiple confounders) is multivariate regression analysis, not used in previous studies."

Correct. But with the large, consistent within-journal OA/NOA differences found across al journals, all disciplines and all years in samples four orders of magnitude larger than Eysenbach's, it is not at all clear that controls for those "multiple confounders" are necessary in order to demonstrate the reality, magnitude and universality of the OA advantage. That does not mean the controls are not useful, just that they are not yet telling us much that we don't already know.

GE: " Harnad's statement that "many [of the confounding variables] are peculiar to this particular... study" suggests that he might still not fully appreciate the issue of confounding. Does he suggest that in his samples there are no differences in these variables (for example, number of authors) between the groups? Did he even test for these? If he did, why was this not described in these previous studies?"

No, we did not test for "confounding effects" of number of authors: What confounding effects does Eysenbach expect from controlling for number of authors in a sample of over a million articles across a dozen disciplines and a dozen years all showing the very same, sizeable OA advantage? Does he seriously think that partialling out the variance in the number of authors would make a dent in that huge, consistent effect?

Not that Eysenbach's tentative findings on 1st-author/last-author differences in his one-journal sample of 1492 are not interesting; but those are merely minor differences in shading, compared to the whopping main effect, which is: substantially more citations (and downloads) for self-archived OA articles.

GE: " The correct method to address problem 2 (the "arrow of causation" problem) is to do a longitudinal (cohort) study, as opposed to a cross-sectional study. This ascertains that OA comes first and THEN the paper is cited highly, while previous cross-sectional studies in the area of "green OA" publishing (self-archiving) leave open what comes first -- impact or being online."

I agree completely that time-based studies are necessary to demonstrate causation, for those who think that the OA advantage might be based on self-selection bias (i.e., that high-impact studies tend to be preferentially self-archived, perhaps even after they have gained their high impact), but Eysenbach's author self-report data certainly don't constitute such a longtitudinal cohort study! (Once there exist reliable deposit dates for self-archived articles, we will be able to do some time-based analyses on green OA too, but, frankly, by that time the outcome is likely to be a foregone conclusion.)

In the meanwhile, the fact that (a) the OA advantage does not diminish for younger articles (as one would expect if it were a post-hoc effect), that (b) OA increases downloads, and that (c) increased downloads in the first 6 months are correlated with increased citations later on -- plus the logic of the fact that (d) unaffordability reduces access and that (e) access is a necessary condition for citation -- all suggest that most of the scepticism about the SOA advantage is because of conflicting interests, not because of objective uncertainty.

GE: " Harnad - who usually carefully distinguishes between "green" and "gold" OA publishing -- ignores that open access is a continuum, much as publishing is a continuum"

I'm afraid I have no idea what Eysenbach means about OA being a continuum: Time is certainly a continuum, and access certainly admits of degrees (access may be easier/harder, narrower/wider, cheaper/dearer, longer/shorter, earlier/later, partial/full) -- but Open Access does not admit of degrees (any more than pregnancy does). OA means immediate, permanent, full-text online access, free for all, now.

And, by the way, green OA is certainly not a lesser degree of gold OA!

For the innocent reader, puzzled as to why this would even be an issue:

Please recall that OA (gold) journals, whether total or optional gold, need authors (and those gold journals with the gold cost-recovery model need paying author/institutions). To attract authors, gold journals need to persuade them of the benefits of OA. So far so good. But there is another thing they have to persuade them of, implicitly or explicitly, and that is the benefits of gold OA over green OA. For if there are no benefits of gold over green, then surely it makes much more sense for authors to publish in their journal of choice, as they always did, and simply self-archive their own articles, rather than switching journals and/or paying for gold OA!

This theme alas keeps recurring, implicitly or explicitly, in the internecine green/gold squabbles, because green OA is indeed a rival to gold OA in gold OA journals' efforts to win over authors. This is regrettable, but a functional fact today, owing to the nature of OA and of the two means of providing it.

Is the effect symmetrical? Is gold OA likewise a rival to green OA? Here the answer is more complicated: No, an author who chooses gold OA (by publishing in an OA journal) is not at all a loss for green OA, because the article is nevertheless OA, and green OA's sole objective is 100% OA, as soon as possible, and nothing else. (Besides, a gold OA article too can be self-archived in the author's Institutional Repository if the author or institution wishes! All gold journals are, a fortiori, also green, in that they endorse author self-archiving.)

But there is a potential problem with gold from the standpoint of green. The problem is not with authors choosing gold. The problem is with gold publishers promoting gold as superior to green, or, worse, with gold publishers implying that green OA is not really OA, or not "fully" OA (along some imaginary OA "continuum").

"Free Access vs. Open Access" (thread started Aug 2003)

Why, you ask, would gold OA want to give the impression that green OA was not "really" OA or not "fully" OA? Because of the rivalry for authors that I just mentioned. The causal arrow is a one-way one insofar as competition for authors is concerned: green OA does not lose an author if that author publishes in a gold OA journal, whereas gold OA does lose an author if an author publishes in a green journal instead of a gold one. However, if gold portrays green as if it were not really or fully OA, and authors believe this, then it loses author momentum for green -- especially among that vast majority of authors who do not yet elect to publish gold. For there is today something still very paradoxical, indeed equivocal, about author behavior and motivation vis-a-vis OA:

Authors profess to want OA. Thirty-four thousand of them even signed the 2001 PLoS Open Letter threatening to boycott their journals if they did not provide (gold) OA (within 6 months of publication). (Most journals did not comply, and most authors did not follow through on their boycott threat: How could they? There were not enough suitable gold journals for them to switch to, and most authors clearly were not interested in switching journals, let alone paying for publication, then or now.)

Yet (and here comes the paradox): if those 34,000 signatories -- allegedly so desirous of OA as to be ready to boycott their journals if they did not provide it -- had simply gone on to self-archive all their papers, they would be well on the road to having the OA they allegedly desired so much! For the green road to 100% OA happens to be based on the (golden!) rule: Self-Archive Unto Others As You Would Have Them Self-Archive Unto You.

Why didn't (and don't) most authors do it (yet)? It is partly (let us state it quite frankly) straightforward foolishness and inconsistency on authors' part. They simply have not thought it through. This cannot be denied. Authors are in a state of self-induced "Zeno's Paralysis" regarding OA, from which FAQs have so far been powerless to free them -- so that it now looks as if self-archiving mandates from their institutions and/or their funders will be the only thing that can induce them to do what will give them what they so want and need.

But the confusion and inaction are partly also the fault of the promotional efforts of (well-meaning) OA advocates. Harold Varmus sent a mixed message with his 1999 "E-biomed" proposal (which led to PLoS, the PLoS Open letter, PubMed Central, Biomed Central, and eventually the PLoS and BMC fleet of OA journals, including PLoS Biology). Was E-biomed a gold proposal, a green proposal, both, or neither? The fact is that it was an incoherent proposal -- a confused and confusing mish-mash of central self-archiving, publishing reform/replacement and rival publishing -- and although it has undeniably led to genuine and valuable progress toward (what was eventually baptized by BOAI as) OA, it has left a continuing legacy of continuing confusion too.

And we are facing part of that legacy of confusion now, with PLoS thinking that the only way (or the best) to reach 100% OA is to publish and promote gold OA journals. That is why PLoS Biology agreed to referee the Eysenbach paper, which seemed to show that OA gold is the only one that increases citation impact, not green self-archiving, which is (when you come right down to it) not even "real" OA at all!

That is also why PLoS Biology editorialised that they found it "surprisingly hard to find" evidence -- "solid evidence" -- that OA articles are read and cited more. And that is why PLoS Biology was happy to make an exception and publish the Eysenbach study, even though scientometrics is not the subject matter of PLoS Biology, but (I'll warrant) PLoS Biology would not have been happy to advertise in its pages the fact that green OA self-archiving was enough to get articles read and cited more!

So green OA does have a bit of an uphill battle against gold OA and the subsidies and support it has received (because gold OA is an attractive and understandable idea, whereas green OA requires a few more mental steps to dope out -- though not many, as none of this is rocket science!).

But, to switch metaphors, the green road to 100% OA (sic) is far wider, faster and surer than the golden road. (Every article can be self-archived, today, and without their authors' having to renounce or switch journals, whereas most articles do not yet have a suitable OA journal to publish in today, even if their authors wished to switch journals, which most do not; and authors can be mandated to self-archive by their institutions and funders, but neither authors' choice of journals nor their publishers' choice of access-provision or cost-recovery model can be mandated by authors' institutions and funders.) Moreover, 100% OA really is beneficial to research and researchers; so the green road of self-archiving is bound to prevail, despite the extra obstacles. And the destination (100% OA) is exactly the same for both roads. (Indeed, I am pretty sure that even the fastest way to reach 100% gold OA -- i.e., not just 100% OA but also the conversion of all journals to gold -- is in fact to take the green road to 100% OA first.

So gold is doing itself a disservice when it tries to devalue green. Read on:

GE: " and this study (and the priority claims in the editorial) was talking about the gold OA end of the spectrum."

Spectrum? Continuum? Degrees of OA?

GE: " Publishing in an open access journal is a fundamentally different process from putting a paper published in a toll-access journal on the Internet. In analogy, printing something on a flyer and handing it out to pedestrians on the street, and publishing an article in a national newspaper can both be called "publishing", but they remain fundamentally different processes, with differences in impact, reach, etc. A study looking at the impact of publishing a newspaper can not be replaced with a study looking at the impact of handing out a flyer to pedestrians, even though both are about "publishing"."

Oh dear! I have a feeling Eysenbach is going to tell as that making a published journal article accessible online free for all by self-archiving it is not OA after all, or not "full OA". If the journal doesn't do it for you, and/or you don't pay for it, it's not the real thing.

I wonder why Eysenbach would want to say that? Could it be because he is promoting an OA (gold) journal (his own)? Could that also have been the reason the PLoS editorial was so sanguine about Eysenbach's findings on the OA gold advantage, and so dismissive of any prior evidence of an OA green advantage?

GE: " Finally, Harnad says that "prior evidence derived from substantially larger and broader-based samples showing substantially the same outcome". I rebut with two points here[:] Regarding "larger samples" I think rigor and quality (leading to internal validity) is more important than quantity (or sample size)."

Even when all within-journal studies -- large and small, approximate and exact -- just keep producing exactly the same outcome, every time (OA increases impact)?

GE: " Going through the laborious effort to extract article and author characteristics for a limited number of articles (n = 1492) in order to control for these confounders provides scientifically stronger evidence than doing a crude, unadjusted analysis of a huge number of online accessible vs non-online accessible articles, leaving open many alternative explanations."

As I said, for those who doubt the causality and think the OA advantage is just a self-selection bias, Eysenbach's study will not convince them otherwise either. For those with eyes to see, the repeated demonstrations, in field after field, of exactly the same effect on incomparably larger samples will already have been demonstration enough. For those with eyes only for gold, evidence that green enhances citations will never be "solid evidence."

If Eysenbach and the editors had portrayed the latest PLoS findings as they should have, namely, as yet another confirmation of the OA impact advantage, with some new details about its fine-tuning, I would have done had nothing but praise for it. But the actual self-interested spin and puffery that instead accompanied this work -- propagating the frankly false idea that this is the first "solid evidence" for the OA impact advantage, and, worse, that it implies that self-archiving itself does not deliver the OA impact advantage -- would have required not the lack of an ego, but the lack of any real fealty to OA itself to have been allowed to stand uncontested.

GE: " Secondly, contrary to what Harnad said, this study is NOT at all "showing substantially the same outcome". On the contrary, the effect of green-OA -- once controlled for confounders - was much less than what others have claimed in previous papers."

Let's be quite explicit about what, exactly, we are discussing here:

Eysenbach found that in a 6-month sample of 1492 articles in one 3-option journal (PNAS):

"While in the crude analysis self-archived papers had on average significantly more citations than non-self-archived papers (mean, 5.46 versus 4.66; Wilcoxon Z = 2.417; p = 0.02), these differences disappeared when stratified for journal OA status (p= 0.10 in the group of articles published originally as non-OA articles, and p = 0.25 in the group of articles published originally as OA).

"In a logistic regression model with backward elimination, which included original OA status and self-archiving OA status as separate independent variables as well as all potential confounders, self-archiving OA status did not remain a significant predictor for being cited. In a linear regression model, the influence of the covariate "article published originally as OA, without being self-archived" (beta = 0.250, p < 0.001) on citations remained stronger than self-archiving status (beta = 0.152, p = 0.02)."

To translate this into english (from an article with exceedingly user-unfriendly data-displays, by the way, making it next to impossible to extract and visualize results from the tables by inspection!): First, the numbers:

NOA (Not OA): (1159 articles 86.2% cited at least once)
POA (Payed OA only): (176 articles 94.3% cited at least once)
SOA (Self-Archived OA only): (121 articles 90.1% cited at least once)
BOA (POA and SOA): ( 36 articles 97.2% cited at least once)

The finding is that (in this PNAS sample, and with many other factors -- e.g., days since publication, number of authors, article type, country, funding, subject, etc. -- statistically isolated so as to be asessable independently): POA, SOA and BOA considered together, and PAO considered alone, all have significantly more citations than NOA; but SOA considered alone ("stratified") does not. Also, if considered jointly (multiple regression), both POA and SOA increase citations, but POA is the stronger effect.

Here are three simple hypotheses, in decreasing order of likelihood, as to why this small PNAS study may have found that the citation counts and their significance ordered themselves as they did: BOA>POA>>SOA>NOA

Hypothesis 1: The POA advantage might be unique to high-profile 3-option journals (POA, SOA, NOA) like PNAS (which are themselves a tiny minority among journals) and occurs because the POA articles are more visible than the SOA articles. (The POA + SOA = BOA articles do the best of all: redundancy enhances visibility.) So the POA authors do get something more for their money (but that something is not OA but high-profile POA in a high-profile journal) -- at least for the time being. This extra POA-over-SOA advantage will of course wash out as SOA and indexed, interoperable Institutional Repositories for self-archiving grow.

Hypothesis 2: The POA advantage might result at least in part from QB (self-selection Quality Bias) because the decision (by a self-selected 15% subset of PNAS authors) to pay for POA is influenced by the author's underlying sense of the potential importance (hence impact) of his article: Simply asking authors about how important they think their article is, and whether that influenced their decision to pick POA or SOA or NOA, and failing to detect any significant difference among the authors, does not settle this matter, and certainly not on the basis of such a small and special sample. (But I think QB is just one of many contributors to the OA citation advantage itself, and certainly not the only determinant or even the biggest one.)

Hypothesis 3: The POA advantage might be either a small-sample chance result or a temporary side-effect of the 3-option journals in early days: a one-stop shopping advantage for PNAS articles, in a high-profile store, today. It needs to be tested for replicability and representativeness in larger samples of articles, journals, and time-bases.

(Note that Lawrence's 2001 as well as Hajjem et al's 2005 finding had been that the proportion of OA articles increases in the higher citation ranges, being lowest among articles with 0-1 citations.)

Eysenbach finds that with logistic regression analysis separating the independent effects of POA, SOA and other correlates, SOA has no significant independent effect in his 1-year PNAS sample. Now let's test whether that replicates in larger samples, both in terms of number of articles, journals, and time-base. (Failure to find a significant effect in a small sample is far less compelling than success in finding a significant effect in a small sample!)

GE: " Harnad, a self-confessed "archivangalist", co-creator of a self-archiving platform, and an outspoken advocate of self-archiving (speaking of vested interests) calls the finding that self-archived articles are... cited less often than [gold] OA articles from the same journal "controversial". In my mind, the finding that the impact of nonOA < greenOA < goldOA < green+goldOA is intuitive and logical: The level of citations correlates with the level of openness and accessibility."

I don't dispute that POA can add more citations, just as BOA can; maybe self-archiving in 10 different places will add still more. But what does this imply, right now, practically speaking? And, even more important, how likely is it that this sort of redundancy will continue to confer significant citation advantages once a critical mass of the literature is in interoperable Institutional Repositories (green SOA) rather than few and far between, as now? It is indeed intuitive and logical that the baseline 15% of the literature as a whole that is being spontaneously self-archived somewhere, somehow on the Web, across all fields, has somewhat less visibility right now than the 15% of PNAS articles that PNAS is making OA for those authors who pay for it (POA). That's a one-stop shopping advantage for PNAS articles, against PNAS articles, in a high-profile store, today.

But the true measure of the SOA advantage today (at its 15% spontaneous baseline) is surely not to be found in PNAS but in the statistically far more numerous, hence far more representative full-spectrum of journals that do not yet offer POA. (I would be delighted if those journals took the Eysenbach findings as a reason for offering a POA option! But not at the expense of authors drawing the absurd conclusion -- not at all entailed by Eysenbach's PNAS-specific results -- that in the journals they currently publish in, SOA alone would not confer citation advantages at least as big as the ones we have been reporting.)

Regarding my self-confessed sin of archivanglizing, however, I do protest that my first and only allegiance is to 100% OA, and I evangelize the green road (and promote the self-archiving software) only because it is so resoundingly obvious that it is the fastest and surest road to 100% OA. (If empirical -- or logical -- evidence were ever to come out showing the contrary, I assure you I too would join the gold rush!)

GE: " Sometimes our egos stand in the way of reaching a larger common goal, and I hope Harnad and other sceptics respond with good science rather than with polemics and politics to these findings."

Well, first, let us not get carried away: There's precious little science involved here (apart from the science we are trying to provide Open Access to). The call to self-archive in order to enhance access and impact is so obvious and trivial that, as I noted, the puzzle is only why anyone would even have imagined otherwise.

But when it comes to polemics and politics (and possibly also egos), it might have kept things more objective if the results of Eysenbach's small but welcome study confirming the OA impact advantage had not been hyped with editorial salvos such as:

"solid evidence to support or refute... that papers freely available in a journal will be more often read and cited than those behind a subscription barrier... has been surprisingly hard to find..."

Or even the heavily-hedged:

"As far as we are aware, no other study has compared OA and non-OA articles from the same journal and controlled for so many potentially confounding factors."

GE: " Unfortunately, in this area a lot more people have strong opinions and beliefs than those having the skills, time, and willingness to do rigorous research. I hope we will change this, and I reiterate a "call for papers" in that area [http://www.jmir.org/2006/2/e8/]"

May I echo that call, adding only that the rigorous research might perhaps be better placed in a journal specializing in scientometrics and in rigorously peer-reviewing it, rather than in The Journal of Medical Internet Research, or even PLoS Biology.

Brody, T., Harnad, S. and Carr, L. (2005) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 56.

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47.

I close with some replies to portions of another version of Eysenbach's response which appeared in his blog.

GE: " Harnad's point that the PLoS paper is about the "citation advantage of open access" and that there have been "previous papers about the citation advantage of open access" (mostly his own studies, mostly not published in peer-reviewed journals) is as meaningful as saying "this paper is about a cancer treatment, and there are previous papers about cancer treatments, so this one doesn't add anything"."

That's not what I said. I said this:

"[T]he only new knowledge from this small, journal-specific sample was (1) the welcome finding of how early the OA advantage can manifest itself, plus (2) some less clear findings about differences between first- and last-author OA practices, plus (3) a controversial finding that will most definitely need to be replicated on far larger samples in order to be credible: "The analysis revealed that self-archived articles are also cited less often than OA [sic] articles from the same journal."

And I do think all of this is as far away from rigorous oncological research as it is from rocket science!

GE: " The statement made by the reviewers and editors of the PLoS paper that this is the first study looking at the citation advantage of an open access/hybrid journal remains correct until somebody can show me a reference where this has been done before."

But who ever contested that far more modest and circumspect statement (which was certainly not the one the accompanying PLoS editorial made)? This is indeed "the first study looking at the citation advantage of an open access/hybrid journal"; indeed, it's the first such study of PNAS. But it's certainly not the first study looking at the citation advantage of OA in general, or OA self-archiving in particular, and looking at it within journals -- within many journals, and many articles.

GE: " In analogy, a small carefully designed cohort study showing a relationship between smoking and cancer with 1500 patients, obtaining through questionnaires and interviews additional variables which could account for the association and controlling for these confounders and still coming to the conclusion that there is a relation between smoking and cancer is scientifically stronger evidence than a quick-and-dirty uncontrolled cross-sectional study showing an association between smoking and cancer, even if this is done in a population of millions."

Indeed it would. And I forgot to add to my list (4) that Eysenbach had tested the hypothesis that the OA citation advantage is merely the result of a self-selection bias by asking 247 authors whether it was, and they replied that it wasn't...

Stevan Harnad
American Scientist Open Access Forum

---

Update Jan 1, 2010: See Gargouri, Y; C Hajjem, V Larivière, Y Gingras, L Carr,T Brody & S Harnad (2010) “Open Access, Whether Self-Selected or Mandated, Increases Citation Impact, Especially for Higher Quality Research”
Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

PLoS, Pipe-Dreams and Peccadillos

(Shorter version of this comment appears as a letter in PLoS Biology)

ABSTRACT: Eysenbach's (2006) study in PloS Biology on 1492 articles published during one 6-month period in one journal (PNAS) found that the Open Access (OA) articles were more cited than the non-OA ones. The online bibliography on the OA citation advantage records a number of prior within-journal comparisons that found exactly the same effect: freely available articles are read and cited more. Eysenbach?s further finding that the OA advantage (in this particular 6-month, 3-option, 1-journal PloS/PNAS study) is greater for articles that have paid for OA publication than for those that have merely been self-archived will require replication on much larger samples as most of the prior evidence for the OA advantage comes from self-archived articles and is based on sample sizes four orders of magnitude larger for both the number of articles and the number of journals tested.

I applaud and welcome the results of the Eysenbach (2006) study on 1492 articles published during one 6-month period in one journal (PNAS), showing that the Open Access (OA) articles were more cited than the non-OA ones. I also agree fully that the findings are unlikely to have been an artifact of PLoS’s “strong and vested interest in publishing results that so obviously endorse our existence,” nor of the fact that “the author of the article is also an editor of an open-access journal” (all quotes are from the PloS Biology editorial by MacCallum & Parthasarthy, 2006).

However, I am less sure that PloS’s and the author’s vested interests are not behind statements (in both the accompanying editorial and the article itself) along the lines that: “solid evidence to support or refute… that papers freely available in a journal will be more often read and cited than those behind a subscription barrier… has been surprisingly hard to find.” The online bibliography on ‘The effect of open access and downloads ('hits') on citation impact’ records a growing number of studies reporting precisely such evidence as of 2001, including studies based on data from much larger samples of journals, disciplines and years than the PloS study on PNAS– and they all find exactly the same effect: freely available articles are read and cited more.

There can be disagreement about what evidence one counts as “solid,” but there can be little dispute that prior evidence derived from substantially larger and broader-based samples showing substantially the same outcome can hardly be described as “surprisingly hard to find.”

In fact, the only new knowledge from this small, journal-specific sample was (1) the welcome finding of how early the OA advantage can manifest itself, plus (2) some less clear findings about differences between first- and last-author OA practices, plus (3) a controversial finding that will most definitely need to be replicated on far larger samples in order to be credible: “The analysis revealed that self-archived articles are also cited less often than OA [sic] articles from the same journal.”

The latter (3) is a within-journal (one journal, PNAS) finding; the overwhelming majority of articles made OA (sic) through author self-archiving today (on which the prior large-sample OA citation advantage findings are based) do not appear in journals with a paid-OA option. Hence on the present evidence I have great difficulty in seeing this secondary advantage as any more than a paid-OA publisher’s pipe-dream at this point.

The following, however, is not a pipe-dream, but a peccadillo: “no other study has compared OA and non-OA articles from the same journal.” To be fair, this observation is hedged with “[a]s far as we are aware” (but the OA-advantage bibliography is surely public knowledge – or should be among advocates of public access to science) and the observation is further qualified with: “and [also] controlled for so many potentially confounding factors.”

But it has to be stated that of these “potentially confounding” variables -- “number of days since publication, number of authors, article type, country of the corresponding author, funding type, subject area, submission track (PNAS has three different ways that authors can submit a paper)… previous citation record of the first and last authors… [and] whether authors choosing the OA option in PNAS chose to do so for only their most important research (they didn't)” – many are peculiar to this particular short-interval, 3-option, single-journal PloS study. And several of them (country, subject, year) had already been analyzed in papers that had been published before this 2006 article and were not taken into account despite the fact that both their preprints and their postprints had been freely accessible since well before publication, and that at least one of them (Brody et al. 2005) had been explicitly drawn to the author’s attention based on a preprint draft well before the article was submitted to PloS.

Brody et al. (2005) had found that, alongside the OA citation advantage, more downloads in the first six months after publication are correlated with more citations 18 months later in physics; and Hajjem et al. (2005) had found higher citations for OA articles -- comparing always within the very same journal and year -- for 1,307,038 articles published across 12 years (1992-2003) in 10 disciplines (Biology, Psychology, Sociology, Health, Political Science, Economics, Education, Law, Business, Management).

REFERENCES

Brody, T., Harnad, S. and Carr, L. (2005) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 56.

Eysenbach, G, (2006) Citation Advantage of Open Access Articles. PLoS Biology 4(5).

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47.

MacCallum, C.J., and Parthasarathy, H. (2006) Open Access Increases Citation Rate. PLoS Biology 4(5).


Stevan Harnad
American Scientist Open Access Forum

As presently drafted, the wording of the the timely and extremely welcome US Federal Research Public Access Act (FRPAA) stands to create needless problems for itself that could even make it fail under the already-gathering opposition from the publisher lobby.

Yet the FRPAA's flaws are ever so easily correctable:

The gist of the problem is all there in this well-meaning quote by Senator John Cornyn (R, co-sponsor of the bill (with Senator Joe Lieberman, D: quotation is from Robin Peek's Newsbreak article in Information Today).

JC: "Making this information available to the public will lead
to faster discoveries, innovations and cures..."

This same logic underlies the Bill itself.

The publisher lobby will (quite rightly) jump straight onto the two profound errors in this reasoning, and they will use it, for all its worth, against the Bill:

(1) For most of the research literature, the public has neither the expertise nor the interest to read it.

(2) Making it accessible to the public, does not make for cures!

Yet the remedy is so absurdly simple: The pressing reason for making research accessible to everyone is not because the general public has a pressing interest in reading it, nor because the public's reading it will result in cures. It is so that researchers -- those specialists by and for whom it was written, the ones with the expertise to use, apply and build upon it -- can access and apply it, to the benefit of the general public who paid for it.

In Senator Cornyn's quote, the following sentence comes second, and too late, already undone by the first statement (and the logic is similarly backwards in the Bill itself):

JC: "This bill will give the American taxpayer a greater return on its research investment."

The right way to put it is:

"Making this information available to all researchers who can use, apply and build upon it will lead to faster discoveries, innovations and cures, thereby giving American taxpayers a greater return on their research investment. As a side bonus, the tax-paying public too will have access to as much as they may feel they wish to read of the research they have funded."

Today, most of published research is not accessible to much of its potential research-user population because no researcher's institution can afford paid access to more than a fraction of it. That's the real public rationale for mandating self-archiving: so that the tax-paying public that funds the research can benefit from the "discoveries, innovations and cures" that will arise from making research findings accessible to all researchers who can use and apply them.

UK: " Maximising the Return on the UK's Public Investment in Research"

Canada: "Making the case for web-based self-archiving"

Europe: "Publish or Perish -- Self-Archive to Flourish"

Australia: "Australia Is Not Maximising the Return on its Research Investment"

Of course, the general public can and will be able to read whatever they wish of research too, if it is made OA. But it is foolish in the extreme to base the case for making research OA on the putative pressing need for the public to read it, and the putative "discoveries, innovations and cures" that the reading public will provide as a result!

Why would the FRPAA make such a silly strategic error? Because, superficially, the right of the tax-paying public to access the research that they have paid for looks like a spinnable "public good" issue as well as a spinnable "public right-to-know" issue. In that (flawed) form, it looks like viable political-campaign material.

But what makes it look like such a compelling naive-voter issue is also its fatal weakness, once the publisher lobby -- which is not at all naive -- attacks it: because the two points I have made above are dead-obvious, and can stop the momentum of OA dead in its tracks if the FRPAA has no stronger rationale to back it up with. Here is what I would immediately say if I were in the publisher lobby (and believe me, the publishers are already busy saying it):

"The government wants to put the revenues of a viable industry at possible risk simply because it thinks the general public has a burning need to read mostly-technical texts written for a small population of specialists. (Here we have some public-library data on the infinitesimal rate at which the general public actually consults this kind of specialized material when it is made freely available to them: Is this what all the fuss is about? Because if it's instead just about access to the kind of clinical-health-related material that we do have evidence the public wants to consult, we can easily work out a side-deal instead of the FRPAA that leaves most of the research literature in closed access, as it is now, with some exceptions for articles of potential clinical relevance and hence public interest).

"And why on earth does the government imagine that giving the general public access to the research literature gives rise to more or faster "discoveries, innovations and cures"? Who does the government imagine is providing those "discoveries, innovations and cures"? It is not the general public but the small population of specialists who already have access to the research."

The requisite stronger rationale to counter these obvious (and valid) criticisms is precisely: research, from researchers, to researchers, for the sake of the research funder, the public -- along with the empirical evidence (from comparative usage and impact data for articles within the same journal issues that have and have not been made OA by their authors by self-archiving them) that the "small population of specialists who already have access to the research" is in reality only a fraction of its potential research usership.

But the primary, solid and unassailable rationale -- the real rationale for OA all along: research for researchers -- needs to be put first, up-front, rather than trying to put the self-archiving mandate across under the banner of the weaker, defeasible rationale: "research for public use."

[This could be supplemented by the case for the need for access to the primary research literature for students who are learning to become researchers (again not the general public).]

Nothing at all is lost from remedying the FRPAA wording in this way. Public access still comes with the OA territory. But it immunizes the Bill, pre-emptively, against these obvious (and valid) prima facie publisher counter-arguments, whereas the current version is positively provoking them.

In addition to this remediable flaw in the FPRAA's fundamental rationale for mandating self-archiving, there is also the functional flaw I mentioned in my previous posting on this topic (that of allowing any delay at all).

This second flaw is also easily remedied (by what Peter Suber has come to call the "dual deposit/release policy") which is simply to mandate immediate deposit for all FRPAA-funded articles, and allow the 6-month delay only for the timing of the OA access-setting (Open Access vs. Closed Access), rather than for the timing of the deposit itself.

That way, the new "Request Email Eprint" button -- now implemented in both the Eprints and Dspace Institutional Repositories and allowing individual users to request an email version directly from the author, semi-automatically -- will tide over any 6-month delay almost as effectively as immediate OA for all those would-be users who need it.

Stevan Harnad
American Scientist Open Access Forum

The US Federal Research Public Access Act (FRPAA) has gone a long way toward correcting the fundamental flaws of the NIH Public Access Policy.

Let me count the ways:

(1) FRPAA self-archiving is no longer requested but mandated.

(2) The absolute time limit on the FRPAA self-archiving is no longer at most 12 months from publication but at most 6.

(3) FRPAA no longer stipulates that the self-archiving must be central: the deposit can now be in the author's own Institutional Repository (IR).

(4) Self-archiving is no longer just for biomedical sciences, but for the full FRPAA spectrum of major US-funded research, in all funded fields.

Cf: "Model Self-Archiving Policies for Research Funders" (Jan 2006)

"DASER 2 IR Meeting and NIH Public Access Policy" (Dec 2005)

"Please Don't Copy-Cat Clone NIH-12 Non-OA Policy!" (Jan 2005)

"Open Access vs. NIH Back Access and Nature's Back-Sliding" (Jan 2005)

"A Simple Way to Optimize the NIH Public Access Policy" (Oct 2004)

And Peter Suber (see item 10 of his list of the top 10 OA stories) pinpoints the exact remedy for the FRPAA's sole remaining flaw (the fact that an embargo of up to 6 months is still allowed):

The remedy is to mandate that all articles must be deposited immediately upon acceptance for publication: the only allowable delay, if any, can only be in the moment when access to the deposit is set to Open Access (not in the moment when it is deposited). This allows authors to use their IRs' new "email eprint" button to provide immediate email access for all would-be users during any delay period: Eprints version; Dspace version.

Bravo to Senators Cornyn and Lieberman (and bravo also to Peter Suber, whose hand is clearly visible in the shape this policy has taken).

Now, the FRPAA is just a Bill, not an implemented policy. There is still time for the RCUK as well as the European Commission to get their acts together and implement their immediate-deposit self-archiving policies before the US does:

And of course nothing is stopping the worldwide network of universities and research institutions from adopting their own immediate-deposit mandates even before the big spenders do!

"Generic Rationale and Model for University Open Access Self-Archiving Mandate" (Mar 2006)

"European Commission recommends Open Access archiving for publicly-funded research" (Apr 2006)

"Wellcome Trust and the 6-month embargo" (Mar 2006)

"Comparing the Wellcome OA Policy and the RCUK (draft) Policy" (May 2005)

"Central versus institutional self-archiving" (Nov 2003)

"PubMed and self-archiving" (Aug 2003)

"Central vs. Distributed Archives" (Jun 1999)

Stevan Harnad
American Scientist Open Access Forum