Publishing Reform

---

SUMMARY: The Royal Society -- a green publisher lobbying against funders mandating that authors should GO on GREEN (by self-archiving) to provide OA -- is instead offering to provide OA for a fee. The asking fee factors in all current publishing expenses and revenues per article, including the print run and other values added (all currently paid for by institutional subscription revenue, with no sign of any decline) on the assumption that subscription revenue will fall to zero. Is it reasonable to expect article author-institutions today to pay for everything that is already being paid for today by other subscriber-institutions, just in order to provide free online access to the author-institution's peer-reviewed final drafts? Is peer review not the only added value they should ever have to pay for? And should they have to pay for it even while it is all being paid for by institutional subscriptions? Should they not just self-archive -- as the UK, US and EC are proposing to mandate -- and then if the "assumption" (really a hypothesis) that subscription revenue will as a result fall to zero ever proves correct, will the institutional subscription revenue savings not be the natural source from which to pay for the peer review? A priori author fees are fine for those with cash to spare, but surely they are not the royal road to 100% OA today: Mandated self-archiving is.

---

In the SPARC Open Access Forum, Ian Russell -- Head of Publishing, The Royal Society and new CEO of the Association of Learned and Professional Society Publishers -- wrote:

IR: "The Publishing Board of the Royal Society... felt... the fees for EXiS Open Choice should be set at a level which would allow for a viable publishing operation should EXiS Open Choice become the dominant model. The fees would therefore allow us to recover our costs should all authors choose the EXiS route and assuming that subscription revenue fell to zero [emphasis added]."

Assuming that subscription revenue fell to zero? But where is the evidence supporting that assumption? Indeed, where is the evidence of subscription revenue falling at all, as a result of authors self-archiving their own peer-reviewed final drafts online?

And in setting the a-priori fee level, were all current products/services and their costs/revenues factored in? for example, the production and distribution of the print edition, which is currently subscribed to by institutions worldwide?

Are the authors then, today, expected to pay to supply the print editions to the subscribing libraries of the world, if they wish to make their own articles OA?

Or does the Royal Society have evidence that if authors self-archived their own final peer-reviewed drafts online, as the research funders propose to mandate, then subscribing institutions would no longer want the print edition?

And if the print edition were no longer wanted, would its costs not need to be subtracted from the products/services and the costs/revenues that the Royal Society is here factoring into the asking price for the author's costs, a priori?

And might there not be other products/services, with their associated costs/revenues (mark-up, PDF, the online edition), that might likewise prove redundant or obsolete in a world in which the author's own final peer-reviewed drafts were all that was wanted? (Is that not all part of "assuming that subscription revenue fell to zero"?)

And if so, might the publisher's contribution not then reduce to merely the provision of peer review?

And is this not a question that the market (rather than publishers, by a priori assumption) should decide, once research institutions and funders have exercised their natural and long-overdue online-age prerogative to mandate that publicly funded research should be made openly accessible to all of its would-be users online (through the self-archiving of the author's peer-reviewed final draft online) rather than just to those who can afford to pay for the publisher's version, as now?

The peer review is indisputably both an added value and a necessity in all this: The peers review for free, but the process of peer review needs to be implemented, and that implementation needs to be paid for. But that can hardly be said a priori about all the other associated products and services, and hence their associated costs and revenues.

Right now, peer review is being paid for by revenues from all those other associated products and services, and paid for by subscribing institutions. If/when the institutional subscription demand for all those other associated products and services were ever to "fall to zero" (as assumed here), surely the institutions' own associated annual windfall subscription savings will correspondingly "rise from zero" pari passu with that fall, to become the natural source out of which to pay for the implementation of the peer review, several times over.

As for the other associated products and services (print edition, PDF, archiving, distribution, access-provision): if indeed subscription revenue from them "falls to zero" (because the author's online final peer-reviewed drafts turn out to be all that anyone wants any more) then the market will have spoken: The author's final peer-reviewed drafts are all that anyone wants any more. Why should research funders, or anyone, keep paying for other things that no one wants enough any more to pay for? (And why should they now agree to do that at this time, in full, and in advance, a priori?)

Or perhaps the assumption that subscription revenue will fall to zero if the authors are mandated to self-archive their final peer-reviewed drafts online is incorrect? Who can know this a priori, without doing the empirical test that the proposed self-archiving mandate itself would amount to?

For until that empirical test is actually performed, the only sure thing is that OA and its already demonstrated benefits to research access, usage and impact are needed, and needed now -- indeed already well overdue. A self-archiving mandate will provide that OA. That too has been demonstrated already. But all the rest is an open empirical question, for the market to decide.

IR: "We feel that most of the author fees in the market at present are set at an unsustainably low level and are setting unrealistic expectations among academics and other stakeholders. Some of the learned society publishers we have been speaking to have expressed concerns that the true author fee they would need to charge in an "author pays" world is much greater that the fees introduced by publishers so far and are concerned about looking uncompetitive. We want to introduce some clarity and go public with the fees that we - as a small learned society publisher with rejection rates of up to 75% - would need to charge in a fully "author pays" world."

The market today is a subscription market, and it is paying all the costs of all products and services associated with journal publishing today. What is missing is OA. Mandated author OA self-archiving will provide that missing OA, and its demonstrated benefits to research, researchers, and the public that funds them, with certainty. The rest is all pre-emptive speculation in place of objective testing and evidence (and rather self-serving speculation at that, aimed at pre-setting arbitrarily what should really be for the market to decide: whether anything else needs to paid for, and if so, when, how much, and how (by user-institution subscription fee or by author-institution publication fee).

Mandating author OA self-archiving, now, is the way to set the natural (and long-overdue) online-age process into motion that will determine whether OA can co-exist with the present subscription system, or requires a transition to OA publication -- and if the latter, what products and services still have a market, and what that market is willing to pay for them. What has already been delayed far too long, and should not be delayed a moment longer, is research institutions and funders mandating that the findings of the research they fund should immediately be made accessible to all of its would-be users online, and not only to those who can afford to pay for the publisher's version, as in the paper era.

IR: "You can of course argue that our fees are high, but they are what they are: what we would need to charge to stay in business without the benefit of philanthropic grants to maintain our publishing operation. We are confident that we are cost efficient and that our fees are in line with what most learned society publishers would need to charge in a fully OA model."

Fine. Now what needs to be determined -- by the market, empirically -- is whether all the products and services wrapped into the current "publishing operation" still have a market in an OA world, or whether perhaps the only thing that is still needed is peer review. That is the way to find out what needs to be charged for what in order to stay in business in an OA world. Not by insisting that the research institutions and funders pay for it all up-front at this time (which is precisely what the Royal Society is doing, if it names its asking price for paid OA a-priori, and opposes funder-mandated OA self-archiving by authors).
Stevan Harnad
American Scientist Open Access Forum

---

SUMMARY: The Royal Society is a green publisher, giving its authors the green light to provide immediate Open Access to their articles by self-archiving them in their own institutional repositories in order to maximise their usage and impact. The Royal Society is now also an optional gold publisher, offering its authors the "Open Choice" of providing Open Access on their behalf, for a fee. But all of this is outweighed by the fact that this most venerable of Learned Societies, contrary to the wishes of at least 64 of its (unconsulted) members, has put its substantial prestige and gravitas behind a vehement -- and so far successful -- lobby against the Research Councils UK proposal to mandate author self-archiving by its fundees, as recommended by the UK parliamentary Select Committee on Science and Technology as well as the U.S. Federal Research Public Access Act, and the European Commission. In this respect, the Royal Society is deporting itself exactly like the crassest of commercial publishers, and is putting a sad blemish on its proud history in the annals of Learned Inquiry and the dissemination of its fruits.

---

It is fine that the Royal Society is experimenting with the "EXiS Open Choice" option (giving individual authors the choice to pay their journal to make their article Open Access [OA] for them), but this is a minor gesture, given that the Royal Society is meanwhile also stoutly -- and so far successfully -- opposing the UK recommendation to mandate that all RCUK fundees must make their own articles OA by depositing them in their own institutional (or central) OA repositories.

What the research world needs today is OA, now: immediate 100% OA (not necessarily OA publishing: OA itself). It is a matter of historical record that (without consulting its membership) the Royal Society, driven by its publishing arm -- and exactly as many other (decidedly non-royal) publishers have done -- has shrilly opposed the RCUK proposal to mandate that UK-funded researchers provide immediate OA by self-archiving their research: opposed it on the basis of no evidence whatsoever, just speculative hypotheses of doom and gloom (eliciting great disappointment in the Royal Society's admirers, as well as an open letter of protest from 64 of its members, including 6 Nobel Laureates, opposed to the Royal Society's stance on OA). (See 1, 2, and 3.)

The fact that the Royal Society, like a number of other publishers, is now trying a leisurely experiment with Open Choice by offering their authors and their institutions the option of paying (a hefty and rather arbitrary fee) for OA is next to ludicrous in this context -- while institutional funds are still tied up in subscriptions, while there is no evidence that self-archiving reduces subscriptions, and while publishers are vigorously opposing self-archiving mandates on the grounds that they might reduce subscriptions.

Although the analogy is unfairly shrill, it is useful in order to make the underlying logic transparent if we note that this is not unlike a call for an immediate public-smoking ban being opposed by a royal tobacco company, with a counter-offer to sell individual clients an alternative smoke-free product, as a matter of (paid) personal choice.

We will never even come near 100% OA if we keep waiting passively for the 24,000 journals to convert to paid OA publishing, one by one, author by author, under these conditions. OA and hybrid OC (Open Choice) journals today are merely a sop for the ongoing worldwide need for immediate OA: They do little to stanch the daily, needless hemorrhaging of research usage and impact.

An OA self-archiving mandate for publicly funded research, as proposed by the RCUK, FRPAA and EC (and already implemented by the Wellcome Trust and 6 universities and research institutions) would (like a public-smoking ban) be a genuine remedy, but the Royal Society is opposing it.

This is a sad historical fact -- even though, to its credit, the Royal Society's 7 journals are among the 94% of journals that have endorsed their authors' right to exercise the choice of self-archiving their own papers, if they wish:

"the author(s) may... post the work in its published form on their personal or their employing institution's web site"

It is just that the choice the Royal Society affirms with one hand, it lobbies vigorously with the other hand to discourage authors, their institutions and funders from actually exercising.

There is absolutely nothing in the Royal Society's ignoble deportment today that warrants making any reference whatsoever to its noble history in the evolution of research and publishing. The less said about that, the better. This is a business, acting in the interests of its bottom line, not a Learned Society acting in the interests of Learned Inquiry.

American Scientist Open Access Forum Topic Thread:
"Not a Proud Day in the Annals of the Royal Society" (Nov 24, 2005)

Stevan Harnad
American Scientist Open Access Forum

---

SUMMARY: Nature reports that Open Access (OA) journals are having trouble making ends meet. This is because institutional publication funds are currently tied up in subscription costs. What is urgently needed now is OA, to maximize research usage and impact. Immediate 100% OA can be reached via OA self-archiving mandates. If/when 100% OA self-archiving should ever generate institutional subscription cancellations, those same institutional windfall savings will be the natural way to pay for institutional OA publication costs. If/when there are signs that that is approaching, then would be the opportune time for journals to convert to OA publishing. Right now, there are no such signs, and it is OA that we need, urgently.

---

Re: Butler, Declan (2006) Open-access journal hits rocky times. Nature 20 June 2006 doi:10.1038/441914a (See also Declan's Blog)

The reason Open Access Journals are having trouble making ends meet today is quite obvious: The funds for paying author-institution publication costs are already tied up in paying user-institution subscription costs.

Open Access (OA) itself is urgent, for research, researchers and the public that funds the research, because every day without OA means another day of needless loss in research usage and impact. But conversion to OA publishing is not urgent, indeed it is premature, while funds are already tied up in subscriptions.

What needs to be done now is for researchers, funders and institutions to mandate OA self-archiving -- i.e., require their researchers to deposit their published articles in their own OA Institutional Repositories, for the sake of maximizing the uptake, usage and impact of their research output. Self-archiving is a supplement to -- not a substitute for -- conventional subscription-based journal publishing.

If and when OA self-archiving should ever generate cancellation pressure on institutional journal subscriptions, then, and only then, need there be a conversion to OA Publishing. For then the institutional windfall savings from the cancellation of incoming subscriptions will be available to pay the costs of outgoing publication, which will be based on downsizing publishing to the essentials, such as peer review, cutting obsolete costs once the network of institutional repositories become the archivers and access-providers.

Right now, however, there is no sign of any cancellation pressure from self-archiving, even in the fields that have been practising it the longest (15 years in physics) and the subfields that already reached 100% OA some time ago. What is urgent now is to mandate OA self-archiving.

Stevan Harnad
American Scientist Open Access Forum

---

Update Jan 1, 2010: See Gargouri, Y; C Hajjem, V Larivière, Y Gingras, L Carr,T Brody & S Harnad (2010) “Open Access, Whether Self-Selected or Mandated, Increases Citation Impact, Especially for Higher Quality Research”
Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

As Eysenbach’s very long and remarkably intemperate response to my prior response mainly repeats prior (answered) points, I will respond only to his very few substantive points:

I had asked: “Does [Eysenbach] seriously think that partialling out the variance in the number of authors would make a dent in that huge, consistent effect [the within-journal citation advantage for self-archived articles]?”

GE: “the answer is ‘absolutely’… If high-author papers are overrepresented in self-archived papers, then this confounder alone will contribute to having a greater number of citations… Only if one statistically controls for all these confounders (there are several of them - see PLoS paper), and one STILL sees an open access citation advantage, then (and only then) one has a SOLID, defendable study. ”

Here is Eysenbach’s list of confounders :

(1) number of authors: As Eysenbach says he is serious, we will now test this. We have the data. Eysenbach’s prediction is that partialling out the effect of the number of authors will make a dent in our huge, consistent citation advantage. Stay tuned...

(2) number of days since publication: This is relevant and feasible in a 1-year, 1-journal study like Eysenbach’s but neither relevant nor feasible for a sample of over a million articles ranging over 12 years, 12 disciplines, and hundreds of journals -- all showing exactly the same citation advantage for self-archived articles in every year and every discipline.

(3) article type: We are able to test this separately too (because we have ISI data on article type) but first let’s see whether partialling out author numbers makes a dent in our basic effect.

(4) country of the corresponding author: This is testable too, but first let’s see how the author-number ‘confounder’ pans out (we could look at the first-author's birth-sign too...).

(5) funding type: Data not available, and extremely far-fetched.

(6) subject area: Already tested and reported in our data, separately for 12 different disciplines : the self-archiving advantage is consistently present in all of them.

(7) submission track (PNAS has three different ways that authors can submit a paper): Not relevant to the journals we tested, which were all non-OA and pre-dated Open Choice.

(8) previous citation record of the first and last authors: This, as I noted, is -- along with the demonstration of how early the OA advantage emerges in PNAS – a potentially interesting variable in the fine-tuning of the OA advantage, but our own studies are concerned with estimating the generality and size of the OA advantage, not with its fine tuning.

(9) whether authors choosing the OA option in PNAS chose to do so for only their most important research (“they didn't”): Neither Eysenbach’s study nor ours can confirm causality or eliminate the possibility of self-selection bias.

GE: "the fact that we look at a immediate (gold-)OA article population in a longitudinal cohort study design takes care of the “arrow of causation” problem, because it makes sure that open access status comes first, then the citations are coming, not the other way round.

I'm afraid it's not quite that easy to take care of the "arrow of causation" problem, which is confounded (sic) with the problem of self-selection bias: For if authors are (contrary to their subjective reports) indeed self-selecting their better papers (or themselves!) for OA-gold (or for self-archiving) then that, and not the OA, could explain why their papers get more citations.

GE: “it is entirely possible that the articles in his sample (which he refers to as green-OA articles) were not “immediately” self-archived after publication, but 1 month, 6 months, or 12 months after original publication, therefore not really what Harnad refers to as green-OA, implying “immediate” deposition.”

This is actually a valid point of definition: OA should be defined as ‘immediate’ in order to rule out claims that delayed/embargoed access is Open Access. The point at which refereed research can and should begin to be used is when the final refereed draft is accepted for publication, and that is the point when it should be made freely accessible online. So a portion of the citation advantage for self-archived articles could well have come from self-archiving later than the publication date; technically speaking this should be called a ‘free access’ advantage, if we reserve the term "OA" for access that is free immediately. But surely nothing of substance rides on this: If there is a self-archiving advantage even for tardy self-archiving, that confirms, a fortiori, the self-archiving advantage of prompt (OA) self archiving too!

GE: “I… made a conscientious decision to submit my paper to a gold-OA journal (PLoS) rather than publishing the study in an obscure scientrometrics journal and then self-archived [sic] it”

Actually, unless I am mistaken, I seem to recall corresondence from GE to the effect that it was first declined by Science (or was it Nature?) – not a gold-OA journal – before being submitted to PloS Biology)…

GE: “The visibility of an article published in a properly promoted OA journal site will always be better than a paper that is published in a toll-access journal site, even if it is self-archived. This is exactly why my study shows an advantage of gold-OA over green-OA, this is also why I personally chose the gold route to publish this paper in PLoS, and not the green route”

Let us not confound a journal's profile/impact level with its OA/non-OA status.

The visibility (and no doubt also the citation impact) of an article will always be better when it is published in a high-profile, high-impact journal, whether it is OA (like PLoS) or non-OA (like Science or Nature) rather than an obscure scientometrics journal (or an obscure OA journal). Its visibility and impact will be higher if self-archived in either case (except perhaps if the journal is both high-profile and optional-OA, which is partly what Eysenbach’s study has shown).

GE: “the PLoS paper is the first study which contains an analysis of both gold and green (thus focuses on “OA itself”), whereas the rest of the studies is actually focused on ‘green’”.

Because most of the existing data for within-journal OA/non-OA comparisons comes from the millions of articles published in the thousands of non-gold journals indexed by ISI and not just the thousands of articles published in the few journals that are as yet (like PNAS) optional-gold...

Stevan Harnad
American Scientist Open Access Forum

---

Update Jan 1, 2010: See Gargouri, Y; C Hajjem, V Larivière, Y Gingras, L Carr,T Brody & S Harnad (2010) “Open Access, Whether Self-Selected or Mandated, Increases Citation Impact, Especially for Higher Quality Research”
Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

PLoS seems to have concluded that it is not in their interest to host further public contributions to the Eysenbach debate from me -- and perhaps they are right (that it is not in their interest)...

From: plos AT plos.org
Date: May 19, 2006 3:13:07 PM EDT (CA)
To: Stevan Harnad harnad AT ecs.soton.ac.uk
Subject: e-Letter: Decision

Unfortunately we decided not to accept your e-Letter. Letters are published at the editors' discretion, and we publish only those that we believe will contribute substantially to the debate. Our editorial decisions about publishing letters are final, and are not open to appeal.

Below is appended the mercilessly compressed fragment that I had submitted to PLoS as a follow-up letter (responding to Eysenbach's PLoS letter responding to my PLoS letter responding to his PLoS article).

The full version of my reply of course appeared on AmSci and in my Archivangelism blog -- but, as Eysenbach's study showed, one gets still further visibility from appearing on the website of a high-profile, high-impact journal! My (valid) rebuttal to Eysenbach's suggestion that self-archiving is to OA publishing as handing out leaflets is to publishing in a newspaper was that we are talking about publications in the case of OA, not unpublished materials! But with a letter, it's more like handing out leaflets when it just appears in a Forum or a Blog, versus the website of a high-impact journal...

Never mind. The content is what matters, and time is on OA's side (even if it is much too dilatory!), because OA is (you've heard the song!): Optimal and Inevitable.

---

Confirming the Within-Journal OA Impact Advantage

(Click for Fuller version)

Given the large within-journal OA citation impact advantages repeatedly found across all journals, disciplines and years in samples four orders of magnitude larger than Eysenbach's, it is not clear that controls for "multiple confounders" are needed to demonstrate the reality, magnitude and universality of the OA advantage. (This does not mean Eysenbach’s controls are not useful, just that they are not yet telling us much that we don't already know.)

Eysenbach (and PLoS) are focussed on gold-OA journals; most other OA impact studies are focused on OA itself. Only ~10% of journals are gold today. Few as yet offer authors "Open Choice" (allowing gold within-journal OA/NOA comparisons) and few authors are as yet choosing paid OA.

Regarding the “arrow of causation: yes, “longtitudinal cohort” data would demonstrate causation (for skeptics who think the OA advantage might be a self-selection bias) but Eysenbach's author self-reports certainly aren’t such data! Meanwhile: (a) the OA advantage does not diminish for younger articles; (b) OA increases downloads; (c) increased downloads in the first 6 months correlate with increased citations later; (d) unaffordability reduces access; (e) access is a necessary condition for citation.

About OA being a “continuum” or “spectrum”: Time is certainly a continuum, and access certainly admits of degrees (access may be easier/harder, narrower/wider, cheaper/dearer, longer/shorter, earlier/later, partial/full) -- but Open Access does not admit of degrees (any more than pregnancy does). OA is defined as: full-text online access, free for all.

Eysenbach likens self-archiving to “printing something on a flyer and handing it out to pedestrians on the street [instead of] publishing an article in a national newspaper." But it is published articles that are being self-archived.

NOA (Not OA): 1159 articles (86.2% cited at least once)
POA (Payed OA only): 176 (94.3%)
SOA (Self-Archived OA only): 121 (90.1%)
BOA (POA and SOA): 36 (97.2%)

In this PNAS sample, POA, SOA and BOA together, and POA alone, all have significantly more citations than NOA, but SOA alone ("stratified") does not; also, both POA and SOA increase citations, but POA does it more.

Three possible hypotheses explaining the BOA>POA>>SOA>NOA outcome:

H1: The POA advantage might be a multiple-archiving effect, maximal for high-profile , 3-option (POA, SOA, NOA) journals like PNAS because POA articles are more visible than SOA. (POA + SOA = BOA highest of all: redundancy helps!) As Institutional Repositories fill, this extra advantage will disappear.

H2: The POA advantage might arise in part from self-selection because the decision to pay for POA is influenced by the author's sense of the potential importance (hence impact) of his article. (But I think self-selection quality-bias is just one of many contributors to the OA advantage itself, not the only one or the biggest.)

H3: The POA advantage might be either a small-sample chance result or a temporary side-effect of the 3-option journals in early days: a one-stop shopping advantage for PNAS articles, in a high-profile store, today. It needs to be tested for replicability and representativeness in larger samples of articles, journals, and time-bases.

The true measure of the SOA advantage today is not found in PNAS but in the far more populous and representative full spectrum of journals not yet offering POA. (I’d be delighted if those journals took Eysenbach’s findings as a reason for offering a POA option! But not at the expense of authors wrongly inferring that for the journals they currently publish in, SOA alone would not confer citation advantages at least as big as the ones we have been reporting.)

Harnad & Brody (2004)
Brody et al (2005)
Hajjem et al (2005)

Stevan Harnad
American Scientist Open Access Forum

---

Update Jan 1, 2010: See Gargouri, Y; C Hajjem, V Larivière, Y Gingras, L Carr,T Brody & S Harnad (2010) “Open Access, Whether Self-Selected or Mandated, Increases Citation Impact, Especially for Higher Quality Research”
Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

Gunther Eysenbach (GE) (in a letter in letter PLoS Biology today) wrote:

GE: "The introduction of the article and two accompanying editorials [1, 2, 3] already answer Harnad's questions why author, editors, and reviewers were critical of the methodology employed in previous studies, which all only looked at "green OA" (self-archived/online-accessible papers)"

I didn't ask why the author and editors were critical of prior self-archiving (green OA) studies; I asked why they said such studies were "surprisingly hard to find" and why the two biggest and latest of them were not even taken into account:

Brody, T., Harnad, S. and Carr, L. (2005) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 56.

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47.

And the reason all prior within-journal studies look only at "green OA" is that the majority of OA today is green; hence almost all OA/NOA impact comparisons are based on green OA (self-archiving) rather than on paid-OA (gold). To compare OA and NOA between rather than within journals would be to compare apples and oranges: See critique of ISI's between-journal OA/NOA comparisons in:

Brody, T. and Harnad, S. (2004) Comparing the Impact of Open Access (OA) vs. Non-OA Articles in the Same Journals. D-Lib Magazine 10(6).

GE: " (hint 1: "confounding") (hint 2: arrow of causation: are papers online because they are highly cited, or the other way round?)."

I am afraid I don't see Eysenbach's point here at all: What exactly does he think is being confounded in within-journal comparisons of self-archived versus non-self-archived articles? The paid-OA effect? But among OA articles today there is almost zero within-journal paid-OA, because so few journals offer it! (Hajjem et al.'s within-journal comparisons were based on over a million articles, across 12 years and hundreds of journals, in 12 disciplines! Eysenbach's were based on 1492 articles, in 6 months, in one journal.)

And is Eysenbach suggesting that his failure to find any significant difference among author self-reports -- about their own article's quality and its causal role in their decision about whether or not to pay for OA (or to self-archive) in his sample of 237 authors -- is an objective test of the arrow of causation? (I agree that Eysenbach's failure to find a difference fails to support the hypothesis of a self-selection bias, but surely that won't convince those who are minded to hold that hypothesis! I would welcome rigorous causal evidence against the self-selection hypothesis as much as Eysenbach would, but author self-reports are alas not that evidence!)

GE: " The statement in the PLoS editorial has to be seen against this background. None of the previous papers in the bibliography mentioned by Harnad employed a similar methodology, working with data from a "gold-OA" journal."

Yes, almost all prior studies on the OA impact advantage are based on green OA, not gold, but so what? It is Eysenbach (and PLoS) who are focussed on gold-OA journals; the rest of the studies are focussed on OA itself. Only about 10% of the planet's peer-reviewed journals are gold today, and most of those are 100% gold, hence allow no within-journal comparisons. Very few journals as yet offer authors the "Open Choice" (optional paid gold) that would allow gold within-journal OA/NOA comparisons; and few authors are as yet taking those journals up on it (about 15% in this PNAS sample), compared to the far larger number that are self-archiving (also 15%, as it happens, though that percentage too is still far too small!). The difference in article sample sizes is about four orders of magnitude (c. 1500 articles in Eysenbach's study to 1.5 million in Hajjem et al's).

GE: " The correct method to control for problem 1 (multiple confounders) is multivariate regression analysis, not used in previous studies."

Correct. But with the large, consistent within-journal OA/NOA differences found across al journals, all disciplines and all years in samples four orders of magnitude larger than Eysenbach's, it is not at all clear that controls for those "multiple confounders" are necessary in order to demonstrate the reality, magnitude and universality of the OA advantage. That does not mean the controls are not useful, just that they are not yet telling us much that we don't already know.

GE: " Harnad's statement that "many [of the confounding variables] are peculiar to this particular... study" suggests that he might still not fully appreciate the issue of confounding. Does he suggest that in his samples there are no differences in these variables (for example, number of authors) between the groups? Did he even test for these? If he did, why was this not described in these previous studies?"

No, we did not test for "confounding effects" of number of authors: What confounding effects does Eysenbach expect from controlling for number of authors in a sample of over a million articles across a dozen disciplines and a dozen years all showing the very same, sizeable OA advantage? Does he seriously think that partialling out the variance in the number of authors would make a dent in that huge, consistent effect?

Not that Eysenbach's tentative findings on 1st-author/last-author differences in his one-journal sample of 1492 are not interesting; but those are merely minor differences in shading, compared to the whopping main effect, which is: substantially more citations (and downloads) for self-archived OA articles.

GE: " The correct method to address problem 2 (the "arrow of causation" problem) is to do a longitudinal (cohort) study, as opposed to a cross-sectional study. This ascertains that OA comes first and THEN the paper is cited highly, while previous cross-sectional studies in the area of "green OA" publishing (self-archiving) leave open what comes first -- impact or being online."

I agree completely that time-based studies are necessary to demonstrate causation, for those who think that the OA advantage might be based on self-selection bias (i.e., that high-impact studies tend to be preferentially self-archived, perhaps even after they have gained their high impact), but Eysenbach's author self-report data certainly don't constitute such a longtitudinal cohort study! (Once there exist reliable deposit dates for self-archived articles, we will be able to do some time-based analyses on green OA too, but, frankly, by that time the outcome is likely to be a foregone conclusion.)

In the meanwhile, the fact that (a) the OA advantage does not diminish for younger articles (as one would expect if it were a post-hoc effect), that (b) OA increases downloads, and that (c) increased downloads in the first 6 months are correlated with increased citations later on -- plus the logic of the fact that (d) unaffordability reduces access and that (e) access is a necessary condition for citation -- all suggest that most of the scepticism about the SOA advantage is because of conflicting interests, not because of objective uncertainty.

GE: " Harnad - who usually carefully distinguishes between "green" and "gold" OA publishing -- ignores that open access is a continuum, much as publishing is a continuum"

I'm afraid I have no idea what Eysenbach means about OA being a continuum: Time is certainly a continuum, and access certainly admits of degrees (access may be easier/harder, narrower/wider, cheaper/dearer, longer/shorter, earlier/later, partial/full) -- but Open Access does not admit of degrees (any more than pregnancy does). OA means immediate, permanent, full-text online access, free for all, now.

And, by the way, green OA is certainly not a lesser degree of gold OA!

For the innocent reader, puzzled as to why this would even be an issue:

Please recall that OA (gold) journals, whether total or optional gold, need authors (and those gold journals with the gold cost-recovery model need paying author/institutions). To attract authors, gold journals need to persuade them of the benefits of OA. So far so good. But there is another thing they have to persuade them of, implicitly or explicitly, and that is the benefits of gold OA over green OA. For if there are no benefits of gold over green, then surely it makes much more sense for authors to publish in their journal of choice, as they always did, and simply self-archive their own articles, rather than switching journals and/or paying for gold OA!

This theme alas keeps recurring, implicitly or explicitly, in the internecine green/gold squabbles, because green OA is indeed a rival to gold OA in gold OA journals' efforts to win over authors. This is regrettable, but a functional fact today, owing to the nature of OA and of the two means of providing it.

Is the effect symmetrical? Is gold OA likewise a rival to green OA? Here the answer is more complicated: No, an author who chooses gold OA (by publishing in an OA journal) is not at all a loss for green OA, because the article is nevertheless OA, and green OA's sole objective is 100% OA, as soon as possible, and nothing else. (Besides, a gold OA article too can be self-archived in the author's Institutional Repository if the author or institution wishes! All gold journals are, a fortiori, also green, in that they endorse author self-archiving.)

But there is a potential problem with gold from the standpoint of green. The problem is not with authors choosing gold. The problem is with gold publishers promoting gold as superior to green, or, worse, with gold publishers implying that green OA is not really OA, or not "fully" OA (along some imaginary OA "continuum").

"Free Access vs. Open Access" (thread started Aug 2003)

Why, you ask, would gold OA want to give the impression that green OA was not "really" OA or not "fully" OA? Because of the rivalry for authors that I just mentioned. The causal arrow is a one-way one insofar as competition for authors is concerned: green OA does not lose an author if that author publishes in a gold OA journal, whereas gold OA does lose an author if an author publishes in a green journal instead of a gold one. However, if gold portrays green as if it were not really or fully OA, and authors believe this, then it loses author momentum for green -- especially among that vast majority of authors who do not yet elect to publish gold. For there is today something still very paradoxical, indeed equivocal, about author behavior and motivation vis-a-vis OA:

Authors profess to want OA. Thirty-four thousand of them even signed the 2001 PLoS Open Letter threatening to boycott their journals if they did not provide (gold) OA (within 6 months of publication). (Most journals did not comply, and most authors did not follow through on their boycott threat: How could they? There were not enough suitable gold journals for them to switch to, and most authors clearly were not interested in switching journals, let alone paying for publication, then or now.)

Yet (and here comes the paradox): if those 34,000 signatories -- allegedly so desirous of OA as to be ready to boycott their journals if they did not provide it -- had simply gone on to self-archive all their papers, they would be well on the road to having the OA they allegedly desired so much! For the green road to 100% OA happens to be based on the (golden!) rule: Self-Archive Unto Others As You Would Have Them Self-Archive Unto You.

Why didn't (and don't) most authors do it (yet)? It is partly (let us state it quite frankly) straightforward foolishness and inconsistency on authors' part. They simply have not thought it through. This cannot be denied. Authors are in a state of self-induced "Zeno's Paralysis" regarding OA, from which FAQs have so far been powerless to free them -- so that it now looks as if self-archiving mandates from their institutions and/or their funders will be the only thing that can induce them to do what will give them what they so want and need.

But the confusion and inaction are partly also the fault of the promotional efforts of (well-meaning) OA advocates. Harold Varmus sent a mixed message with his 1999 "E-biomed" proposal (which led to PLoS, the PLoS Open letter, PubMed Central, Biomed Central, and eventually the PLoS and BMC fleet of OA journals, including PLoS Biology). Was E-biomed a gold proposal, a green proposal, both, or neither? The fact is that it was an incoherent proposal -- a confused and confusing mish-mash of central self-archiving, publishing reform/replacement and rival publishing -- and although it has undeniably led to genuine and valuable progress toward (what was eventually baptized by BOAI as) OA, it has left a continuing legacy of continuing confusion too.

And we are facing part of that legacy of confusion now, with PLoS thinking that the only way (or the best) to reach 100% OA is to publish and promote gold OA journals. That is why PLoS Biology agreed to referee the Eysenbach paper, which seemed to show that OA gold is the only one that increases citation impact, not green self-archiving, which is (when you come right down to it) not even "real" OA at all!

That is also why PLoS Biology editorialised that they found it "surprisingly hard to find" evidence -- "solid evidence" -- that OA articles are read and cited more. And that is why PLoS Biology was happy to make an exception and publish the Eysenbach study, even though scientometrics is not the subject matter of PLoS Biology, but (I'll warrant) PLoS Biology would not have been happy to advertise in its pages the fact that green OA self-archiving was enough to get articles read and cited more!

So green OA does have a bit of an uphill battle against gold OA and the subsidies and support it has received (because gold OA is an attractive and understandable idea, whereas green OA requires a few more mental steps to dope out -- though not many, as none of this is rocket science!).

But, to switch metaphors, the green road to 100% OA (sic) is far wider, faster and surer than the golden road. (Every article can be self-archived, today, and without their authors' having to renounce or switch journals, whereas most articles do not yet have a suitable OA journal to publish in today, even if their authors wished to switch journals, which most do not; and authors can be mandated to self-archive by their institutions and funders, but neither authors' choice of journals nor their publishers' choice of access-provision or cost-recovery model can be mandated by authors' institutions and funders.) Moreover, 100% OA really is beneficial to research and researchers; so the green road of self-archiving is bound to prevail, despite the extra obstacles. And the destination (100% OA) is exactly the same for both roads. (Indeed, I am pretty sure that even the fastest way to reach 100% gold OA -- i.e., not just 100% OA but also the conversion of all journals to gold -- is in fact to take the green road to 100% OA first.

So gold is doing itself a disservice when it tries to devalue green. Read on:

GE: " and this study (and the priority claims in the editorial) was talking about the gold OA end of the spectrum."

Spectrum? Continuum? Degrees of OA?

GE: " Publishing in an open access journal is a fundamentally different process from putting a paper published in a toll-access journal on the Internet. In analogy, printing something on a flyer and handing it out to pedestrians on the street, and publishing an article in a national newspaper can both be called "publishing", but they remain fundamentally different processes, with differences in impact, reach, etc. A study looking at the impact of publishing a newspaper can not be replaced with a study looking at the impact of handing out a flyer to pedestrians, even though both are about "publishing"."

Oh dear! I have a feeling Eysenbach is going to tell as that making a published journal article accessible online free for all by self-archiving it is not OA after all, or not "full OA". If the journal doesn't do it for you, and/or you don't pay for it, it's not the real thing.

I wonder why Eysenbach would want to say that? Could it be because he is promoting an OA (gold) journal (his own)? Could that also have been the reason the PLoS editorial was so sanguine about Eysenbach's findings on the OA gold advantage, and so dismissive of any prior evidence of an OA green advantage?

GE: " Finally, Harnad says that "prior evidence derived from substantially larger and broader-based samples showing substantially the same outcome". I rebut with two points here[:] Regarding "larger samples" I think rigor and quality (leading to internal validity) is more important than quantity (or sample size)."

Even when all within-journal studies -- large and small, approximate and exact -- just keep producing exactly the same outcome, every time (OA increases impact)?

GE: " Going through the laborious effort to extract article and author characteristics for a limited number of articles (n = 1492) in order to control for these confounders provides scientifically stronger evidence than doing a crude, unadjusted analysis of a huge number of online accessible vs non-online accessible articles, leaving open many alternative explanations."

As I said, for those who doubt the causality and think the OA advantage is just a self-selection bias, Eysenbach's study will not convince them otherwise either. For those with eyes to see, the repeated demonstrations, in field after field, of exactly the same effect on incomparably larger samples will already have been demonstration enough. For those with eyes only for gold, evidence that green enhances citations will never be "solid evidence."

If Eysenbach and the editors had portrayed the latest PLoS findings as they should have, namely, as yet another confirmation of the OA impact advantage, with some new details about its fine-tuning, I would have done had nothing but praise for it. But the actual self-interested spin and puffery that instead accompanied this work -- propagating the frankly false idea that this is the first "solid evidence" for the OA impact advantage, and, worse, that it implies that self-archiving itself does not deliver the OA impact advantage -- would have required not the lack of an ego, but the lack of any real fealty to OA itself to have been allowed to stand uncontested.

GE: " Secondly, contrary to what Harnad said, this study is NOT at all "showing substantially the same outcome". On the contrary, the effect of green-OA -- once controlled for confounders - was much less than what others have claimed in previous papers."

Let's be quite explicit about what, exactly, we are discussing here:

Eysenbach found that in a 6-month sample of 1492 articles in one 3-option journal (PNAS):

"While in the crude analysis self-archived papers had on average significantly more citations than non-self-archived papers (mean, 5.46 versus 4.66; Wilcoxon Z = 2.417; p = 0.02), these differences disappeared when stratified for journal OA status (p= 0.10 in the group of articles published originally as non-OA articles, and p = 0.25 in the group of articles published originally as OA).

"In a logistic regression model with backward elimination, which included original OA status and self-archiving OA status as separate independent variables as well as all potential confounders, self-archiving OA status did not remain a significant predictor for being cited. In a linear regression model, the influence of the covariate "article published originally as OA, without being self-archived" (beta = 0.250, p < 0.001) on citations remained stronger than self-archiving status (beta = 0.152, p = 0.02)."

To translate this into english (from an article with exceedingly user-unfriendly data-displays, by the way, making it next to impossible to extract and visualize results from the tables by inspection!): First, the numbers:

NOA (Not OA): (1159 articles 86.2% cited at least once)
POA (Payed OA only): (176 articles 94.3% cited at least once)
SOA (Self-Archived OA only): (121 articles 90.1% cited at least once)
BOA (POA and SOA): ( 36 articles 97.2% cited at least once)

The finding is that (in this PNAS sample, and with many other factors -- e.g., days since publication, number of authors, article type, country, funding, subject, etc. -- statistically isolated so as to be asessable independently): POA, SOA and BOA considered together, and PAO considered alone, all have significantly more citations than NOA; but SOA considered alone ("stratified") does not. Also, if considered jointly (multiple regression), both POA and SOA increase citations, but POA is the stronger effect.

Here are three simple hypotheses, in decreasing order of likelihood, as to why this small PNAS study may have found that the citation counts and their significance ordered themselves as they did: BOA>POA>>SOA>NOA

Hypothesis 1: The POA advantage might be unique to high-profile 3-option journals (POA, SOA, NOA) like PNAS (which are themselves a tiny minority among journals) and occurs because the POA articles are more visible than the SOA articles. (The POA + SOA = BOA articles do the best of all: redundancy enhances visibility.) So the POA authors do get something more for their money (but that something is not OA but high-profile POA in a high-profile journal) -- at least for the time being. This extra POA-over-SOA advantage will of course wash out as SOA and indexed, interoperable Institutional Repositories for self-archiving grow.

Hypothesis 2: The POA advantage might result at least in part from QB (self-selection Quality Bias) because the decision (by a self-selected 15% subset of PNAS authors) to pay for POA is influenced by the author's underlying sense of the potential importance (hence impact) of his article: Simply asking authors about how important they think their article is, and whether that influenced their decision to pick POA or SOA or NOA, and failing to detect any significant difference among the authors, does not settle this matter, and certainly not on the basis of such a small and special sample. (But I think QB is just one of many contributors to the OA citation advantage itself, and certainly not the only determinant or even the biggest one.)

Hypothesis 3: The POA advantage might be either a small-sample chance result or a temporary side-effect of the 3-option journals in early days: a one-stop shopping advantage for PNAS articles, in a high-profile store, today. It needs to be tested for replicability and representativeness in larger samples of articles, journals, and time-bases.

(Note that Lawrence's 2001 as well as Hajjem et al's 2005 finding had been that the proportion of OA articles increases in the higher citation ranges, being lowest among articles with 0-1 citations.)

Eysenbach finds that with logistic regression analysis separating the independent effects of POA, SOA and other correlates, SOA has no significant independent effect in his 1-year PNAS sample. Now let's test whether that replicates in larger samples, both in terms of number of articles, journals, and time-base. (Failure to find a significant effect in a small sample is far less compelling than success in finding a significant effect in a small sample!)

GE: " Harnad, a self-confessed "archivangalist", co-creator of a self-archiving platform, and an outspoken advocate of self-archiving (speaking of vested interests) calls the finding that self-archived articles are... cited less often than [gold] OA articles from the same journal "controversial". In my mind, the finding that the impact of nonOA < greenOA < goldOA < green+goldOA is intuitive and logical: The level of citations correlates with the level of openness and accessibility."

I don't dispute that POA can add more citations, just as BOA can; maybe self-archiving in 10 different places will add still more. But what does this imply, right now, practically speaking? And, even more important, how likely is it that this sort of redundancy will continue to confer significant citation advantages once a critical mass of the literature is in interoperable Institutional Repositories (green SOA) rather than few and far between, as now? It is indeed intuitive and logical that the baseline 15% of the literature as a whole that is being spontaneously self-archived somewhere, somehow on the Web, across all fields, has somewhat less visibility right now than the 15% of PNAS articles that PNAS is making OA for those authors who pay for it (POA). That's a one-stop shopping advantage for PNAS articles, against PNAS articles, in a high-profile store, today.

But the true measure of the SOA advantage today (at its 15% spontaneous baseline) is surely not to be found in PNAS but in the statistically far more numerous, hence far more representative full-spectrum of journals that do not yet offer POA. (I would be delighted if those journals took the Eysenbach findings as a reason for offering a POA option! But not at the expense of authors drawing the absurd conclusion -- not at all entailed by Eysenbach's PNAS-specific results -- that in the journals they currently publish in, SOA alone would not confer citation advantages at least as big as the ones we have been reporting.)

Regarding my self-confessed sin of archivanglizing, however, I do protest that my first and only allegiance is to 100% OA, and I evangelize the green road (and promote the self-archiving software) only because it is so resoundingly obvious that it is the fastest and surest road to 100% OA. (If empirical -- or logical -- evidence were ever to come out showing the contrary, I assure you I too would join the gold rush!)

GE: " Sometimes our egos stand in the way of reaching a larger common goal, and I hope Harnad and other sceptics respond with good science rather than with polemics and politics to these findings."

Well, first, let us not get carried away: There's precious little science involved here (apart from the science we are trying to provide Open Access to). The call to self-archive in order to enhance access and impact is so obvious and trivial that, as I noted, the puzzle is only why anyone would even have imagined otherwise.

But when it comes to polemics and politics (and possibly also egos), it might have kept things more objective if the results of Eysenbach's small but welcome study confirming the OA impact advantage had not been hyped with editorial salvos such as:

"solid evidence to support or refute... that papers freely available in a journal will be more often read and cited than those behind a subscription barrier... has been surprisingly hard to find..."

Or even the heavily-hedged:

"As far as we are aware, no other study has compared OA and non-OA articles from the same journal and controlled for so many potentially confounding factors."

GE: " Unfortunately, in this area a lot more people have strong opinions and beliefs than those having the skills, time, and willingness to do rigorous research. I hope we will change this, and I reiterate a "call for papers" in that area [http://www.jmir.org/2006/2/e8/]"

May I echo that call, adding only that the rigorous research might perhaps be better placed in a journal specializing in scientometrics and in rigorously peer-reviewing it, rather than in The Journal of Medical Internet Research, or even PLoS Biology.

Brody, T., Harnad, S. and Carr, L. (2005) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 56.

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47.

I close with some replies to portions of another version of Eysenbach's response which appeared in his blog.

GE: " Harnad's point that the PLoS paper is about the "citation advantage of open access" and that there have been "previous papers about the citation advantage of open access" (mostly his own studies, mostly not published in peer-reviewed journals) is as meaningful as saying "this paper is about a cancer treatment, and there are previous papers about cancer treatments, so this one doesn't add anything"."

That's not what I said. I said this:

"[T]he only new knowledge from this small, journal-specific sample was (1) the welcome finding of how early the OA advantage can manifest itself, plus (2) some less clear findings about differences between first- and last-author OA practices, plus (3) a controversial finding that will most definitely need to be replicated on far larger samples in order to be credible: "The analysis revealed that self-archived articles are also cited less often than OA [sic] articles from the same journal."

And I do think all of this is as far away from rigorous oncological research as it is from rocket science!

GE: " The statement made by the reviewers and editors of the PLoS paper that this is the first study looking at the citation advantage of an open access/hybrid journal remains correct until somebody can show me a reference where this has been done before."

But who ever contested that far more modest and circumspect statement (which was certainly not the one the accompanying PLoS editorial made)? This is indeed "the first study looking at the citation advantage of an open access/hybrid journal"; indeed, it's the first such study of PNAS. But it's certainly not the first study looking at the citation advantage of OA in general, or OA self-archiving in particular, and looking at it within journals -- within many journals, and many articles.

GE: " In analogy, a small carefully designed cohort study showing a relationship between smoking and cancer with 1500 patients, obtaining through questionnaires and interviews additional variables which could account for the association and controlling for these confounders and still coming to the conclusion that there is a relation between smoking and cancer is scientifically stronger evidence than a quick-and-dirty uncontrolled cross-sectional study showing an association between smoking and cancer, even if this is done in a population of millions."

Indeed it would. And I forgot to add to my list (4) that Eysenbach had tested the hypothesis that the OA citation advantage is merely the result of a self-selection bias by asking 247 authors whether it was, and they replied that it wasn't...

Stevan Harnad
American Scientist Open Access Forum

---

Update Jan 1, 2010: See Gargouri, Y; C Hajjem, V Larivière, Y Gingras, L Carr,T Brody & S Harnad (2010) “Open Access, Whether Self-Selected or Mandated, Increases Citation Impact, Especially for Higher Quality Research”
Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

PLoS, Pipe-Dreams and Peccadillos

(Shorter version of this comment appears as a letter in PLoS Biology)

ABSTRACT: Eysenbach's (2006) study in PloS Biology on 1492 articles published during one 6-month period in one journal (PNAS) found that the Open Access (OA) articles were more cited than the non-OA ones. The online bibliography on the OA citation advantage records a number of prior within-journal comparisons that found exactly the same effect: freely available articles are read and cited more. Eysenbach?s further finding that the OA advantage (in this particular 6-month, 3-option, 1-journal PloS/PNAS study) is greater for articles that have paid for OA publication than for those that have merely been self-archived will require replication on much larger samples as most of the prior evidence for the OA advantage comes from self-archived articles and is based on sample sizes four orders of magnitude larger for both the number of articles and the number of journals tested.

I applaud and welcome the results of the Eysenbach (2006) study on 1492 articles published during one 6-month period in one journal (PNAS), showing that the Open Access (OA) articles were more cited than the non-OA ones. I also agree fully that the findings are unlikely to have been an artifact of PLoS’s “strong and vested interest in publishing results that so obviously endorse our existence,” nor of the fact that “the author of the article is also an editor of an open-access journal” (all quotes are from the PloS Biology editorial by MacCallum & Parthasarthy, 2006).

However, I am less sure that PloS’s and the author’s vested interests are not behind statements (in both the accompanying editorial and the article itself) along the lines that: “solid evidence to support or refute… that papers freely available in a journal will be more often read and cited than those behind a subscription barrier… has been surprisingly hard to find.” The online bibliography on ‘The effect of open access and downloads ('hits') on citation impact’ records a growing number of studies reporting precisely such evidence as of 2001, including studies based on data from much larger samples of journals, disciplines and years than the PloS study on PNAS– and they all find exactly the same effect: freely available articles are read and cited more.

There can be disagreement about what evidence one counts as “solid,” but there can be little dispute that prior evidence derived from substantially larger and broader-based samples showing substantially the same outcome can hardly be described as “surprisingly hard to find.”

In fact, the only new knowledge from this small, journal-specific sample was (1) the welcome finding of how early the OA advantage can manifest itself, plus (2) some less clear findings about differences between first- and last-author OA practices, plus (3) a controversial finding that will most definitely need to be replicated on far larger samples in order to be credible: “The analysis revealed that self-archived articles are also cited less often than OA [sic] articles from the same journal.”

The latter (3) is a within-journal (one journal, PNAS) finding; the overwhelming majority of articles made OA (sic) through author self-archiving today (on which the prior large-sample OA citation advantage findings are based) do not appear in journals with a paid-OA option. Hence on the present evidence I have great difficulty in seeing this secondary advantage as any more than a paid-OA publisher’s pipe-dream at this point.

The following, however, is not a pipe-dream, but a peccadillo: “no other study has compared OA and non-OA articles from the same journal.” To be fair, this observation is hedged with “[a]s far as we are aware” (but the OA-advantage bibliography is surely public knowledge – or should be among advocates of public access to science) and the observation is further qualified with: “and [also] controlled for so many potentially confounding factors.”

But it has to be stated that of these “potentially confounding” variables -- “number of days since publication, number of authors, article type, country of the corresponding author, funding type, subject area, submission track (PNAS has three different ways that authors can submit a paper)… previous citation record of the first and last authors… [and] whether authors choosing the OA option in PNAS chose to do so for only their most important research (they didn't)” – many are peculiar to this particular short-interval, 3-option, single-journal PloS study. And several of them (country, subject, year) had already been analyzed in papers that had been published before this 2006 article and were not taken into account despite the fact that both their preprints and their postprints had been freely accessible since well before publication, and that at least one of them (Brody et al. 2005) had been explicitly drawn to the author’s attention based on a preprint draft well before the article was submitted to PloS.

Brody et al. (2005) had found that, alongside the OA citation advantage, more downloads in the first six months after publication are correlated with more citations 18 months later in physics; and Hajjem et al. (2005) had found higher citations for OA articles -- comparing always within the very same journal and year -- for 1,307,038 articles published across 12 years (1992-2003) in 10 disciplines (Biology, Psychology, Sociology, Health, Political Science, Economics, Education, Law, Business, Management).

REFERENCES

Brody, T., Harnad, S. and Carr, L. (2005) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 56.

Eysenbach, G, (2006) Citation Advantage of Open Access Articles. PLoS Biology 4(5).

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47.

MacCallum, C.J., and Parthasarathy, H. (2006) Open Access Increases Citation Rate. PLoS Biology 4(5).


Stevan Harnad
American Scientist Open Access Forum

SH: Here is a quick summary of points of agreement and disagreement with the University of California (UC) view of Open Access (OA) and Institutional Repositories (IRs) as described by Catherine Candee (CC) in her interview with Richard Poynder (RP) in Changing the paradigm:

(1) UC considers publication reform to be the goal and OA merely a means: I would consider OA to be the goal and publication reform merely a hypothetical possibility that might or might not follow from OA.

(2) UC considers providing OA to postprints (i.e., final drafts of published journal articles) a lesser priority for IRs; I think they are the first priority.

(3) UC moved away from Eprints and postprint self-archiving because of the extremely low level of spontaneous uptake by UC faculty, assuming the low uptake was because it was "too difficult." It is far more likely that the low uptake was because UC did not adopt an institutional self-archiving mandate. Those institutions that have done so have dramatically higher self-archiving rates.

(4) UC instead outsourced self-archiving to an expensive service that, being a secondary publisher, needs to expend a lot of resources on following up rights problems for each published paper; the result so far is that UC's eScholarship IR is still not self-archiving more than the c. 15% worldwide self-archiving baseline for postprints.

(5) The other reason UC moved away from Eprints and postprint archiving is because of its publishing reform goals, including university self-publishing (of journals and monographs). I think monographs are (for the time being) a separate matter, and should be handled separately from journal article OA, and that peer review needs to be implemented by a neutral 3rd party, not the author or the author's institution. The immediate priority is postprint OA.

In summary, UC seems to be giving its own hypothetical conjectures on the future of scholarly publishing -- and its own aspirations for the hypothetical new publishing system -- priority over an immediate, pressing, and remediable practical problem: the needless, daily loss of 25% - 250% or more of the usage and impact of 85% of UC research output. Because researchers are relatively uninformed and uninvolved in all this, they do not have a clear sense of the implicit trade-off between (a) the actual daily, cumulative usage/impact loss for their own research output, with its tested and demonstrated remedy, and (b) the untested hypothetical possibilities with which some in the UC library community (and elsewhere) seem to be preoccupied.

---

[Note: all hyperlinks have been added: they were not in the original RP interview]

RP: Initially you built the eScholarship Repository with the EPrints software, which was developed at Southampton University in the UK?

CC: Right. We started with Eprints, and the aim was to create what people now call an institutional repository — a repository where faculty could put materials (text and images) that they wanted to disseminate, or actually publish.

RP: You later switched to the bepress software. Why?

CC: We found it so, so, so difficult to get faculty even to test the EPrints software that we abandoned the idea of providing a platform for faculty to individually publish [sic, emphasis added] their own works.

SH: I think here is where the strategic error occurred. Not in switching softwares (since the software makes absolutely no difference) but in abandoning the goal of 100% OA for UC postprint output. The reason is implicit in the words CC uses to describe it: The self-archiving of already published postprints is not publishing at all, but merely OA-provision -- except if the underlying goal is not OA, but self-publishing!

CC: Around the same time we serendipitously encountered the bepress software, and right away we could see that it would allow us to do something much more important. We could see that if we used the bepress software the repository could also support peer-reviewed publications. Consequently, by the time we launched we had switched to a different model, and we had adopted the bepress software.

SH: Again, it is hypotheses about publishing reform and aspirations for UC self-publishing that motivated the change of "model." (Model for what, one wonders? OA is not a model. It just a means of making journal articles free for all online. It is publishing reform that involves models. Better if UC had done the tested, demonstrated part first, by adopting an institutional self-archiving policy, as at least four other universities have since done, successfully, and once the doable part was successfully done, moved on to the hypothetical part...)

RP: How was the model different?

CC: The bepress software allowed different units within the University of California to become the gatekeepers, with all the editorial and administrative ability resting with an academic department, an institute, or a research unit, rather than with individual faculty, or with the library.

SH: There are two issues here: (1) Did the Eprints software allow departments or research units to be their own gate-keepers for self-archiving? Of course it did, either within one Eprints installation, or, optionally, across many, thanks to OAI interoperability. But much more important: (2) Is local gate-keeping the goal of UC researchers? Has the gate-keeping not already been done by the peer-reviewers for the journal in which it was published? It looks here as if, once again, the hypotheses about publishing reform and UC self-publishing are driving the agenda, not researchers' immediate needs (which are to maximize research access, usage and impact, via OA).

RP: So where EPrints software assumed that researchers would do the inputting of papers themselves, bepress software was more suited to third-parties depositing them?

CC: That is one difference — although, because the software is difficult to use, Eprints submissions are often managed centrally.

SH: As this is not about defending the Eprints software in particular, I only note in passing that the difficulty was not the software but the fact that UC researchers were not required to self-archive, and hence didn't. In institutions where self-archiving is required, it is done, easily, by researchers themselves, not centrally. The central proxy self-archiving is a start-up strategy, used successfully by some institutions to set the practice firmly into motion; it is not a feature of the software:

Carr, L. and Harnad, S. (2005) Keystroke Economy: A Study of the Time and Effort Involved in Self-Archiving.

CC: Additionally, the bepress software lent itself to the size of UC; and it allowed the University to decide exactly what it wanted to put in, and to brand everything in the way it wished.

SH: All the free softwares are likewise configurable in exactly the same way.

RP: You were also able to outsource the hosting of the eScholarship Repository to bepress?

CC: Yes. It is hosted by bepress, but preserved here at CDL. What we are doing is harvesting citations. We then send them to faculty members saying that the listed works may be eligible for inclusion in the eScholarship repository. It is a way to alert them to the repository, and to the fact that they have content that could be placed in it.The message sent to faculty is clickable, and when they click on the link it brings them directly into the repository, where the citation data for the paper automatically fills out the repository metadata fields for them. This, by the way, is the one case where we allow authors to put their content in directly themselves. However, we also allow them to use a proxy — so they can legally assign someone else to put their papers in for them. The aim is to make the process as easy as possible, because time is the biggest constraint when it comes to getting faculty to participate.

SH: So far, this is all excellent practice, and an ingenious start-up strategy (though only if coupled with an institutional self-archiving requirement). But it is the next step that defeats it:

RP: And you have contracted bepress to do rights clearance on the papers?

CC: Right. After the papers are submitted we pay bepress to check the rights on them. That was a concession to the fact that bepress' business would be threatened if they got sued for allowing something illegal to be put into the repository. This part of the process is both onerous and expensive, and we hope we will not need to do it at some point in the future.

SH: So because UC have gotten into a 3rd-party publisher situation, they face rights problems they would not face if it were all in-house UC self-archiving. They are also incurring considerable additional expense needlessly (and at a time when institutions are being deterred from IRs and OA out of the false impression that it is expensive). Worst of all, so far the result is still not more UC postprints becoming OA than the global 15% average:

RP: I'm told you have acquired about 1,000 papers in this way... 1,000 postprints is a small drop in the ocean I guess. How many researchers are there within the UC system?

CC: UC is the largest public research university I know of. It has ten campuses and around 16,000 faculty and researchers.

RP: When you ask faculty for a postprint is it a request or a demand?

CC: It is not a demand. Clearly, incentive is the single biggest issue for getting content in. Awareness is another issue, so we are just starting some market research to discover what percentage of UC faculty even know about the repository. I suspect it is less than half.

SH: Perhaps a UC self-archiving requirement would be worth considering after all, since several international surveys have now reported that 70% - 95% of faculty say they would comply with a self-archiving requirement, and the 4 institutions that have adopted such a policy so far confirm that it works.

Swan, A. and Brown, S. (2005) Open access self-archiving: An author study. JISC Technical Report, Key Perspectives Inc.

RP: So you still have work to do in publicising the repository?

CC: We do. While we are very excited that we have more than 200 departments participating in the repository we have no idea what percentage of the faculty know about it; and we have no idea what percentage would participate if they did know — because there is no overriding incentive for them to do so today. We need to understand the situation.

SH: The missing element is the institutional requirement to deposit the final accepted, peer-reviewed draft (not the publisher's PDF) as an institutional record-keeping matter: a fulfillment condition for annual review, for research assessment, and for standard CV creation/submission.

RP: As your experience shows, creating a repository is only half the task. You then have to fill it. For that reason there are growing calls for funders to mandate researchers to self-archive their papers. Do you think that that is the best way of filling institutional repositories?

CC: Well, I wouldn’t say that our purpose is simply to fill institutional repositories. We built an institutional repository as one way of providing an alternative to the current publishing system, and to give faculty something to do with that copyrighted material that we keep saying shouldn’t be given away to publishers.

SH: Filling an OA IR with the institution's annual research article output may not be the only possible goal for an IR, but it is surely the most important priority at this moment for researchers, whose need is not for an alternative to the current publishing system but for OA and the enhanced research impact it brings. Copyright retention is not an end in itself for researchers either: OA is. And with OA, copyright retention becomes moot.

CC: It may turn out that institutional repositories aren’t the way to go however. For that reason we are also interested in encouraging faculty to manage their copyrights differently, and to consider who they give their manuscripts to, and where they commit their editing and reviewing time. So our main focus is in accomplishing that, rather than filling repositories.

SH: Why all this when, in and of itself, this is not what faculty want and need? It would be fine if copyright retention were an essential means to an end that faculty do want and need, but it is not. OA is an end in itself, and it does not require copyright retention when 93% of journals have already given OA author self-archiving their green light:

RP: Do you nevertheless anticipate that funders will eventually introduce mandates?

CC: Actually we expect that universities will make some sort of a mandate before funding agencies do. In this regard there are a number of white papers floating around the University of California right now. We are waiting to see what happens to those.

SH: But the UC proposal is for copyright retention, whereas what is needed is a self-archiving requirement. Copyright retention requires needless re-negotiation with the 93% of journals that have already endorsed OA self-archiving, and it puts 100% of authors at risk of an unsuccessful re-negotiation, instead of just requiring that 100% of them deposit, leaving the 7% to set access as restricted access instead of OA, pending negotiations, if they wish.

RP: ...Given what you say about rights, I 'd be interested to hear more about the Scholarly Work Copyright Rights Policy white paper. This proposes that UC faculty "routinely grant to The Regents of the University of California a limited, irrevocable, perpetual, worldwide, non-exclusive licence to place the faculty member's scholarly work in a non-commercial open-access online repository." Would this apply only to journal articles or all the works of faculty, including books?

CC: Ultimately it is intended to apply to all works, but starting with journal articles.

SH: Does it make sense to hold back (and weigh down) the sure research benefits of the self-archiving of published journal articles (postprints) for the much vaguer and more controversial case of books?

RP: If it does go ahead would you envisage a postprint mandate following behind it?

CC: Yes.

SH: A postprint mandate should not come behind a copyright blanket retention mandate! That is like making a local emission-reduction plan's adoption contigent on first getting all nations to agree to sign the Kyoto Accord!

RP: And you would welcome that?

CC: I would. While I don’t find the postprint issue as interesting or exciting as trying to encourage new forms of communication, it is strategically important — because it would allow us to put in place a production-level service capable of managing UC copyrighted material, which would better prepare us for the future.

SH: Then why not adopt a posprint self-archiving mandate immediately, instead of waiting for agreement on the much more demanding and controversial copyright-retention policy?

CC: ... eScholarship Editions are scholarly monographs encoded in XML. ...As you know, the corollary to the serials crisis is that libraries have less money to buy monographs, and so fewer monographs are being published. The fact is, however, that an awful lot of monographs could be published if the UC Press had more editorial bandwidth. So we have been experimenting with empowering UC Press editorials boards, or faculty editorial boards, to become, essentially, publishers. In this way we can extend the work of UC Press.

SH: This is the UC self-publishing agenda, and it is fine, but why is it being coupled with the OA IR issue? and worse, why is it being allowed to hold OA back? The (1) UC authors who publish their articles in established peer-reviewed journals may often be the same individuals as the (2) UC authors of monographs, but their situations are very different. The article authors already have publishers (not UC!) and need only OA. The monograph authors may or may not have a publisher, which may or may not be UC, and they may or may not want OA. Why should the straightforward solution for (1) be constrained by the much less straightforward solution for (2)?

RP: It's clear you have a very broad view of the role of an institutional repository. Advocates of self-archiving, by contrast, insist that an institutional repository should only ever be viewed as a postprint archive. What's your response to that view?

CC: I think it is unfortunate that the term institutional repository has come to mean something narrower. As I say, the postprint component is the least interesting and ultimately the least important part of this. So while right now it is tactically extremely important to deposit postprints, ultimately I envision a very different arrangement between universities and publishers than we have now.

SH: The reasoning here is unclear: Postprint OA is clearly the heart of the OA movement, and an end in itself (even if there are further ends thereafter). CC agrees that "right now it is tactically extremely important to deposit postprints." Yet UC is not doing what needs to be done to achieve that "narrower" immediate goal. It is instead aiming at the "wider" hypothetical one, and the result is that only 1000 of the "extremely important" postprints have been deposited in the UC IR to date, five years after the IR was created, while white papers are being written about copyright retention, publishing reform, and UC self-publishing plans. If the narrower postprint target is indeed an important prerequisite for the rest, then why not make a concerted effort to reach it first and leave the more hypothetical phase for afterward? (Or at least do them in parallel.)

RP: You believe universities should be in control of the publishing process, rather than managing papers that have been published by someone else?

CC: That's right. Eventually I hope all the content will be hosted and managed by universities themselves, and the publishing services would be in the form of added value. So, for instance, a published article would refer back to the raw article in the repository.

SH: This is all fine, but completely speculative. The course that will be taken by journal publishing and monograph publishing, whether published by universities or published by others, is right now a matter of pure speculation, whereas the course that is taken in access-provision for a university's own postprint output is a practical matter, entirely in the hands of the university and its researchers. Why is immediate OA to postprints being held hostage to hypotheses about eventual publishing reform?

RP: What worries self-archiving advocates about this is that if universities try to make institutional repositories too broad in functionality they could delay the transition to an open access environment; that we need to stay focused on the narrower view until OA is achieved. You are arguing that we need to plan for the longer-term future from day one are you?

CC: I think so. Moreover, I don’t see why a broader view would slow OA down. It is a matter of getting the right platform and getting things moving so that faculty can see that there are other things that can be done.

SH: But we have a clear example of "why a broader view would slow OA down"! In 2001, UC adopted Eprints and waited to see whether its IR would fill spontaneously. It did not. So instead of adopting a self-archiving policy (as Southampton, QUT, Minho, and CERN have since done, successfully filling their archives -- Eprints, Eprints, Dspace, and CDSware, respectively), UC adopted another software -- and another agenda instead of OA: publishing reform, copyright retention, and university self-publishing.

RP: I wonder if we might see increasing tension between researchers and librarians over the issue of institutional repositories? I ask because the primary aim of researchers is to achieve maximum impact for their research; librarians, by contrast, are looking to create large digital libraries or even, as in the case of UC, complete publishing systems. Could this threaten the historic relationship between librarians and researchers?

CC: I can see such a tension theoretically: where resources were limited, for instance, the aim of building a digital library could seem to stand in the way of getting publishing out quickly. But ultimately I think you are presenting a false dichotomy.

SH: The only tension is about lost time. UC, the world's biggest university system, 5 years down the line after establishing one of the first IRs, has 10285 items therein, 1000 of them postprints of UC published journal articles. Meanwhile, tiny Minho has 3297 items, QUT has 2194, Southampton 7745 plus 9795 for its ECS department alone, and CERN, larger but nowhere near UC in size, has 73898 items. Assuming that (as reported by Southampton and CERN) 40-70% of these, at least, are postprints, it looks very much as if an institutional postprint self-archiving mandate has served these other institutions well. Particularly instructive is CERN: Now that it is firmly on the road to 100% postprint OA for its own vast annual output -- and only now -- CERN is turning to the question of publishing reform. If all other universities and research institutions (including the biggest, UC) were to do likewise (in that order!), we would already be there (at 100% OA) and in a far better position to contemplate the hypothetical horizons of ensuing publication reform.

Stevan Harnad

The second of an occasional column series called "Skywritings:
Scholarly and Leisurely" has appeared on Haworth Press's Website:

Harnad, S. (2005) The Green and Gold Roads to Maximizing Journal Article Access, Usage and Impact Haworth Press, July 1, 2005
http://www.haworthpress.com/library/StevanHarnad/07012005.asp

AmSci Ref.

On Thu, 14 Jul 2005, Anthony Watkinson wrote (in liblicense):

AW: "The quotation from Raym Crow (whose work incidentally I admire) needs to be taken in the context of his model in the same piece. To repeat - this disaggregated model leaves almost no role for publishers..."

Raym Crow's 2002 SPARC Position paper "The Case for Institutional Repositories" lost a lot of its potential usefulness because it made far too much of a completely untested (and, I suspect, ultimately incoherent) speculation (from J.W.T. Smith) about "Disaggregated Journals."

It is a great pity that a concrete, tested, and proven practical means of maximizing research usage and impact -- namely, authors self-archiving their published (traditional) journal articles in their own institutional repositories (aka archives) -- was conflated with a mere piece of speculation in what should have been an authoritative SPARC document.

Here is my original critique of Crow's paper (which I was, alas, persuaded not to post publicly in the American Scientist Open Access Forum at the time (2002), ostensibly on the grounds of maintaining solidarity among OA advocates; but that was a mistake -- it's always a mistake to remain mute about a flawed idea, even among allies). I archived it (without ever actually posting it) only 2 years later, in 2004, too late.

J.W.T. Smith's "Disaggregated Journal" idea had already been discussed extensively much earlier in the American Scientist OA Forum (then called the September-Forum) in 1999. The idea has been neither tested nor patched up since.

(So much of the slow history of OA seems to consist in recycling speculations and fallacies instead of moving ahead and doing what has already been demonstrated to be doable, and effective. Here we are in 2005, rediscovering the "Disaggregated Journal" -- and still not providing the OA that has been within reach for at least a decade and a half. -- I'm sure the pundits will now chime in with their wise saws about why it had be so...)


Your humble but impatient archivangelist,

Stevan Harnad