Hopeful Ad Hoc Critiques of OA Study After OA Study: Will Wishful Thinking Ever Wane?

Comment on Elsevier Editors' Update by Henk Moed:
"Does Open Access publishing increase citation rates? Studies conducted in this area have not yet adequately controlled for various kinds of sampling bias."

No study based on sampling and statistical significance-testing has the force of an unassailable mathematical proof.

But how many studies showing that OA articles are downloaded and cited more have to be published before the ad hoc critiques (many funded and promoted by an industry not altogether disinterested in the outcome!) and the special pleading tire of the chase?

There are a lot more studies to try to explain away here.

Most of them just keep finding the same thing...

(By the way, on another stubborn truth that keeps bouncing back despite untiring efforts to say it isn't so: Not only is OA research indeed downloaded and cited more -- as common sense would expect, since it accessible free for all, rather than just to those whose institutions can afford a subscription -- but requiring (mandating) OA self-archiving does indeed increase OA self-archiving. Where on earth did Henk get the idea that some institutions' self-archiving "did not increase when their OA regime was transformed from non-mandatory into mandatory"? Or is Henk just referring to the "mandates" that state that "You must self-archive -- but only if and when your publisher says you may, and not if your publisher says 'you may if you may but you may not if you must'"...? Incredulous? See here and weep (for the credulous -- or chuckle for the sensible)...)

---

My friend Henk Moed (whose work I admire and whose scientific integrity I am in no way calling into question!) has replied to my query:

"Where on earth did Henk get the idea that some institutions' self-archiving 'did not increase when their OA regime was transformed from non-mandatory into mandatory'?"

Henk wrote to tell me that he got the idea from our own paper! (Gargouri et al 2010, Figure 1)

The figure shows the self-archiving rates from 2002-2006 for four mandated repositories, compared to the unmandated baseline self-archiving rate of about 20% per year. The four mandated repositories all have a self-archiving rate of about 60% for each of the six years.

Now where Henk got the idea that the mandates may not increase self-archiving was from the fact that the date on which the mandate was adopted differed for the four repositories, the earliest mandate being in 2002, the latest in 2004. So he inferred from the fact that the 2002-2006 rates were flat in all cases, that some, at least, of the mandates did not increase self-archiving.

There are two important details that Henk did not take into account:

(1) The date is the date the articles were published, not the date they were self-archived.

(2) When a mandate is adopted, the self-archiving is not just done for articles published on or after the mandate: it is also done retroactively, for articles published before the mandate, especially for recent years.

So the reason the self-archiving rates are flat is retroactive self-archiving. A clue is already there in Figure 1, because both the post-mandate self-archiving rates and the pre-mandate self-archiving rates are about three times the baseline (unmandated) rates (60% vs 20%).

(The baseline rate was derived from comparing the percentage of the articles that our robot found freely accessible on the web for the reference sample of articles in each of the publication years for the four mandated institutions with the percentage the robot found for articles published in the same journals and years, but from other institutions.)

The practice of retroactive self-archiving in the mandated repositories was confirmed in a later study that we will soon report, comparing the self-archiving rate for the same publishing years (from 2002 onward) as sampled by our robot several years later: The percentage for each year continued to grow years after adoption of the mandate.

One important thing to note, however, is that our estimate of the self-archiving rate for mandated institutions was actually an underestimate: We know the rates were higher than 60%, but we used the noisier and less reliable robot method rather than counting what was in the repository directly, in order to make the estimates comparable with the robot's estimate for the unmandated self-archiving rate. (The unmandated papers were not even necessarily self-archived in the author's instituitional repository: many were on their authors' personal or lab websites.)