Scientometrics

In announcing Alma Swan's Review of Studies on the Open Access Impact Advantage, I had suggested that the growing number of studies on the OA Impact Advantage were clearly ripe for a meta-analysis. Here is an update:

---

David Wilson wrote:

"Interesting discussion. Phil Davis has a limited albeit common view of meta-analysis. Within medicine, meta-analysis is generally applied to a small set of highly homogeneous studies. As such, the focus is on the overall or pooled effect with only a secondary focus on variability in effects. Within the social sciences, there is a strong tradition of meta-analyzing fairly heterogeneous sets of studies. The focus is clearly not on the overall effect, which would be rather meaningless, but rather on the variability in effect and the study characteristics, both methodological and substantive, that explain that variability.

"I don't know enough about this area to ascertain the credibility of [Phil Davis's] criticism of the methodologies of the various studies involved. However, the one study that [Phil Davis] claims is methodologically superior in terms of internal validity (which it might be) is clearly deficient in statistical power. As such, it provides only a weak test. Recall, that a statistically nonsignificant finding is a weak finding -- a failure to reject the null and not acceptance of the null.

"Meta-analysis could be put to good use in this area. It won't resolve the issue of whether the studies that Davis thinks are flawed are in fact flawed. It could explore the consistency in effect across these studies and whether the effect varies by the method used. Both would add to the debate on this issue."

Lipsey, MW & Wilson DB (2001) Practical Meta-Analysis. Sage.

David B. Wilson, Ph.D.
Associate Professor
Chair, Administration of Justice Department
George Mason University
10900 University Boulevard, MS 4F4
Manassas, VA 20110-2203

---

Added Mar 15 2010
See also (thanks to Peter Suber for spotting this study!):

Wagner, A. Ben (2010) Open Access Citation Advantage: An Annotated Bibliography. Issues in Science and Technology Librarianship. 60. Winter 2010

---

On Mar 12, 2010 Gene V Glass wrote the following:

"Far more issues about OA and meta analysis have been raised in this thread for me to [be able to] comment on. But having dedicated 35 years of my efforts to meta analysis and 20 to OA, I can’t resist a couple of quick observations.

Holding up one set of methods (be they RCT or whatever) as the gold standard is inconsistent with decades of empirical work in meta analysis that shows that “perfect studies” and “less than perfect studies” seldom show important differences in results. If the question at hand concerns experimental intervention, then random assignment to groups may well be inferior as a matching technique to even an ex post facto matching of groups. Randomization is not the royal road to equivalence of groups; it’s the road to probability statements about differences.

Claims about the superiority of certain methods are empirical claims. They are not a priori dicta about what evidence can and can not be looked at."

Glass, G.V.; McGaw, B.; & Smith, M.L. (1981). Meta-analysis in Social Research. Beverly Hills, CA: SAGE.

Rudner, Lawrence, Gene V Glass, David L. Evartt, and Patrick J. Emery (2000). A user's guide to the meta-analysis of research studies. ERIC Clearinghouse on Assessment and Evaluation, University of Maryland, College Park.

GVG's Publications

The following is a (belated) critique of:

"Impact Assesment," by Paul Chrisp (publisher, Core Medical Publishing) & Kevin Toale (Dove Medical Press). Pharmaceutical Marketing September 2008

---

"Open access has emerged in the last few years as a serious alternative to traditional commercial publishing models, taking the benefits afforded by technology one step further. In this model, authors are charged for publishing services, and readers can access, download, print and distribute papers free at the point of use."

Incorrect.

Open Access (OA) means free online access and OA Publishing ("Gold OA") is just one of the two ways to provide OA (and not the fastest, cheapest or surest):

The fastest, cheapest and surest way to provide OA is OA Self-Archiving (of articles published in conventional non-OA journals: "Green OA") in the author's Institutional Repository.

"Although its ultimate goal is the free availability of information online, open access is not the same as free access – publishing services still cost money."

Incorrect.

There are two forms of OA: (1) Gratis OA (free online access) and (2) Libre OA (free online access plus certain re-user rights)

"Other characteristics of open access journals are that authors retain copyright and they must self-archive content in an independent repository."

Incorrect.

This again conflates Green and Gold OA:

Gold OA journals make their own articles free online.

In Green OA, articles self-archive their articles.

"researchers are depositing results in databases rather than publishing them in journal articles"

Incorrect.

This conflates unrefereed preprint self-archiving with refereed, published postprint self-archiving.

Green OA is the self-archiving of refereed, published postprints.

The self-archiving of unrefereed preprints is an optional supplement to, not a substitute for, postprint OA.

"a manuscript may be read more times than it is cited, and research shows that online hits per article do not correlate with IF".

Incorrect.

"Research shows" that online hits (downloads) do correlate with citations (and hence with citation impact factors).

See references cited below.

"Faculty of 1000 (www.f1000medicine.com)... asks opinion leaders in clinical practice and research to select the most influential articles in 18 medical specialties. Articles are evaluated and ranked..."

Expert rankings are rankings and metrics (such as hit or citation counts) are metrics.

Metrics can and should be tested and validated against expert rankings. Validated metrics can then be used as supplements to -- or even substitutes for -- rankings. But the validation has to be done a much broader and more systematic basis than Faculty of 1000, and on a much richer set of candidate metrics.

Nor is the purpose of metrics "pharmaceutical marketing": It is to monitor, predict, navigate, analyze and reward research influence and importance.

Bollen, J., Van de Sompel, H., Hagberg, A. and Chute, R. (2009) A principal component analysis of 39 scientific impact measures in PLoS ONE 4(6): e6022,

Brody, T., Harnad, S. and Carr, L. (2006) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 57(8) 1060-1072.

Harnad, S. (2008) Validating Research Performance Metrics Against Peer Rankings . Ethics in Science and Environmental Politics 8 (11) doi:10.3354/esep00088 The Use And Misuse Of Bibliometric Indices In Evaluating Scholarly Performance

Harnad, S. (2009) Open Access Scientometrics and the UK Research Assessment Exercise. Scientometrics 79 (1) Also in Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics 11(1), pp. 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds. (2007)

Lokker, C., McKibbon, K. A., McKinlay, R.J., Wilczynski, N. L. and Haynes, R. B. (2008) Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study BMJ, 2008;336:655-657

Moed, H. F. (2005) Statistical Relationships Between Downloads and Citations at the Level of Individual Documents Within a Single Journal. Journal of the American Society for Information Science and Technology 56(10): 1088- 1097

O'Leary, D. E. (2008) The relationship between citations and number of downloads Decision Support Systems 45(4): 972-980

Watson, A. B. (2009) Comparing citations and downloads for individual articles Journal of Vision 9(4): 1-4

---

Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research

Authors: Yassine Gargouri, Chawki Hajjem, Vincent Larivière, Yves Gingras, Les Carr, Tim Brody, Stevan Harnad
Click here for preprint
Abstract: Articles whose authors make them Open Access (OA) by self-archiving them online are cited significantly more than articles accessible only to subscribers. Some have suggested that this "OA Advantage" may not be causal but just a self-selection bias, because authors preferentially make higher-quality articles OA. To test this we compared self-selective self-archiving with mandatory self-archiving for a sample of 27,197 articles published 2002-2006 in 1,984 journals. The OA Advantage proved just as high for both. Logistic regression showed that the advantage is independent of other correlates of citations (article age; journal impact factor; number of co-authors, references or pages; field; article type; or country) and greatest for the most highly cited articles. The OA Advantage is real, independent and causal, but skewed. Its size is indeed correlated with quality, just as citations themselves are (the top 20% of articles receive about 80% of all citations). The advantage is greater for the more citeable articles, not because of a quality bias from authors self-selecting what to make OA, but because of a quality advantage, from users self-selecting what to use and cite, freed by OA from the constraints of selective accessibility to subscribers only.

The Times Higher Education Supplement (THES) has reported the results of a study they commissioned by Evidence Ltd that found that the ranking criteria for assessing and rewarding research performance in the UK Research Assessment Exercise (RAE) changed from RAE 2001 to RAE 2008. The result is that citations, which correlated highly with RAE 2001, correlated less highly with RAE 2008, so a number of universities whose citation counts had decreased were rewarded more in 2008, and a number of universities whose citation counts had increased were rewarded less.

(1) Citation counts are only one (though an important one) among many potential metrics of research performance.

(2) If the RAE peer panel raters' criteria for ranking the universities varied or were inconsistent between RAE 2001 and RAE 2008 then that is a problem with peer ratings rather than with metrics (which, being objective, remain consistent).

(3) Despite the variability and inconsistency, peer ratings are the only way to initialise the weights on metrics: Metrics first have to be jointly validated against expert peer evaluation by measuring their correlation with the peer rankings, discipline by discipline; then the metrics' respective weights can be updated and fine-tuned, discipline by discipline, in conjunction with expert judgment of the resulting rankings and continuing research activity.

(4) If only one metric (e.g., citation) is used, there is the risk that expert ratings will simply echo it. But if a rich and diverse battery of multiple metrics is jointly validated and initialized against the RAE 2008 expert ratings, then this will create an assessment-assistant tool whose initial weights can be calibrated and used in an exploratory way to generate different rankings, to be compared by the peer panels with previous rankings as well as with new, evolving criteria of research productivity, uptake, importance, influence, excellence and impact.

(5) The dawning era of Open Access (free web access) to peer-reviewed research is providing a wealth of new metrics to be included, tested and assigned initial weights in the joint battery of metrics. These include download counts, citation and download growth and decay rates, hub and authority scores, interdisciplinarity scores, co-citations, tag counts, comment counts, link counts, data-usage, and many other openly accessible and measurable properties of the growth of knowledge in our evolving "Cognitive Commons."

Brody, T., Kampa, S., Harnad, S., Carr, L. and Hitchcock, S. (2003) Digitometric Services for Open Archives Environments. In Proceedings of European Conference on Digital Libraries 2003, pp. 207-220, Trondheim, Norway.

Brody, T., Carr, L., Harnad, S. and Swan, A. (2007) Time to Convert to Metrics. Research Fortnight pp. 17-18.

Brody, T., Carr, L., Gingras, Y., Hajjem, C., Harnad, S. and Swan, A. (2007) Incentivizing the Open Access Research Web: Publication-Archiving, Data-Archiving and Scientometrics. CTWatch Quarterly 3(3).

Carr, L., Hitchcock, S., Oppenheim, C., McDonald, J. W., Champion, T. and Harnad, S. (2006) Extending journal-based research impact assessment to book-based disciplines. Technical Report, ECS, University of Southampton.

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47.

Harnad, S. (2001) Research access, impact and assessment. Times Higher Education Supplement 1487: p. 16.

Harnad, S. (2007) Open Access Scientometrics and the UK Research Assessment Exercise. In Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics 11(1), pp. 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds.

Harnad, S. (2008) Self-Archiving, Metrics and Mandates. Science Editor 31(2) 57-59

Harnad, S. (2008) Validating Research Performance Metrics Against Peer Rankings. Ethics in Science and Environmental Politics 8 (11) doi:10.3354/esep00088 The Use And Misuse Of Bibliometric Indices In Evaluating Scholarly Performance

Harnad, S. (2009) Multiple metrics required to measure research performance. Nature (Correspondence) 457 (785) (12 February 2009)

Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated online RAE CVs Linked to University Eprint Archives: Improving the UK Research Assessment Exercise whilst making it cheaper and easier. Ariadne 35.

Harnad, S., Carr, L. and Gingras, Y. (2008) Maximizing Research Progress Through Open Access Mandates and Metrics. Liinc em Revista.

---

Update Jan 1, 2010: See Gargouri, Y; C Hajjem, V Larivière, Y Gingras, L Carr,T Brody & S Harnad (2010) “Open Access, Whether Self-Selected or Mandated, Increases Citation Impact, Especially for Higher Quality Research”
Update Feb 8, 2010: See also "Open Access: Self-Selected, Mandated & Random; Answers & Questions"

---

---

SUMMARY: Evans & Reimer (2009) (E & R) show that a large portion of the increased citations generated by making articles freely accessible online ("Open Access," OA) comes from Developing-World authors citing OA articles more. It is very likely that a within-US comparison based on the same data would show much the same effect: making articles OA should increase citations from authors at the Have-Not universities (with the smaller journal subscription budgets) more than from Harvard authors. Articles by Developing World (and US Have-Not) authors should also be cited more if they are made OA, but the main beneficiaries of OA will be the best articles, wherever they are published. This raises the question of how many citations – and how much corresponding research uptake, usage, progress and impact – are lost when articles are embargoed for 6-12 months or longer by their publishers against being made OA by their authors.
    (It is important to note that E & R's results are not based on immediate OA but on free access after an embargo of up to a year or more. Theirs is not an estimate of the increase in citation impact that results from immediate Open Access; it is just the increase that results from ending Embargoed Access. In a fast-moving field of science, an access lag of a year can lose a lot of research impact, permanently.)

---

The portion of Evans & Reimer's (2009) study (E & R) is valid is timely and useful, showing that a large portion of the Open Access citation impact advantage comes from providing the developing world with access to the research produced by the developed world. Using a much bigger database, E & R refute (without citing!) a recent flawed study (Frandsen 2009) that reported that there was no such effect (as well as a premature response hailing it as "Open Access: No Benefit for Poor Scientists").

E & R found the following. (Their main finding is number #4):

#1 When articles are made commercially available online their citation impact becomes greater than when they were commercially available only as print-on-paper. (This is unsurprising, since online access means easier and broader access than just print-on-paper access.)

#2 When articles are made freely available online their citation impact becomes greater than when they were not freely available online. (This confirms the widely reported "Open Access" (OA) Advantage.) 

(E & R cite only a few other studies that have previously reported the OA advantage, stating that those were only in a few fields, or within just one journal. This is not correct; there have been many other studies that likewise reported the OA advantage, across nearly as many journals and fields as E & R sampled. E & R also seem to have misunderstood the role of prepublication preprints in those fields (mostly physics) that effectively already have post-publication OA. In those fields, all of the OA advantage comes from the year(s) before publication -- "the Early OA Advantage", which is relevant to the question, raised below, about the harmful effects of access embargoes. And last, E&R cite the few negative studies that have been published -- mostly the deeply flawed studies of Phil Davis -- that found no OA Advantage or even a negative effect (as if making papers freely available reduced their citations!). 

#3 The citation advantage of commercial online access over commercial print-only access is greater than the citation advantage of free access over commercial print plus online access only. (This too is unsurprising, but it is also somewhat misleading, because virtually all journals have commercial online access today: hence the added advantage of free online access is something that occurs over and above mere online (commercial) access -- not as some sort of competitor or alternative to it! The comparison today is toll-based online access vs. free online access.)

(There may be some confusion here between the size of the OA advantage for journals whose contents were made free online after a pospublication embargo period, versus those whose contents were made free online immediately upon publication -- i.e., the OA journals. Commercial online access is of course never embargoed: you get access as soon as its paid for! Previous studies have made within-journal comparisons, field by field, between OA and non-OA articles within the same journal and year. These studies found much bigger OA Advantages because they were comparing like with like and because they were based on a longer time-span: The OA advantage is still small after only a year, because it takes time for citations to build up; this is even truer if the article becomes "OA" only after it has been embargoed for a year or longer!)

#4 The OA Advantage is far bigger in the Developing World (i.e., Developing-World first-authors, when they cite OA compared to non-OA articles). This is the main finding of this article, and this is what refutes the Frandsen study.

What E & R have not yet done (and should!) is to check for the very same effect, but within the Developed World, by comparing the "Harvards vs. the Have-Nots" within, say the US: The ARL has a database showing the size of the journal holdings of most research university libraries in the US. Analogous to their comparison's between Developed and Developing countries, E & R could split the ARL holdings into 10 deciles, as they did with the wealth (GNI) of countries. I am almost certain this will show that a large portion of the OA impact advantage in the US comes from the US's "Have-Nots", compared to its Harvards.

The other question is the converse: The OA advantage for articles authored (rather than cited) by Developing World authors. OA does not just give the Developing World more access to the input it needs (mostly from the Developed World), as E & R showed; but OA also provides more impact for the Developing World's research output, by making it more widely accessible (to both the Developing and Developed world) -- something E & R have not yet looked at either, though they have the data! Because of what Seglen (1992) called the "skewness of science," however, the biggest beneficiaries of OA will of course be the best articles, wherever their authors: 90% of citations go to the top 10% of articles.

Last, there is the crucial question of the effect of access embargoes. It is essential to note that E & R's results are not based on immediate OA but on free access after an embargo of up to a year or more. Theirs is hence not an estimate of the increase in citation impact that results from immediate Open Access; it is just the increase that results from ending Embargoed Access.

It will be important to compare the effect of OA on embargoed versus unembargoed content, and to look at the size of the OA Advantage after an interval of longer than just a year. (Although early access is crucial in some fields, citations are not instantaneous: it may take a few years' work to generate the cumulative citation impact of that early access. But it is also true in some fast-moving fields that the extra momentum lost during a 6-12-month embargo is never really recouped.)

Evans, JA & Reimer, J. (2009) Open Access and Global Participation in Science Science 323(5917) (February 20 2009)

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47.

Seglen PO (1992) The skewness of science. Journal of the American Society for Information Science 43:628-38

Stevan Harnad
American Scientist Open Access Forum

Peter Suber wrote in Open Access News:

Notifying authors when they are cited

Elsevier has launched CiteAlert, a free service notifying authors when one of their papers is cited by an Elsevier journal. (Thanks to ResourceShelf.) The service only covers citations to articles published since 2005 in journals indexed by Scopus.

Comments [by Peter Suber]

"This is useful as far as it goes, and I can see why Elsevier can't take it much further on its own. But imagine if all journal publishers offered similar services. The utility of receiving their reports, knowing that they comprehensively covered the field, would be immense. But the labor of signing up for each one separately would also be immense, not to mention the labor of re-creating the service at thousands of different publishers. The bother of reading separate reports from separate publishers would also be immense. I understand that Elsevier's portfolio is larger than anyone else's, but the long tail of academic publishing means that Elsevier's titles still constitute less than 10% of all of peer-reviewed journals.

"I'd like to see a service that notifies authors when one of their works is cited by any journal, regardless of its publisher. If this can't be done by a creative developer harvesting online information (because the harvester doesn't have access to TA sites), then how about a consortial solution from the publishers themselves? And don't stop at emails to authors. Create RSS feeds which users can mash-up in any way they like. Imagine getting a feed of your citations from this hypothetical service and a feed of your downloads from your institutional repository. Imagine your IR feeding the citations in your articles to an OA database, upon which anyone could draw, including this hypothetical service.

"Who could do this? OpenURL? CrossRef? ParaCite? Google Scholar? OCLC (after it acquires OAIster)? A developer at an institution like Harvard with access to the bulk of TA journals? Perhaps someone could build the OA database now, with the citation-input and email- and RSS-output functions, and worry later about how to recruit publishers and repositories and/or how to harvest their citations."

It is clear who should notify whom -- once the global research community's (Green OA ) task is done. Our task is first to get all refereed research journal articles self-archived in their authors' Institutional Repositories (IRs) immediately upon acceptance for publication. (To accomplish that we need universal Green OA self-archiving mandates to be adopted by all institutions and funders, worldwide.)

Once all current and future articles are being immediately deposited in their authors' IRs, the rest is easy:

The articles are all in OAI-compliant IRs. The IR software treats the articles in the reference list of each of its own deposited articles as metadata, to be linked to the cited article, where it too is deposited in the distributed network of IRs. A citation harvesting service operating over this interlinked network of IRs can then provide (among many, many other scientometric services) a notification service, emailing each author of a deposited article whenever a new deposit cites it. (No proporietary firewalls, no toll- or access-barriers: IR-to-IR, i.e., peer-to-peer.)

Stevan Harnad
American Scientist Open Access Forum

On 22-Jan-09, at 5:18 AM, Francis Jayakanth wrote on the eprints-tech list:

"Till recently, we used to include references for all the uploads that are happening into our repository. While copying and pasting metadata content from the PDFs, we don't directly paste the copied content onto the submission screen. Instead, we first copy the content onto an editor like notepad or wordpad and then copy the content from an editor on to the submission screen. This is specially true for the references.

"Our experience has been that when the references are copied and pasted on to an editor like notepad or wordpad from the PDF file, invariably non-ascii characters found in almost every reference. Correcting the non-ascii characters takes considerable amount of time. Also, as to be expected, the references from difference publishers are in different styles, which may not make reference linking straight forward. Both these factors forced us take a decision to do away with uploading of references, henceforth. I'll appreciate if you could share your experiences on the said matter."

The items in an article's reference list are among the most important of metadata, second only to the equivalent information about the article itself. Indeed they are the canonical metadata: authors, year, title, journal. If each Institutional Repository (IR) has those canonical metadata for every one of its deposited articles as well as for every article cited by every one of its deposited articles, that creates the glue for distributed reference interlinking and metric analysis of the entire distributed OA corpus webwide, as well as a means of triangulating institutional affiliations and even name disambiguation.

Yes, there are some technical problems to be solved in order to capture all references, such as they are, filtering out noise, but those technical problems are well worth solving (and sharing the solution) for the great benefits they will bestow.

The same is true for handling the numerous (but finite) variant formats that references may take: Yes, there are many, including different permutations in the order of the key components, abbreviations, incomplete components etc., but those too are finite, can be solved once and for all to a very good approximation, and the solution can be shared and pooled across the distributed IRs and their softwares. And again, it is eminently worthwhile to make the relatively small effort to do this, because the dividends are so vast.

I hope the IR community in general -- and the EPrints community in particular -- will make the relatively small, distributed, collaborative effort it takes to ensure that this all-important OA glue unites all the IRs in one of their most fundamental functions.

(Roman Chyla has replied to eprints-tech with one potential solution: "The technical solution has been there for quite some time, look at citeseer where all the references are extracted automatically (the code of the citeseer, the old version, was available upon request - I dont know if that is the case now, but it was in the past). That would be the right way to go, imo. I think to remember one citeseer-based library for economics existed, so not only the computer-science texts with predictable reference styles are possible to process. With humanities it is yet another story.")

Stevan Harnad
American Scientist Open Access Forum

"[A]n investigation of the use of open access by researchers from developing countries... show[s] that open access journals are not characterised by a different composition of authors than the traditional toll access journals... [A]uthors from developing countries do not cite open access more than authors from developed countries... [A]uthors from developing countries are not more attracted to open access than authors from developed countries. [underscoring added]"(Frandsen 2009, J. Doc. 65(1))
(See also "Open Access: No Benefit for Poor Scientists")

Open Access is not the same thing as Open Access Journals.

Articles published in conventional non-Open-Access journals can also be made Open Access (OA) by their authors -- by self-archiving them in their own Institutional Repositories.

The Frandsen study focused on OA journals, not on OA articles. It is problematic to compare OA and non-OA journals, because journals differ in quality and content, and OA journals tend to be newer and fewer than non-OA journals (and often not at the top of the quality hierarchy).

Some studies have reported that OA journals are cited more, but because of the problem of equating journals, these findings are limited. In contrast, most studies that have compared OA and non-OA articles within the same journal and year have found a significant citation advantage for OA. It is highly unlikely that this is only a developed-world effect; indeed it is almost certain that a goodly portion of OA's enhanced access, usage and impact comes from developing-world users.

It is unsurprising that developing world authors are hesitant about publishing in OA journals, as they are the least able to pay author/institution publishing fees (if any). It is also unsurprising that there is no significant shift in citations toward OA journals in preference to non-OA journals (whether in the developing or developed world): Accessibility is a necessary -- not a sufficient -- condition for usage and citation: The other necessary condition is quality. Hence it was to be expected that the OA Advantage would affect the top quality research most. That's where the proportion of OA journals is lowest.

The Seglen effect ("skewness of science") is that the top 20% of articles tend to receive 80% of the citations. This is why the OA Advantage is more detectable by comparing OA and non-OA articles within the same journal, rather than by comparing OA and non-OA journals.

We will soon be reporting results showing that the within-journal OA Advantage is higher in "higher-impact" (i.e., more cited) journals. Although citations are not identical with quality, they do correlate with quality (when comparing like with like). So an easy way to understand the OA Advantage is as a quality advantage -- with OA "levelling the playing field" by allowing authors to select which papers to cite on the basis of their quality, unconstrained by their accessibility. This effect should be especially strong in the developing world, where access-deprivation is greatest.

---

Leslie Chan -- "Associate Director of Bioline International, co-signatory of the Budapest Open Access Initiative, supervisor in the new media and international studies programs at the University of Toronto, and tireless champion for the needs of the developing world" (Poynder 2008) -- has added the following in the American Scientist Open Access Forum:

I concur with Stevan's comments, and would like to add the following:

1. From our perspective, OA is as much about the flow of knowledge from the South to the North as much as the traditional concern with access to literature from the North. So the question to ask is whether with OA, authors from the North are starting to cite authors from the South. This is a study we are planning. We already have good evidence that more authors from the North are publishing in OA journals in the South (already an interesting reversal) but we need a more careful analysis of the citation data.

2. The more critical issue regarding OA and developing country scientists is that most of those who publish in "international" journals cannot access their own publications. This is where open repositories are crucial, to provide access to research from the South that is otherwise inaccessible.

3. The Frandsen study focuses on biology journals and I am not sure what percentage of them are available to DC researchers through HINARI/AGORA. This would explain why researchers in this area would not need to rely on OA materials as much. But HINARI etc. are not OA programs, and local researchers will be left with nothing when the programs are terminated. OA is the only sustainable way to build local research capacity in the long term.

4. Norris et. al's (2008) "Open access citation rates and developing countries" focuses instead on Mathematics, a field not covered by HINARI and they conclude that "the majority of citations were given by Americans to Americans, but the admittedly small number of citations from authors in developing countries do seem to show a higher proportion of citations given to OA articles than is the case for citations from developed countries. Some of the evidence for this conclusion is, however, mixed, with some of the data pointing toward a more complex picture of citation behaviour."

5. Citation behaviour is complex indeed and more studies on OA's impact in the developing world are clearly needed. Davis's eagerness to pronounce that there is "No Benefit for Poor Scientists" based on one study is highly premature.

If there should be a study showing that people in developing countries prefer imported bottled water over local drinking water, should efforts to ensure clean water supplies locally be questioned?

Leslie Chan

Stevan Harnad
American Scientist Open Access Forum