Open Access Archivangelism

Comment on:
Richard Monastersky, The Number That's Devouring Science,
Chronicle of Higher Education (CHE), October 1, 2005
(truncated version of reply below appeared in
CHE November 14, 2005)

---

Although Richard Monasterky describes a real problem -- the abuse of journal impact factors -- its solution is so obvious one hardly required so many words on the subject:

A journal's citation impact factor (CIF) is the average number of citations received by articles in that journal (ISI -- somewhat arbitrarily -- calculates CIFs on the basis of the preceding two years, although other time-windows may also be informative.)

There is an undeniable relationship between the usefulness of an article and how many other articles use and hence cite it. Hence CIF does measure the average usefulness of the articles in a journal. But there are three problems with the way CIF itself is used, each of them readily correctable:

(1) A measure of the average usefulness of the articles in the journal in which a given article appears is no substitute for the actual usefulness of each article itself: In other words, the journal CIF is merely a crude and indirect measure of usefulness; each article's own citation count is the far more direct and accurate measure. (Using the CIF instead of an article's own citation count [or the average citation count for the author] for evaluation and comparison is like using the average marks for the school from which a candidate graduated, rather than the actual marks of the candidate.)

(2) Whether comparing CIFs or direct article/author citation counts, one must always compare like with like. There is no point comparing either CIFs between journals in different fields, or citation counts for articles/authors in different fields. (A normalised citation count can also be used, adjusting for different baseline citation levels and variability in different fields.)

(3) Both CIFs and citation counts can be distorted and abused. Authors can self-cite, or cite their friends; some journal editors can and do encourage self-citing their journal. These malpractices are deplorable, but most are also detectable, and then name-and-shame-able and correctable. ISI could do a better job policing them, but soon the playing field will widen, for as authors make their articles open access online, other harvesters -- such as citebase and citeseer and even google scholar -- will be able to harvest and calculate citation counts, and average, compare, expose, enrich and correct them in powerful ways that were inconceivable in the Gutenberg era.

So, yes, CIFs are being misused and abused currently, but the cure is already obvious -- and a wealth of powerful new resources are on the way for measuring and analyzing research usage and impact online, including (1) download counts, (2) co-citation counts (co-cited with, co-cited by), (3) hub/authority ranks (authorities are highly cited papers cited by many highly cited papers; hubs cite many authorities), (4) download/citation correlations and other time-series analyses, (5) download growth-curve and peak latency scores, (6) citation growth-curve and peak-latency scores, (7) download/citation longevity scores, (8) co-text analysis (comparing similar texts, extrapolating directional trends), and much more. It will no longer be just CIFs and citation counts but a rich multiple regression equation, with many weighted predictor variables based on these new measures. And they will be available for both navigators and evaluators online, and based not just on the current ISI database but on all of the peer-reviewed research literature.

Meanwhile, use the direct citation counts, not the CIFs.

Some self-citations follow:

Brody, T. (2003) Citebase Search: Autonomous Citation Database for e-print Archives, sinn03 Conference on Worldwide Coherent Workforce, Satisfied Users - New Services For Scientific Information, Oldenburg, Germany, September 2003

Brody, T. (2004) Citation Analysis in the Open Access World Interactive Media International

Brody, T. , Harnad, S. and Carr, L. (2005) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST, in press).

Hajjem, C., Gingras, Y., Brody, T., Carr, L. & Harnad, S. (2005) Across Disciplines, Open Access Increases Citation Impact. (manuscript in preparation).

Hajjem, C. (2005) Analyse de la variation de pourcentages d'articles en accès libre en fonction de taux de citations

Harnad, S. and Brody, T. (2004a) Comparing the Impact of Open Access (OA) vs. Non-OA Articles in the Same Journals. D-Lib Magazine, Vol. 10 No. 6

Harnad, S. and Brody, T. (2004) Prior evidence that downloads predict citations. British Medical Journal online.

Harnad, S. and Carr, L. (2000) Integrating, Navigating and Analyzing Eprint Archives Through Open Citation Linking (the OpCit Project). Current Science 79(5):pp. 629-638.

Harnad, S. , Brody, T. , Vallieres, F. , Carr, L. , Hitchcock, S. , Gingras, Y. , Oppenheim, C. , Stamerjohanns, H. and Hilf, E. (2004) The Access/Impact Problem and the Green and Gold Roads to Open Access. Serials Review, Vol. 30, No. 4, 310-314

Hitchcock, S. , Brody, T. , Gutteridge, C. , Carr, L. , Hall, W. , Harnad, S. , Bergmark, D. and Lagoze, C. (2002) Open Citation Linking: The Way Forward. D-Lib Magazine 8(10).

Hitchcock, S. , Carr, L. , Jiao, Z. , Bergmark, D. , Hall, W. , Lagoze, C. and Harnad, S. (2000) Developing services for open eprint archives: globalisation, integration and the impact of links. In Proceedings of the 5th ACM Conference on Digital Libraries, San Antonio, Texas, June 2000., pages pp. 143-151.

Hitchcock, S. , Woukeu, A. , Brody, T. , Carr, L. , Hall, W. and Harnad, S. (2003) Evaluating Citebase, an open access Web-based citation-ranked search and impact discovery service. Technical Report ECSTR-IAM03-005, School of Electronics and Computer Science, University of Southampton

Stevan Harnad

This is the text of the letter that will appear in Research Fortnight. That's half of what I submitted. The full text I submitted follows immediately afterward.

---

Letter to appear in Research Fortnight

In its editorial in the last issue, Research Fortnight declares neutrality and in the same breath breaches it:

"Research Fortnight does not publish learned journals and has no reason to defend commercial publishers, but the publishers are right when they say that self-archiving as proposed by Research Councils UK will stop new journals being launched and cause existing journals to close."

There is a profound conflict of interest between - on the one hand - what is in the best interests of research, researchers, their institutions, their funding councils and the tax-paying public that funds the research and - on the other hand - what is in the best interests of the publishing community. Research Fortnight's position is virtually identical with that of the publishing community, as expressed most vocally by Sally Morris of the Association for Learned and Professional Society Publishers.

The concerns of the publishing community are with preventing any risk of subscription cancellations that might be induced by self-archiving. But self-archiving has now been going on for over 14 years, and in physics reached 100 per cent in some fields years ago. Yet both of the major physics publishers (the American Physical Society and the Institute of Physics) report that (1) they detect no diminished subscriptions due to self-archiving, (2) they do not consider self-archiving a threat, and (3) they cooperate with and even host a mirror site of the physics self-archives at their own websites.

Hence the hypothetical risk from which publishers seek protection does not even have any objective evidence in its support. The evidence is that self-archiving and journal-publishing co-exist peacefully. In contrast, the objective evidence for the actual benefits of self-archiving to research exists and is very strong: Articles that are self-archived are cited (hence used, applied and built upon) 50 per cent - 250 per cent more than articles that are not.

---

Full submitted text (including what will not appear):

Research Councils UK (RCUK) have proposed mandating that all RCUK fundees must deposit ("self-archive") the final drafts of their research on the web, free for all would-be users whose institutions cannot afford the published version, as a condition of receiving public funding, in order to maximise the usage and impact of their UK research output, and thereby the return on the UK public's investment in funding it. RF declares neutrality, and in the same breath breaches it:

"Research Fortnight does not publish learned journals and has no reason to defend commercial publishers, but the publishers are right when they say that self-archiving as proposed by Research Councils UK will stop new journals being launched and cause existing journals to close."

There is a profound conflict of interest between what is in the best interests of research, researchers, their institutions, their funding councils, and the tax-paying public that funds the research, on the one hand, and what is in the best interests of the publishing community on the other hand. RF's position is virtually identical with that of the publishing community, as expressed most vocally by Sally Morris of the Association for Learned and Professional Society Publishers (ALPSP).

The concerns of the publishing community are with preventing any risk of subscription cancellations that might be induced by self-archiving. But self-archiving has now been going on for over 14 years, and in physics reached 100% in some fields years ago, yet both of the major physics publishers (APS and IOP) report that (1) they detect no diminished subscriptions due to self-archiving, (2) they do not consider self-archiving a threat, and (3) they cooperate with and even host a mirror site of the Physics self-archives at their own websites.

Hence the hypothetical risk from which publishers seek protection does not even
have any objective evidence in its support, all evidence being that
self-archiving and journal-publishing co-exist peacefully. In contrast, the
objective evidence for the actual benefits of self-archiving to research exists
and is very strong: Articles that are self-archived are cited (hence used,
applied and built upon) 50%-250% more than articles that are not. I did a
matchbox calculation showing that since only 15% of UK articles are currently
being self-archived this amounts to a loss of at least 50% x 85% = £1.5 billion
worth of potential return on RCUK's annual £3.5 billion research investment,
reckoning that return in terms of lost potential research usage. RF
wrote:

"This argument... [not] peer-reviewed or published... is so
ludicrous it would be a waste of space bothering to knock it down."

A transparent match-box calculation whose outcome seems to be uncongenial to
some ears is discounted by RF for not having been "peer-reviewed." One wonders if, following the same logic, RF would have discounted the following unrefereed observation:

Prior (published) evidence has shown that placing unused batteries (cost, £1 apiece) in the refrigerator increases their hours of usage by 50%, but only 15% of users refrigerate them. We accordingly point out here the following match-box calculation: The 85% of battery-users who are not refrigerating their batteries are losing 50p's worth of potential usage, hence 50p's worth of value for their money.

Ludicrous? In need of peer review? A waste of space to bother refuting? Or dismissed only as a consequence of having listened only to the battery-makers who say that self-refrigerating "will stop new battery-makers being launched and cause existing battery-makers to close" rather than to the needs of the battery-using (and subsidising) public?

Stevan Harnad