Open Access Archivangelism

Since almost the beginning, the inadvertent and intentional conflation of Green OA (Self-Archiving) with Gold OA (publishing) has been a great obstacle to OA itself. I used to call it "Drubbing Peter to Pox Paul" and Peter Suber has since 2006 been calling it "JAM" (Journal/Archive Mix-Up).

Please do whatever you can to dispel this error as it is really holding up progress. In most cases, it is merely the result of ignorance or misunderstanding. But in the case of those who are lobbying against the NIH Bill, it is deliberately being used as a way to impugn Green OA Self-Archiving mandates by pretending they are Gold-OA publishing mandates.

Stevan Harnad
American Scientist Open Access Forum

Comments on UUK Press Release 8 November 2007:
UUK report looks at the use of bibliometrics

"This report will help Universities UK to formulate its position on the development of the new framework for replacing the RAE after 2008."

Some of the points for consideration in the report include:

Bibliometrics are probably the most useful of a number of variables that could feasibly be used to measure research performance.

What metrics count as "bibliometrics"? Do downloads? hubs/authorities? Interdisciplinarity metrics? Endogamy/exogamy metrics? chronometrics, semiometrics?

There is evidence that bibliometric indices do correlate with other, quasi-independent measures of research quality - such as RAE grades - across a range of fields in science and engineering.

Meaning that citation counts correlate with panel rankings in all disciplines tested so far. Correct.

There is a range of bibliometric variables as possible quality indicators. There are strong arguments against the use of (i) output volume (ii) citation volume (iii) journal impact and (iv) frequency of uncited papers.

The "strong" arguments are against using any of these variables alone, or without testing and validation. They are not arguments against including them in the battery of candidate metrics to be tested, validated and weighted against the panel rankings, discipline by discipline, in a multiple regression equation.

'Citations per paper' is a widely accepted index in international evaluation. Highly-cited papers are recognised as identifying exceptional research activity.

Citations per paper is one (strong) candidate metric among many, all of which should be co-tested, via multiple regression analysis, against the parallel RAE panel rankings (and other validated or face-valid performance measures).

Accuracy and appropriateness of citation counts are a critical factor.

Not clear what this means. ISI citation counts should be supplemented by other citation counts, such as Scopus, Google Scholar, Citeseer and Citebase: each can be a separate metric in the metric equation. Citations from and to books are especially important in some disciplines.

There are differences in citation behaviour among STEM and non-STEM as well as different subject disciplines.

And probably among many other disciplines too. That is why each discipline's regression equation needs to be validated separately. This will yield a different constellation of metrics as well as of beta weights on the metrics, for different disciplines.

Metrics do not take into account contextual information about individuals, which may be relevant.

What does this mean? Age, years since degree, discipline, etc. are all themselves metrics, and can be added to the metric equation.

They also do not always take into account research from across a number of disciplines.

Interdisciplinarity is a measurable metric. There are self-citations, co-author citations, small citation circles, specialty-wide citations, discipline-wide citations, and cross-disciplinary citations. These are all endogamy/exogamy metrics. They can be given different weights in fields where, say, interdisciplinarity is highly valued.

The definition of the broad subject groups and the assignment of staff and activity to them will need careful consideration.

Is this about RAE panels? Or about how to distribute researchers by discipline or other grouping?

Bibliometric indicators will need to be linked to other metrics on research funding and on research postgraduate training.

"Linked"? All metrics need to be considered jointly in a multiple regression equation with the panel rankings (and other validated or face-valid criterion metrics).

There are potential behavioural effects of using bibliometrics which may not be picked up for some years

Yes, metrics will shape behaviour (just as panel ranking shaped behaviour), sometimes for the better, sometimes for the worse. Metrics can be abused -- but abuses can also be detected and named and shamed, so there are deterrents and correctives.

There are data limitations where researchers' outputs are not comprehensively catalogued in bibliometrics databases.

The obvious solution for this is Open Access: All UK researchers should deposit all their research output in their Institutional Repositories (IRs). Where it is not possible to set access to a deposit as OA, access can be set as Closed Access, but the bibliographic metadata will be there. (The IRs will not only provide access to the texts and the metadata, but they will generate further metrics, such as download counts, chronometrics, etc.)

The report comes ahead of the HEFCE consultation on the future of research assessment expected to be announced later this month. Universities UK will consult members once this is published.

Let's hope both UUK and HEFCE are still open-minded about ways to optimise the transition to metrics!

References

Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated online RAE CVs Linked to University Eprint Archives: Improving the UK Research Assessment Exercise whilst making it cheaper and easier. Ariadne 35.

Brody, T., Kampa, S., Harnad, S., Carr, L. and Hitchcock, S. (2003) Digitometric Services for Open Archives Environments. In Proceedings of European Conference on Digital Libraries 2003, pp. 207-220, Trondheim, Norway.

Harnad, S. (2006) Online, Continuous, Metrics-Based Research Assessment. Technical Report, ECS, University of Southampton.

Harnad, S. (2007) Open Access Scientometrics and the UK Research Assessment Exercise. In Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics 11(1), pp. 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds.

Brody, T., Carr, L., Harnad, S. and Swan, A. (2007) Time to Convert to Metrics. Research Fortnight pp. 17-18.

Brody, T., Carr, L., Gingras, Y., Hajjem, C., Harnad, S. and Swan, A. (2007) Incentivizing the Open Access Research Web: Publication-Archiving, Data-Archiving and Scientometrics. CTWatch Quarterly 3(3).

See also: Prior Open Access Archivangelism Postings on RAE and metrics

Stevan Harnad
American Scientist Open Access Forum

In "OA, OK?" Richard Gallagher (2007) is quite right to say "we're still waiting" for the "optimal and inevitable" [Open Access]. I was already in full agreement in the previous millennium (Harnad 1999):

"I have a feeling that when Posterity looks back at the last decade of the 2nd A.D. millennium of scholarly and scientific research on our planet, it may chuckle at us. It is not the pace of our scholarly and scientific research that will look risible, nor the tempo of technological change. On the contrary, the astonishing speed and scale of both will make the real anomaly look all the more striking.

"For staring us in the face in this last decade has been an obvious new way to augment that already impressive speed and scale by perhaps an order of magnitude, yet we simply haven't twigged on it...

"I don't think there is any doubt in anyone's mind as to what the optimal and inevitable outcome of all this will be: The Give-Away literature will be free at last online, in one global, interlinked virtual library... and its [peer-review] expenses will be paid for up-front, out of the [subscription cancellation] savings. The only question is: When? This piece is written in the hope of wiping the potential smirk off Posterity's face by persuading the academic cavalry, now that they have been led to the waters of self-archiving, that they should just go ahead and drink!" -- (Harnad 1999)

But Gallagher is not quite right that "most scientists became indifferent about Open Access." The syndrome is not quite indifference but a combination of ignorance and indolence (Swan 2005) concerning what is already demonstrably in their own best interests and fully within their reach. I have dubbed the syndrome "Zeno's Paralysis" (Harnad 2006); the affliction is, fortunately, curable. The medicine is OA self-archiving mandates (Harnad 2001, Harnad et al. 2003; Harnad 2007) by researchers' institutions and funders.

And those mandates are on the way. The inertia is and always was merely a matter of keystrokes: getting those digits to deposit those digits. "Publish or perish" mandates managed to induce otherwise busy, curiosity-driven researchers to find the time to set their (peer-reviewed) findings to paper, and self-archiving mandates will now ensure the few additional minutes it takes to make all published papers immediately and permanently accessible free for all their potential users online, rather than just for those whose institutions can afford subscription access to the journal in which they happen to be published (Carr & Harnad 2005).

To close, a few loose ends:

(1) OA is not about journal affordability but about research accessibility.

(2) Harnad (1998) is available not because of "an online forum" but because it was self-archived in my institutional repository (without waiting for a mandate).

(3) The optimal/inevitable is not just overdue since 1998, but at least since 1994 (Harnad 1995, 2004; Poynder 2004), and actually at least a decade longer.

(4) OA self-archiving mandates have been mooted since at least 2001 (Harnad 2001, Harnad et al 2003).

(5) A more optimistic version of Esposito's (2007) "nautilus" was already mooted seventeen years ago, as "scholarly skywriting" (Harnad 1990).

References

Carr, L. and Harnad, S. (2005) Keystroke Economy: A Study of the Time and Effort Involved in Self-Archiving. Technical Report, ECS, University of Southampton.

Esposito, J. (2007) Open Access 2.0. The Scientist 21(11) 52

Gallagher, R. (2007) OA: OK? The Scientist 21(11) 13

Harnad, S. (1990) Scholarly Skywriting and the Prepublication Continuum of Scientific Inquiry. Psychological Science 1: 342 - 343 (reprinted in Current Contents 45: 9-13, November 11 1991).

Harnad, S. (1995) Universal FTP Archives for Esoteric Science and Scholarship: A Subversive Proposal. In: Ann Okerson & James O'Donnell (Eds.) Scholarly Journals at the Crossroads; A Subversive Proposal for Electronic Publishing. Washington, DC., Association of Research Libraries, June 1995.

Harnad, S. (1998) On-Line Journals and Financial Fire-Walls. Nature 395 (6698): 127-128

Harnad, S. (1999) Free at Last: The Future of Peer-Reviewed Journals. D-Lib Magazine 5(12).

Harnad, Stevan (2001/2003/2004) For Whom the Gate Tolls? Published as: (2003) Open Access to Peer-Reviewed Research Through Author/Institution Self-Archiving: Maximizing Research Impact by Maximizing Online Access. In: Law, Derek & Judith Andrews, Eds. Digital Libraries: Policy Planning and Practice. Ashgate Publishing 2003. [Shorter version: Harnad S. (2003) Journal of Postgraduate Medicine 49: 337-342.] and in: (2004) Historical Social Research (HSR) 29:1. [French versions: Harnad, S. (2003) Ciélographie et ciélolexie: Anomalie post-gutenbergienne et comment la résoudre. In: Origgi, G. & Arikha, N. (eds) Le texte à l'heure de l'Internet. Bibliotheque Centre Pompidou: Pp. 77-103. ]

Harnad, S. (2004) June 27 2004: The 1994 "Subversive Proposal" at 10.
American Scientist Open Access Forum. June 27 2004.

Harnad, S. (2006) Opening Access by Overcoming Zeno's Paralysis, in Jacobs, N., Eds. Open Access: Key Strategic, Technical and Economic Aspects. Chandos.

Harnad, S. (2007) The Green Road to Open Access: A Leveraged Transition. In: Anna Gacs. The Culture of Periodicals from the Perspective of the Electronic Age. L'Harmattan. 99-106.

Harnad, S., Carr, L., Brody, T. and Oppenheim, C. (2003) Mandated online RAE CVs Linked to University Eprint Archives. Ariadne 35.

Poynder, R. (2004) Ten Years After. Information Today. October 2004

Swan, A. (2005) Open access self-archiving: An Introduction. JISC Technical Report.

Stevan Harnad
American Scientist Open Access Forum

This is a response to a query from a Southampton colleague who received an unsolicited invitation from an unknown individual to contribute a chapter to an "Open Access" book (author pays) on the basis of a paper he had deposited in the ECS Southampton Institutional Repository (IR) -- and possibly on the basis of its download statistics:

The colleague asked:

(1) Is the book chapter that [identity deleted] is soliciting an example of Open Access?

(2) Are download counts legitimate metrics for (2a) CVs, (2b) website statistics, (2c) departmental/institutional repository (IR) statistics?

(3) Can download statistics be abused?

(4) Should institutional authors be able to "opt out" of (4a) depositing their paper in their IR and/or (4b) having their download statistics displayed?

(1) Yes, Open Access (OA) books are instances of OA just as OA articles are. The big difference is that all peer-reviewed journal/conference articles, without exception, are written exclusively for research usage and impact, not for royalty income, whereas this is not true of all or even most books. Articles are all author give-aways, but most books are not. So articles are OA's primary target; books are optional and many will no doubt follow suit after systematic OA-provision for research articles has taken firm root globally. (Also important: article deposit in the IR can be mandated by researchers' employers and funders, as Southampton ECS and RCUK have done, but book deposit certainly cannot -- and should not -- be mandated.)

(2) Yes, download metrics, alongside citation metrics and other new metric performance indicators can and should be listed in CVs, website stats and IR stats. In and of themselves they do not mean much, as absolute numbers, but in an increasingly OA world, where they can be ranked and compared in a global context, they are potentially useful aids to navigation, evaluation, prediction and other forms of assesment and analysis. (We have published a study that shows there is a good-sized positive correlation between earlier download counts and later citation counts: Brody, T., Harnad, S. and Carr, L. (2006) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 57(8) pp. 1060-1072.)

(3) Yes, download statistics can be -- and will be -- abused, as many other online innovations (like email, discussion lists, blogs, search engines, etc.) can be abused by spammers and other motivated mischief-makers or self-promoters. But it is also true that those abuses can and will breed counter-abuse mechanisms. And in the case of academic download metrics inflation, there will be obvious, powerful ways to counteract and deter it if/when it begins to emerge: Anomalous download patterns (e.g., self-hits, co-author IP hits, robotic hits, lack of correlation with citations, etc.) can be detected, named and shamed. (It is easier for a commercial spammer to abuse metrics and get away with it than for an academic with a career that stands at risk once discovered!)

(4) No, researchers should definitely not be able to "opt out" of a deposit mandate: That would go against both the letter and spirit of a growing worldwide movement among researchers, their institutions and their funders to mandate OA self-archiving for the sake of its substantial benefits to research usage and impact. There is always the option of depositing a paper as Closed Access rather than Open Access, but I think a researcher would be shooting himself in the foot if he chose to do that on account of worries about the abuse of download statistics: It would indeed reduce the download counts, usage and citations of that researcher's work, but it would not accomplish much else. (On the question of opting out of the display of download (and other) metrics, I have nothing objective to add: It is technically possible to opt out of displaying metrics, and if there is enough of a demand for it, it should be made a feature of IRs; but it seems to me that it will only bring disadvantages and handicaps to those who choose to opt out of displaying their metrics, not only depriving them of data to guide potential users and evaluators of their work, but giving the impression that they have something to hide.)

I would also add that the invitation to contribute a book chapter by [identity deleted] might possibly be a scam along the lines of the bogus conference scams we have heard much about. The public availability of metadata, papers and metrics will of course breed such "invitations" too, but one must use one's judgment about which of them are eminently worth ignoring.

Stevan Harnad
American Scientist Open Access Forum

The Max Planck Society would do incomparably more for Open Access (and its own research impact) if it mandated deposit in its own Institutional Repository (IR), Edoc, rather than just canceling journal subscriptions.

For some time now, the reply from the MP Institutes and German universities has been: "We cannot mandate!"

But of course they can! The policy need not be coercive; it need not have sanctions for noncompliance. It need merely be officially adopted. And there are obvious and simple administrative ways to make it worth researchers' while to comply (if the enhanced research impact that OA vouchsafes is not enough): Simply declare the IR as the official institutional submission format for all performance review for its employees!

So there are no administrative barriers. Nor are there any legal barriers: For performance review, it is sufficient to deposit the final, revised, refereed, accepted draft -- the postprint -- immediately upon acceptance for publication, and set access the postprint full-text as Closed Access (administrative access -- with only the bibliographic metadata, not the postprint, visible webwide) rather than immediate Open Access (if the journal in which the article is published is non-Green and demands an embargo).

(Since the only thing that has been standing between us and 100% OA for years now is keystrokes, an administrative keystroke mandate is all that is needed. The increasingly palpable benefits of OA itself will take care of the rest, as carrots, rather than sticks.)

Stevan Harnad
American Scientist Open Access Forum