Prior Amsci Topic Thread:
"How to compare research impact of toll- vs. open-access research" (started June 2003)
Harnad, S. (2005) Maximising the Return on UK's Public Investment in Research
On Mon, 3 Oct 2005, Sally Morris (SM) of
ALPSP wrote in
SOAF:
SM: "The problem is, there is no evidence of correlation between citations and the return on research expenditure."
Citations
are one direct, face-valid measure of return on research expenditure. Research is funded in order to be applied and built-upon, i.e., to be
used; citations are an index of that usage. Uncited, unused research may as well not have been conducted, and represents
no return on the research investment. Whatever increases usage and citations, increases the return on the research investment. Any loss of such a potential increase is a loss of potential return on the research investment.
Self-archiving increases citations 50%-250%. Hence the failure to self-archive loses 50%-250% of the potential return on the research investment.
SM: "I haven't been able to trace many analyses which do look at this (Don King will know, if anyone does) but those I've read look at output of articles, registration of patents, and Gross Domestic Product."
Article counts are a measure of the return on the research investment, but far too crude a measure, for, as noted, the articles may not be used.
Patents are pertinent only to a tiny portion of the research literature, so have insufficient generality to be a useful general measure of research impact. Moreover, they are often based on
unpublished research, whereas self-archiving and the OA movement are directed specifically at published research. However, patent counts and citation counts are in fact positively correlated:
"patent volume is positively correlated with paper citations, suggesting that patent counts may be reasonable measures of research impact" Agarwal, A. & Henderson, R. (2002) Putting Patents in Context: Exploring Knowledge Transfer from MIT. Management Science 48 (1), 44-60
Gross domestic product is again too crude. Most basic research is too far from practical applications to contribute to the GDP. But one thing is certain: If a piece of research is to make a contribution to the GDP, it must be accessible to its potential appliers. Self-archiving substantially increases accessibility, as indicated by the fact that it generates substantially more citations.
I too would be interested, however, to know of studies correlating GDP with citation counts.
SM: "Clearly, we are a long way off being able to analyse whether or not self-archiving (or any other form of open access) does or does not contribute to these objective output measures."
I thought the question was about whether citation counts are correlated with these measures. We already know that self-archiving is correlated with increased citation counts.
SM: "But to pretend that we 'know' citations are a proxy for any of them is not, to my mind, an argument that holds any water"
The claim was not that citations are a proxy for GDP, but that citations are a (face-valid) measure of the return on the investment of public funds in research -- and, more particularly, that the loss of potential citations is the loss of potential returns on the investment of public funds in research (lost "value for money").
SM: "Stevan, I know what you're going to say so please don't bother - frankly, I am more interested in hearing what other people have to say"
Sally, I'd be pleased to obey your request not to reply to you, if this were only a private conversation between you and me. But, you see, others are involved too, in particular, researchers and their interests. You appear to be concerned about hypothetical future losses to publishers because of self-archiving -- losses for which there exists no evidence at all to date. I am concerned about actual current losses to researchers because of
not self-archiving -- losses for which the sizeable positive correlations between self-archiving and citation counts, and between citations counts and researcher revenue (in terms of both salary and research funding) constitute strong positive evidence.
See, for example, the many studies showing the correlation between RAE rankings and citation counts, as cited in Harnad, Carr, Brody & Oppenheim (2003): "
Mandated online RAE CVs linked to university eprint archives: Enhancing UK research impact and assessment"
In particular,
Eysenck & Smith (2002) write:
"Correlation between RAE ratings and mean departmental citations +0.91 (1996) +0.86 (2001) (Psychology)"
"RAE and citation counting measure broadly the same thing"
Stevan Harnad
