Critique of "Impact Assessment," Chrisp & Toale, Pharmaceutical Marketing 2008

The following is a (belated) critique of:

"Impact Assesment," by Paul Chrisp (publisher, Core Medical Publishing) & Kevin Toale (Dove Medical Press). Pharmaceutical Marketing September 2008

---

"Open access has emerged in the last few years as a serious alternative to traditional commercial publishing models, taking the benefits afforded by technology one step further. In this model, authors are charged for publishing services, and readers can access, download, print and distribute papers free at the point of use."

Incorrect.

Open Access (OA) means free online access and OA Publishing ("Gold OA") is just one of the two ways to provide OA (and not the fastest, cheapest or surest):

The fastest, cheapest and surest way to provide OA is OA Self-Archiving (of articles published in conventional non-OA journals: "Green OA") in the author's Institutional Repository.

"Although its ultimate goal is the free availability of information online, open access is not the same as free access – publishing services still cost money."

Incorrect.

There are two forms of OA: (1) Gratis OA (free online access) and (2) Libre OA (free online access plus certain re-user rights)

"Other characteristics of open access journals are that authors retain copyright and they must self-archive content in an independent repository."

Incorrect.

This again conflates Green and Gold OA:

Gold OA journals make their own articles free online.

In Green OA, articles self-archive their articles.

"researchers are depositing results in databases rather than publishing them in journal articles"

Incorrect.

This conflates unrefereed preprint self-archiving with refereed, published postprint self-archiving.

Green OA is the self-archiving of refereed, published postprints.

The self-archiving of unrefereed preprints is an optional supplement to, not a substitute for, postprint OA.

"a manuscript may be read more times than it is cited, and research shows that online hits per article do not correlate with IF".

Incorrect.

"Research shows" that online hits (downloads) do correlate with citations (and hence with citation impact factors).

See references cited below.

"Faculty of 1000 (www.f1000medicine.com)... asks opinion leaders in clinical practice and research to select the most influential articles in 18 medical specialties. Articles are evaluated and ranked..."

Expert rankings are rankings and metrics (such as hit or citation counts) are metrics.

Metrics can and should be tested and validated against expert rankings. Validated metrics can then be used as supplements to -- or even substitutes for -- rankings. But the validation has to be done a much broader and more systematic basis than Faculty of 1000, and on a much richer set of candidate metrics.

Nor is the purpose of metrics "pharmaceutical marketing": It is to monitor, predict, navigate, analyze and reward research influence and importance.

Bollen, J., Van de Sompel, H., Hagberg, A. and Chute, R. (2009) A principal component analysis of 39 scientific impact measures in PLoS ONE 4(6): e6022,

Brody, T., Harnad, S. and Carr, L. (2006) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 57(8) 1060-1072.

Harnad, S. (2008) Validating Research Performance Metrics Against Peer Rankings . Ethics in Science and Environmental Politics 8 (11) doi:10.3354/esep00088 The Use And Misuse Of Bibliometric Indices In Evaluating Scholarly Performance

Harnad, S. (2009) Open Access Scientometrics and the UK Research Assessment Exercise. Scientometrics 79 (1) Also in Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics 11(1), pp. 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds. (2007)

Lokker, C., McKibbon, K. A., McKinlay, R.J., Wilczynski, N. L. and Haynes, R. B. (2008) Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study BMJ, 2008;336:655-657

Moed, H. F. (2005) Statistical Relationships Between Downloads and Citations at the Level of Individual Documents Within a Single Journal. Journal of the American Society for Information Science and Technology 56(10): 1088- 1097

O'Leary, D. E. (2008) The relationship between citations and number of downloads Decision Support Systems 45(4): 972-980

Watson, A. B. (2009) Comparing citations and downloads for individual articles Journal of Vision 9(4): 1-4