WEIGHING ARTICLES/AUTHORS INSTEAD OF JOURNALS IN RESEARCH ASSESSMENT

The United Kingdom ranks and rewards the research productivity of all its universities through a national Research Assessment Exercise (RAE) conducted every four years.

If, as lately proposed, the "RAE shifts focus from prestige journals" (THS, 22 July 2005) as a basis for its ranking, what will it shift focus to?

An established journal's prestige and track-record are correlated with its selectivity and peer-review standards, hence its quality level, and often also its citation impact. What would it mean to ignore or de-emphasise that? The correlations will be ignored, all articles will be given equal weight -- and then what?

What gain in accuracy and fairness of research assessment is to be expected from ignoring the known predictors -- for correlation is predictive -- of research quality? Are all articles to be re-peer-reviewed by the RAE itself, bottom-up? Is that efficient, desirable, realistic? The most prestigious international journals draw upon international expertise in their peer review: Is the UK to reduplicate all this effort in-house every 4 years? Why? Isn't our time better spent getting the peer-reviewing done right the first time, and then getting on with our research?

Research assessment used to be publish-or-perish bean-counting; it is now weighted by the quality level of the journal in which the bean is planted. RAE outcome is already highly correlated with counts of the citations that articles sprout, even though the RAE never actually counts citations directly. That's because a journal's prestige is correlated with its articles' citation counts.

So if we're going to start ignoring journal prestige, shouldn't we begin to count article (and author) citations directly in its place?