Research Assessment

Kudos to the University of Oxford! Although the Oxford 2013 "Statement on Open Access" had been rather weak and vague, it has now been reinforced by Open Access Oxford's 2015 system (see text at end, below) for implementing HEFCE/REF2020, and this time it's the optimal system.


Now all that's needed in order to monitor and ensure compliance is to make the date-of-acceptance (year, month, day) field in the Oxford repository (ORA) a mandatory field (and advise authors to make sure to retain their acceptance letter for possible audit).

The repository software can then calculate the (likewise mandatory) deposit date D and the Acceptance date A and subtract A from D. If D - A < 3 (months) then the article is HEFCE-compliant and eligible for REF2020.

If D - A > 3 then the author is alerted that the article risks not being eligible for REF2020 and that for future articles D - A must be less than 3 months.

Automated D-A monitoring and feedback to authors should be continuous and immediate.

HEFCE/REF2020 will probably be flexible about the start-up 1-2 years, but not longer than that. Oxford is right to get the system in place as early as possible.

Vincent-Lamarre, Philippe, Boivin, Jade, Gargouri, Yassine, Larivière, Vincent and Harnad, Stevan (2015) Estimating Open Access Mandate Effectiveness: I. The MELIBEA Score. Journal of the  Association for Information Science and Technology (JASIST) (2015, in press) /

Swan, Alma; Gargouri, Yassine; Hunt, Megan; & Harnad, Stevan (2015) Open Access Policy: Numbers, Analysis, Effectiveness. Pasteur4OA Workpackage 3 Report.

Oxford prepares for ‘Act on Acceptance’

We are continuing as planned with preparations for a single, simple message for open access: Act on Acceptance. Over the summer, we are putting in place mechanisms to ensure all researchers employed by the collegiate University are able to deposit their article manuscripts in our repository ORA within 3 months of acceptance by the journal - to meet HEFCE’s OA requirements and the general push to improve the accessibility of research.

Research Services, the Libraries and IT Services are in the middle of this work, and will be producing materials and guidance for departments & researchers from Autumn 2015. This is when our campaign around ‘act on acceptance’ will begin so as to be ready for HEFCE’s policy start date of 1 April 2016. We will provide further information later in the summer.

If you have any questions please contact our email helpline: open-access-enquiries@bodleian.ox.ac.uk.

The HEFCE/REF exception is not to the deposit requirement but to the OA requirement, and that makes all the difference in the world.

No publisher can block deposit; all they can do is embargo the date on which access to the deposit is set as Open Access (OA).

All REF submissions must be deposited immediately upon acceptance for publication -- embargo or no embargo. The length of the allowable OA embargo, and exemptions from it, are an entirely separate matter.

Immediate-deposit allows a uniform mandate to be adopted by all institutions and funders, regardless of publisher OA embargo policy.

Once deposited, even if embargoed, access to an individual copy for research purposes can nevertheless be requested and provided on a one-to-one basis by one click each from the requestor to request and one click from the author to comply, thanks to the institutional repositories' copy-request Button.

But only if -- and when -- the papers are deposited.

Sale, Arthur; Couture, Marc; Rodrigues, Eloy; Carr, Les and Harnad, Stevan (2014) Open Access Mandates and the "Fair Dealing" Button. In: Dynamic Fair Dealing: Creating Canadian Culture Online (Rosemary J. Coombe & Darren Wershler, Eds.). University of Toronto Press.

Steven Hill of HEFCE has posted “an overview of the work HEFCE are currently commissioning which they are hoping will build a robust evidence base for research assessment” in LSE Impact Blog 12(17) 2014 entitled Time for REFlection: HEFCE look ahead to provide rounded evaluation of the REF

Let me add a suggestion, updated for REF2014, that I have made before (unheeded):

Scientometric predictors of research performance need to be validated by showing that they have a high correlation with the external criterion they are trying to predict. The UK Research Excellence Framework (REF) -- together with the growing movement toward making the full-texts of research articles freely available on the web -- offer a unique opportunity to test and validate a wealth of old and new scientometric predictors, through multiple regression analysis: Publications, journal impact factors, citations, co-citations, citation chronometrics (age, growth, latency to peak, decay rate), hub/authority scores, h-index, prior funding, student counts, co-authorship scores, endogamy/exogamy, textual proximity, download/co-downloads and their chronometrics, tweets, tags, etc.) can all be tested and validated jointly, discipline by discipline, against their REF panel rankings in REF2014. The weights of each predictor can be calibrated to maximize the joint correlation with the rankings. Open Access Scientometrics will provide powerful new means of navigating, evaluating, predicting and analyzing the growing Open Access database, as well as powerful incentives for making it grow faster.

Harnad, S. (2009) Open Access Scientometrics and the UK Research Assessment Exercise. Scientometrics 79 (1)
(Also in Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics 11(1), pp. 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds. 2007)

See also:
The Only Substitute for Metrics is Better Metrics (2014)
and
On Metrics and Metaphysics (2008)
REF2014 gives the 2014 institutional and departmental rankings based on the 4 outputs submitted.

That is then the criterion against which the many other metrics I list below can be jointly validated, through multiple regression, to initialize their weights for REF2020, as well as for other assessments. In fact, open access metrics can be — and will be — continuously assessed, as open access grows. And as the initialized weights of the metric equation (per discipline) are optimized for predictive power, the metric equation can replace the peer rankings (except for periodic cross-checks and updates) -- or at least supplement it.

Single metrics can be abused, but not only can abuses be named and shamed when detected, but it becomes harder to abuse metrics when they are part of a multiple, inter-correlated vector, with disciplinary profiles of their normal interactions: someone dispatching a robot to download his papers would quickly be caught out when the usual correlation between downloads and later citations fails to appear. Add more variables and it gets even harder.

In a weighted vector of multiple metrics like the sample I had listed, it’s no use to a researcher if told in advance that for REF2020 the metric equation will be the following, with the following weights for their particular discipline:

REF2020Rank =

w1(pubcount) + w2(JIF) + w3(cites) +w4(art-age) + w5(art-growth) + w6(hits) + w7(cite-peak-latency) + w8(hit-peak-latency) + w9(citedecay) + w10(hitdecay) + w11(hub-score) + w12(authority+score) + w13(h-index) + w14(prior-funding) +w15(bookcites) + w16(student-counts) + w17(co-cites + w18(co-hits) + w19(co-authors) + w20(endogamy) + w21(exogamy) + w22(co-text) + w23(tweets) + w24(tags), + w25(comments) + w26(acad-likes) etc. etc.

The potential list could be much longer, and the weights can be positive or negative, and varying by discipline.

"The man who is ready to prove that metric knowledge is wholly impossible… is a brother metrician with rival metrics…”

"The UK funding councils have narrowed the scope of their proposed open access mandate for the post-2014 research excellence framework."
-- (Paul Jump, "Open access mandate narrowed in formal proposals" Times Higher Education)
.

1. Model. The HEFCE proposal to mandate immediate (not retrospective) deposit of journal articles in the author's institutional repository in order to make them eligible for evaluation in the next Research Excellence Framework (REF) is wise and timely, and, if adopted, will serve as a model for the rest of the world. It will also complement the Green (self-archiving) component of the RCUK Open Access (OA) mandate, providing it with an all-important mechanism for monitoring and ensuring compliance.

2. Monographs. Exempting monographs for now was a good decision. The HEFCE mandate, like the RCUK mandate, applies only to peer-reviewed journal articles. These are all author giveaways, written solely for research impact, not royalty income. This is not true of all monographs. (But a simple compromise is possible: recommend -- but don't require -- monograph deposit too, but with access set as Closed Access rather than Open Access, with no limit on the length of the OA embargo. Author choice.)

3. Data. Ditto for open data: It's good judgment not to force it on researchers. Researchers must be allowed a fair period of first-expoitation rights on the data they have gathered. If it's immediately open to all, why bother to gather data at all? Just analyze the data of others immediately after they take the time and trouble to gather it. (But here too, a simple compromise would be to recommend -- but not require -- Closed Access deposit. Eventually, fair embargo length limits can be decided, on a discipline by discipline and project by project basis.)

4. Exceptions. The required compliance rate has not been reduced from 100% to 60-75% (and should not be). HEFCE is merely asking in the consultation, whether the research community prefers a reduced target percentage or case-by-case consideration of exceptions. The latter is a far better way of making the policy realistic and successful. Most of the notional reasons for non-compliance (e.g., publisher embargoes) are based on misunderstandings anyway. (Articles can be deposited immediately, even if there is a publisher OA embargo: access to the immediate-deposit can be set as Closed Access instead of OA during the embargo.) Percentage-targets would simply ensure that compliance rates were no higher than the allowable percentages.

5. Embargoes. The HEFCE mandate moots OA embargoes because it requires immediate deposit, whether or not access is immediately OA. This is the core reason the HEFCE mandate is so very important and provides an optimal mandate model for the rest of the world: Publisher OA embargoes no longer determine whether and when an article is deposited. And the institutional repositories have an eprint request Button with which individual users worldwide can request a single copy of a Closed Access article for research purposes with one click; and the author can choose to comply or not comply with one click. This tides over research needs during any allowable OA embargo with "Almost-OA."

6. Licenses. Once the allowable embargo (if any) elapses, any OA deposit can be accessed, read, searched, linked, downloaded, stored, printed off or locally data-mined by any user webwide. It will also be harvested and indexed for Boolean full text search by engines like Google. No further license is needed for any of this. Further re-use rights will come once effective Green OA mandates on the combined HEFCE/RCUK model are adopted globally by funders and institutions worldwide. Universal Green OA will also hasten the inevitable natural demise of all remaining OA embargoes.

7. Start-Date. The HEFCE consultation also inquires about when the mandate should start, and contemplates a grace period of two years, from 2014-2016. But there is really no reason why an immediate-deposit mandate for REF 2020 should not start immediately after REF 2014 for authors at UK institutions, for any article accepted after that date: Everyone begins preparing for the new REF the day after the old REF anyway.

8. Date-Stamp: Only one of the consultation questions is critical for the success of the HEFCE mandate model, and that is whether the requirement that the deposit be "immediate" refers to the date of publication or the date of acceptance for publication. It is crucially important that the date should be acceptance, not publication. Acceptance date is marked by a determinate date-stamped acceptance letter and is a natural point for deposit in the author's workflow. Authors usually don't even know when their accepted article will appear, or has appeared; the lag may be months or even years from acceptance. Nor is the date on the journal issue a marker, because issues often appear long after their calendar dates. Publication lags can be even longer than OA embargoes! Meanwhile, precious access and impact are being lost. The HEFCE immediate-deposit mandate will only succeed if it is pegged to the determinate acceptance date rather than the indeterminate publication date.

[Please post your own response to the HEFCE REF OA Policy Consultation HERE]

Executive Summary:

I. The HEFCE proposal to mandate immediate repository deposit of articles as a precondition for eligibility for REF is excellent. If adopted and effectively implemented, it will serve as a model for OA mandates worldwide. It will also reinforce and complement the RCUK OA mandate, providing it with a uniform compliance monitoring and verification mechanism.

II. The immediate-deposit mandate should apply to the refereed, accepted version of peer-reviewed research articles (or refereed conference articles).

III. The deposit should be in the author’s institutional repository, immediately upon acceptance for publication. Acceptance date is determinate; publication date is variable and indeterminate and may lag acceptance by as much as two years.

IV. Access to the deposit should be immediately OA where possible, or, where deemed necessary, it can be made Closed Access if the publisher requires an OA embargo.

V. Repositories should implement the eprint request Button that allows individual users to request – and others to provide – one copy for research purposes with one click each.

VI. Once any allowable embargo period elapses, OA deposits can be accessed, read, searched, linked, downloaded, printed out, stored, and locally data-mined by individual users, as well as harvested and indexed for Boolean search by harvesters like Google. This makes license policy less urgent. Further re-use rights will come when OA mandates have made OA universal.

VII. What is crucial is that the deposit should be made at time of acceptance, time-stamped as such, with a copy of the acceptance letter to serve as the date marker.

VIII. Unlike articles, monographs are not all author give-aways, published solely for research impact rather than royalty outcome; and researchers need to have exclusive first data-mining rights on the data they collect. So monograph and data deposit should only be recommended for the time being, not mandatory; access to the deposits can be set as Closed Access.

IX. The start date for 2020 REF eligibility should be immediately after the 2014 REF, not two years afterward.

X. The target should be 100% compliance. Exceptions can be dealt with on a case by case basis: It would be a great mistake to stipulate a percentage compliance figure instead.

---

Question 1: Do you agree that the criteria for open access are appropriate (subject to clarification on whether accessibility should follow immediately on acceptance or on publication)?

YES

1.1 The HEFCE REF OA Policy should apply to the refereed, accepted version of peer-reviewed research articles or refereed conference articles.

1.2 It should be deposited in the author’s HEI repository, immediately upon acceptance for publication.

1.3 Access to the deposit should be immediately Open Access where possible, or, where deemed necessary, it can be made Closed Access if the publisher requires an OA embargo.

1.4 The crucial thing is that the deposit should be made at time of acceptance, time-stamped as such, with a copy of the acceptance letter to serve as the date marker.

The proposal is excellent. And if adopted and effectively implemented, it will serve as a model for OA policies worldwide.

Question 2a: Do you agree with the role outlined for institutional repositories, subject to further work on technical feasibility?

YES

Fortunately, most UK HEI institutions already have institutional repositories (IRs) that are already configured, or readily configurable, to be compliant with HEFCE’s proposed policy for REF. They also already have a date of deposit tag. The dated acceptance letter can be uploaded as a supplementary document. The full text can be uploaded with access set as either Open Access or Closed Access (during an embargo, in which case the repositories also have a facilitated eprint request Button that can tide over the usage needs of UK and worldwide researchers for the deposited research during the allowable embargo).

Many HEIs are already use their IRs for submission to REF. The only change required by the HEFCE policy will be to require the deposit to be made immediately upon acceptance, rather than in batch, at the end of the year, or the end of the REF cycle. But this is the crucial core of the policy (and what will also make it an effective compliance mechanism for the RCUK Mandate as well).

The IR software is also easily configurable so researchers can keep updating their REF choices as they publish further articles, substituting a later one for an earlier one, if they judge it more suitable for REF. What is brilliant about the HEFCE proposal is that it ensures that all potentially suitable articles are deposited immediately, in order to ensure that they are eligible, even if they might later be superseded by a more suitable article.

Question 2b: Should the criteria require outputs to be made accessible through institutional repositories at the point of acceptance or the point of publication?

Deposit should definitely be required at point of acceptance rather than at point of publication, for the following reasons:

1. The point of acceptance has a definite date, with the editor’s dated letter of acceptance serving as the time marker.

2. The point of acceptance is also the natural point in the author’s workflow to do the deposit, again marked by a clear, unambiguous, dated event: the letter of acceptance for publication.

3. The date of publication is extremely vague and uncertain for journals.

4. The author does not know, at point of acceptance, when the article will be published.

5. The publication date of the article often has no calendar date.

6. The publication date usually does not correspond to the date at which an article actually appears: the article may appear earlier than the publication date, but more often it appears later, sometime very much later.

7. The author often only finds out the date of publication after the fact – sometimes long after the fact.

8. All these possibilities are vague and uncertain, and the span of uncertainty can be from several months to two years or even more, which is even longer than most publishers’ OA embargo length.

9. Hence publication date is no basis for reliably and systematically complying with a HEFCE immediate-deposit requirement by the author, nor for monitoring and ensuring fulfilment by the author’s HEI or by HEFCE.

10. A further advantage of the acceptance date is that it is earlier, and hence allows more and earlier access and usage of the funded research.

IR deposit, at point of acceptance, is a simple, clear, natural, readily implementable and verifiable procedure for the author, the HEI and HEFCE, as well as an excellent compliance verification mechanism for the RCUK OA mandate. It is also an optimal model for the rest of the research world to adopt globally. With it, HEFCE will be performing a great service not only for UK and worldwide access to UK research output, but also for UK access to the rest of the world’s research output, with an exemplary policy, suited for use by all.

Question 3a: Do you agree that the proposed embargo periods should apply by REF main panel?

NEUTRAL

The length of the embargo is far less important than the requirement to deposit in the author’s institutional repository, and to deposit immediately upon acceptance.

Embargoes should be as short as possible, but they can, if desired, be allowed to vary by discipline. The IRs have the facilitated eprint request Button to help tide over the usage needs of UK and worldwide researchers for the deposited research during the allowable embargo.

Question 3b: Do you agree with the proposed requirements for appropriate licences?

NO

It is not clear from the documentation what these license/re-use requirements will be. I strongly urge not get bogged down in them. We are talking here about UK research output. Once it is deposited and any embargo elapses, deposits will be OA and hence can be searched, linked, downloaded, printed, stored and text-mined by individual researchers and research groups. They will also be harvested and full-text inverted for Boolean search by Google and other harvesters. All of this comes with the territory in making them Open Access, and does not require any further license.

What would require further license permissions would be the right for databases to harvest, data-mine and republish the texts. Do not get bogged down in this now, if it creates any obstacles. We are only talking about UK research output: 6% of worldwide research output. If the rest of the world adopts the HEFCE immediate-deposit requirement too, OA will become 100% globally, and all re-use rights authors wish to provide and users need will follow soon after. But it would be a needless risk to let licensing requirements hold back adoption or compliance of the HEFCE OA policy at this point. And there are discipline differences here too, potentially even bigger ones than differences in embargo length.

Go easy on licensing: It will all come after the HEFCE policy succeeds and is adopted worldwide. Don’t let licenses and re-use rights become a sticking point even before the HEFCE mandate is adopted. Access is infinitely more urgent than re-use/license needs; access needs are universal across disciplines; re-use/license needs are not. And access is a prerequisite for re-use rights, not vice versa. First things first.

Be flexible and pragmatic on licensing. Immediate IR deposit is the crucial thing.

Question 4: Do you agree that the criteria for open access should apply only to journal articles and conference proceedings for the post-2014 REF?

YES

Refereed journal articles and refereed conference articles have from its inception been the primary targets of the worldwide Open Access movement, because they are the only form of research output that is, without exception, author giveaway content, written only for research uptake and impact, not for royalty revenue.

It is for this reason that all authors of articles will readily comply with an OA mandate: They all want their findings to be accessible to all their potential users worldwide, not just to those at institutions that can afford subscription access to the journal in which it happens to be published.

For researchers, loss of access to their work means loss of uptake, usage, applications and impact for their work. And the progress and funding of their research, as well as their careers, depend on the uptake, usage, applications and impact of their work.

Books. But all of this becomes much more complicated and exception-ridden when we move to monographs and books. Some books may fall in the same motivational framework, but many are written in hope of royalty income, so authors are not eager to give them away free for all. Also the economics of book publication entail a much bigger investment in each book by the publisher, who would likewise be reluctant to make the investment if the book was made available as an online give-away.

But there is a simple solution for books: Don’t require them to be deposited, just recommend it. And authors have the option of depositing books as Closed Access rather than Open Access, with no limit on how long they can embargo OA. (Meanwhile, if they wish, they can provide individual copies via the Button as and when they choose.)

Data. Data are complicated in another way. The problem is not potential royalties but first-exploitation rights. Researchers are not just data-gatherers. They gather data because they want to do something with it. To analyze and process it. They must be given a fair allotment of time to do this. Otherwise, if they must make their data open to all immediately, so anyone can analyze it, then they may as well not bother gathering it at all, and simply wait to analyze the data that others have taken the time and trouble to gather – and were then obliged to make open immediately.

The moral is that if article embargo lengths and licensing needs vary from discipline to discipline, then the fair length of the period of exclusive first-exploitation rights for data varies even more, not just from discipline to discipline, but from research project to research project.

And again the solution is to encourage (but not require) depositing the data and making it open as soon as possible. But no fixed embargo lengths.

A successful HEFCE immediate-deposit policy for refereed journal and conference articles will be an enormous positive contribution, and more than enough as a first step. All the rest (re-use rights, the gradual disappearance of article OA embargoes, and the extension of OA to other kinds of content) will follow as a natural matter of course. It should not be allowed to complicate what is otherwise an extremely timely and powerful means of making UK research articles OA.

Question 5: Do you agree that a notice period of two years from the date of the policy announcement is appropriate to allow for the publication cycle of journal articles and conference proceedings?

NO

I think two years is needlessly and unjustifiably long.

We are still now in the phase of REF 2014. As soon as that ends, researchers and HEIs begin to prepare for REF 2020.

There is no reason at all why immediate-deposit upon acceptance for articles accepted for publication starting 2014 should not begin in 2014 rather than in 2016, as a condition for REF 2020 eligibility.

Not even those HEIs that don’t yet have IRs should be exceptions: Their authors can start depositing at once in OpenDepot, the UK back-up repository designed for that purpose.

That said, there is no reason why HEFCE cannot show some flexibility in the first two years, for inadvertent failures to comply immediately. But this potential flexibility should not be publicized, for it will only encourage lax compliance during the two designated years.

Question 6: Do you agree that criteria for open access should apply only to those outputs listing a UK HEI in the output’s ‘address’ field for the post-2014 REF?

NO

Every UK researcher who is submitting an article for REF should have to deposit it in their IR immediately upon acceptance (except if they came to the institution after the acceptance date).

Better to be as inclusive as possible and handle would-be exceptions on a case by case basis rather than declare explicit exceptions.

Question 7: Which approach to allowing exceptions is preferable?

I support Option a: Full compliance; exceptions considered on case by case basis, first by the HEI, and if not resolved, by the REF panel.

There will be no basis for objections by publishers to immediate-deposit in Closed Access. The embargo length for Open Access is less important (because of the Button) and will not (and should not) constrain authors’ choice of journals).

External collaborators will certainly not object to Closed Access immediate-deposit, and are very unlikely to object to OA either – and certainly not post-embargo OA.

Percentage compliance criteria would be a very bad idea, and would virtually be inviting institutions not to strive for 100%. Case-by-case handling is an infinitely better way to exercise flexibility.

David Sweeney's new HEFCE/REF OA mandate proposal for consultation comes very close to providing the optimal OA mandate model:

(1) It separates the date on which deposit must be made (immediately upon acceptance for publication, with no differences across disciplines) from the date on which the deposit must be made OA (preferably immediately, but, at the latest, within an allowable embargo whose length will be adapted to the needs of each discipline).

(2) It specifies that the deposit must be made in the author's institutional repository (whence the metadata can be exported or harvested to institution-external discipline repositories immediately -- and the full-text once any embargo has elapsed).

(3) It makes immediate-deposit (but not immediate-OA) an eligibility precondition for submission to research evaluation (REF), thereby (very sensibly) recruiting institutions in monitoring and ensuring timely compliance with the mandate.

(4) It expresses no preference for gold OA publishing, leaving authors free to publish in whatever journal they choose.

(5) It expresses a preference for licensing certain re-use rights, but again leaves this to author choice.

I have been a strident critic of the Willetts/Finch/RCUK policy's preference for gold over green and its constraints on authors' freedom of journal choice. This new HEFCE mandate proposal would remedy all that and would make the UK's OA mandate once again compatible with green OA mandates the world over -- indeed, with (3) and (4) it provides the all-important compliance-verification mechanism that most OA mandates still lack.

I hope that once they have seriously reflected upon and understood this new mandate proposal, researchers and their institutions will see that it moots all the objections that have been raised to the Finch/RCUK mandate. And I profoundly hope that David Willetts will realize and understand that too.

I also hope that those who are impatient for immediate, embargo-free OA, CC-BY licenses and Gold OA will allow this HEFCE compromise mandate to be adopted and succeed, rather than trying to force their less urgent, less universal, and much more divisive conditions into the policy yet again.

The price of Green OA (per paper deposited) is negligibly small, compared to Gold OA. And institutional repositories are already created and paid up (for a variety of purposes) but they remain near-empty of their target OA content -- unless deposit is mandated.

Green deposit mandates have to have carrots and sticks to be effective. Funder mandates provide the carrot/stick for institutions (funding eligibility -- and enhanced impact -- if you deposit; ineligibility if you don't)

Double-paying publishers pre-emptively for gold now is fine -- if you have effectively mandated a green deposit mandate for all articles first (and you have the extra cash to double-pay publishers for subscriptions and gold).

But if you have not effectively mandated a green deposit mandate for all articles first, instead double-paying publishers pre-emptively for gold is not only a gratuitous waste of scarce research money, but a counterproductive retardant on OA growth, both in the UK and worldwide (in encouraging subscription publishers to offer hybrid gold and to increase their embargo lengths on green in order to ensure that UK authors must pick and pay for gold).

(Where gold [or a fee waiver] is offered for free to authors (& their institutions) by a journal they freely choose as suitable, authors are of course welcome to choose it -- as long as they also deposit their article in their Green OA institutional repository, just as everyone else is mandated to do.)

Global green OA grows anarchically, not journal by journal. If and when competition from green starts causing journal cancellations, journals will be forced to start cutting costs by downsizing, phasing out the obsolete print and online edition and offloading all access-provision and archiving onto the global network of green OA institutional repositories. The institutional cancellation savings will then (single-) pay for post-Green Fair Gold at an affordable, sustainable price (for peer review alone).

To instead double-pay publishers pre-emptively for gold now (in the name of "cushioning" the transition) while publishers promise to "plough back" all Gold OA double-payment into subscription savings (all publishers? all subscribers?) is simply to give publishers a license to keep charging as much as they like and never bother to do the cost-cutting and downsizing that universal mandatory green would force them to do.

If the UK double-pays for Gold pre-emptively rather than first effectively mandating Green for all UK research output, it has chosen the losing option in an unforced Prisoner's Dilemma: the UK loses and the rest of the world gains. Less an admirable moral stance or idealism or a "front-mover" advantage than an unreflective and somewhat stubborn rush for Fool's Gold.

Mark Thorley wrote: "Stevan, ...As an advocate of Open Access I would like to think that you appreciate the fact that the UK is leading the world here..."

Mark, no, the UK is no longer leading the world with its new Finch/RCUK/BIS OA policy.

It's time to heed OA advocates that have been at this far longer than you, and fix the RCUK Policy:

Peter Suber: "The UK can do better. In fact, the RCUK can do better. Its 2006 policy was better than the new policy. It only needed to be enforced."

The RCUK Policy is fixable. Indeed it can be made much better than the old RCUK policy. And the UK can once again take the worldwide lead in OA Policy:

I. Drop the 9 words that make the RCUK Policy say the opposite of what it means.

II. Adopt an effective compliance-verification mechanism for Green OA self-archiving:

(IIa) Deposit must be in the fundee's institutional repository.
      (This makes each UK institution responsible for monitoring and verifying timely compliance.)
(IIb) All articles must be deposited immediately upon acceptance for publication.
      (Publisher embargoes apply only to the date on which the deposit is made OA.)
(IIc) Repository deposit must be designated the sole mechanism for submitting publications for UK research assessment (REF).
      (Articles' deposit URL required in all requests for RCUK funding or renewal.)

It is still widely hoped that RCUK will act in a flexible, constructive way rather than a rigid, dogmatic one, in the face of the growing expression of the concerns of the research community and its OA advocates, in the UK and worldwide, about the ambiguity and the potential for perverse effects of the new RCUK OA Policy.

Stevan Harnad

BOAI-10 RECOMMENDATIONS
-- 1.1. Every institution of higher education should have a policy assuring that peer-reviewed versions of all future scholarly articles by faculty members are deposited in the institution’s designated repository...

-- Deposits should be made as early as possible, ideally at the time of acceptance, and no later than the date of formal publication.

-- University policies should respect faculty freedom to submit new work to the journals of their choice. [emphasis added]

-- University policies should encourage but not require publication in OA journals [emphasis added] ...

-- 1.3. Every research funding agency, public or private, should have a policy assuring that peer-reviewed versions of all future scholarly articles reporting funded research are deposited in a suitable repository and made OA as soon as practicable.

-- Deposits should be made as early as possible, ideally at the time of acceptance, and no later than the date of formal publication...

U Liege's Rector, Bernard Rentier, reports that over the past year deposits to the U. Liege repository (ORBi) have grown from 10 to 40 thousand publications, 25 thousand of them full-text. According to ROAR, this is the 3rd highest growth rate among the world's thousand identified institutional repositories. Viewed 650 thousand times and downloaded 61 thousand times, these 40 thousand deposits coincide with the first year in which, as a part of U Liege's Open Access Mandate, ORBi has served as U Liege's sole official means of submitting publications for performance review for academic promotion.

Universities UK recommends making all the research outputs submitted to the UK's new Research Excellence Framework (REF) Open Access (OA).

The UUK's recommendation is of course very welcome and timely.

All research funded by the RCUK research councils is already covered by the fact that all the UK councils already mandate OA. It is this policy, already adopted by the UK, that the US is now also contemplating adopting, in the form of the proposed Federal Research Public Access Act (FRPAA), as well as the discussion in President Obama's ongoing OSTP Public Access Policy Forum.

But if HEFCE were to follow the UUK's recommendation, it would help to ensure Open Access to UK research funded by the EU (for which OA is only partially mandated thus far) and other funders, as well as to unfunded research -- for which OA is mandated by a still small but growing number of universities in the UK and worldwide. (The same UUK proposal could of course be taken up by UK's universities, for once they mandate OA for all their research output, all UK research, funded and unfunded, becomes OA!)

There is an arbitrary constraint on REF submissions, however, which would greatly limit the scope of an OA requirement (as well as the scope of REF itself): Only four research outputs per researcher may be submitted, for a span covering at least four years, rather than all research output in that span.

This limitation arises because the REF retains the costly and time-consuming process of re-reviewing, by the REF peer panels, of all the already peer-reviewed research outputssubmitted. This was precisely what it had earlier been proposed to replace by metrics, if they prove sufficiently correlated with -- and hence predictive of -- the peer panel ranklings. Now it will only be partially supplemented by a few metrics.

This is a pity, and an opportunity lost, both for OA and for testing and validating a rich and diverse new battery of metrics and initializing their respective weights, discipline by discipline. Instead, UUK has endorsed a simplistic (and likewise untested and arbitrary) a-priori weighting ("60/20/20 for outputs, impact and environment").

Harnad, S. (2009) Open Access Scientometrics and the UK Research Assessment Exercise. Scientometrics 79 (1) Also in Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics 11(1), pp. 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds. (2007)

Commentary on:
Collini, S. (2009) Impact on humanities: Researchers must take a stand now or be judged and rewarded as salesmen. Times Literary Supplement. November 13 2009.

One can agree whole-heartedly with Professor Collini that much of the spirit and the letter of the RAE and the REF and their acronymous successors are wrong-headed and wasteful -- while still holding that measures ("metrics") of scholarly/scientific impact are not without some potential redeeming value, even in the Humanities. After all, even expert peer judgment, if expressed rather than merely silently mentalized, is measurable. (Bradley's observation on the ineluctability of metaphysics applies just as aptly to metrics: "Show me someone who wishes to refute metaphysics and I'll show you a metaphysician with a rival system.")

The key is to gather as rich, diverse and comprehensive a spectrum of candidate metrics as possible, and then test and validate them jointly, discipline by discipline, against the existing criteria that each discipline already knows and trusts (such as expert peer judgment) so as to derive initial weights for those metrics that prove to be well enough correlated with the discipline's trusted existing criteria to be useable for prediction on their own.

Prediction of what? Prediction of future "success" by whatever a discipline's (or university's or funder's) criteria for success and value might be. There is room for putting a much greater weight on the kinds of writings that fellow-specialists within the discipline find useful, as Professor Collini has rightly singled out, rather than, say, success in promoting those writings to the general public. The general public may well derive more benefit indirectly, from the impact of specialised work on specialists, than from its direct impact on themselves. And of course industrial applications are an impact metric only for some disciplines, not others.

Ceterum censeo: A book-citation impact metric is long overdue, and would be an especially useful metric for the Humanities.

Harnad, S. (2001) Research access, impact and assessment. Times Higher Education Supplement 1487: p. 16.

Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated online RAE CVs Linked to University Eprint Archives: Improving the UK Research Assessment Exercise whilst making it cheaper and easier. Ariadne 35.

Brody, T., Carr, L., Harnad, S. and Swan, A. (2007) Time to Convert to Metrics. Research Fortnight pp. 17-18.

Harnad, S. (2008) Open Access Book-Impact and "Demotic" Metrics Open Access Archivangelism October 10, 2008.

Harnad, S. (2008) Validating Research Performance Metrics Against Peer Rankings. Ethics in Science and Environmental Politics 8 (11) doi:10.3354/esep00088 Special Issue on "The Use And Misuse Of Bibliometric Indices In Evaluating Scholarly Performance"

Harnad, S. (2009) Open Access Scientometrics and the UK Research Assessment Exercise. Scientometrics 79 (1)