Re: UK "RAE" Evaluations

From: Tom Wilson <t.d.wilson_at_SHEFFIELD.AC.UK>
Date: Thu, 23 Nov 2000 10:21:53 -0000

As a member of the LIM panel in 1996, my understanding was that the HEFC had
issued instructions that electronic documents were to be evaluated in the
same way as printed documents, and that this applied to all subject areas. I
had assumed that this ruling stood for the 2001 Assessment.

Tom Wilson

----- Original Message -----
From: "Fytton Rowland" <J.F.Rowland_at_lboro.ac.uk>
To: <lis-elib_at_mailbase.ac.uk>; <american-scientist-open-access-forum_at_amsci.org>
Sent: Wednesday, November 22, 2000 10:20 AM
Subject: Re: UK "RAE" Evaluations


> This issue may vary between disciplines. The UK Research Assessment
> Exercise is carried out by subject panels (made up of senior academics in
> the discipline), who have considerable independence in how they make their
> assessments. Whatever the basic rules of the RAE may say, individual
> subject panels may well apply different rules. The Library and
Informatiom
> Management panel has no problem with electronic journals without page
> numbers, so far as I know; the quite well thought of Journal of Digital
> Information (jointly edited from Stevan's department and mine, as it
> happens) is certainly accepted by them, and it has no page numbers.
>
> Chemistry is a deeply conservative discipline, and probably the most
> obsessed with journal publication. But it should be possible to obtain a
> statement from the Chair of the Chemistry Panel about the eligibility of
> elctronic journals.
>
> Fytton Rowland.
>
>
> At 06:31 PM 11/21/00 +0000, Stevan Harnad wrote:
> >On Tue, 21 Nov 2000, "Rzepa, Henry" <h.rzepa_at_ic.ac.uk wrote:
> >
> >> The UK runs "Research assessment exercises" in subject disciplines
> >> periodically. Ours (Chemistry) is coming up shortly.
> >>
> >> I enquired what the status of my "e-only" work might be. The answer,
> >> approximately, was that if it does not carry a page number, its not a
> >> journal.
> >>
> >> Publication in "journals" is of course one way in which my particular
> >> discipline judges quality and international significance.
> >>
> >> As the editor of four "e-print" conferences in chemistry, archived as
> >> four CDROMs "without page numbers", I infer (to be confirmed) that
> >> none of this counts towards the RAE, or at least not at the same level
> >> as "page numbers".
> >>
> >> As it happens, I have hedged my bets, and published, hopefully
> >> adequately, in journals that DO carry page numbers. But if ones tenure,
> >> or future, were to depend on "page numbers", it could be a worrying
> >> period for some.
> >>
> >> If push came to shove, and someone were "shoved" because their "page
> >> numbers" were inadequate, I wonder if that could be tested legally, and
> >> whether the "e-print" would prevail?
> >
> >I seriously doubt that page-numbers are or ever were a stumbling block
> >with RAE. Here is the pertinent statement from the last RAE:
> >
> > 46. Material embodying research outcomes published by electronic
> > means may be cited. Refereed journal articles published through
> > electronic means will be treated on the same basis as those
> > appearing in printed journals.
> >
> > http://www.niss.ac.uk/education/hefc/rae96/c2_95.html
> >
> >This does not sound like any sort of double-standard to me.
> >
> >On the other hand, please bear and mind (and keep separate) the
> >following further issues:
> >
> >(1) For the RAE or any other evaluation, not all refereed journals are
> >on a par: There are journals of higher quality and lower quality; this
> >is sometimes (but not always) correlated with the journal's ISI "impact
> >factor," and sometimes also with its age (older, established journals
> >may tend to be of higher quality and impact than newer start-ups, or
> >may have established their eventual quality and prestige more fully).
> >
> >(2) These quality differences among journals can and and should be taken
> >into account if such assessment exercises are to be conducted at all.
> >It is then likely that newer start-ups will not be weighted as heavily
> >as older established journals; it is also likely that the online-only
> >journals will be among the newer start-ups, rather than the older,
> >established journals, which will all be paper-based (although most will
> >now also have an on-line version too). There are, however, exceptions
> >to the rule that newer (hence on-line-only) start-up journals will have
> >lower quality and impact, viz., The Journal of High Energy Physics
> ><http://jhep.sissa.it/>:
> >
> > "JHEP has proved to be the right answer to the need for a new path
> > for scientific communication. Indeed, JHEP is an efficient and
> > cheap alternative to conventional publishing that nevertheless
> > maintains essential publishing features: quality control, easy
> > retrieval, and archival responsibility for the future.
> >
> > "In order to illustrate the success of the experiment, launched in
> > July 1997, let the figures do the talking. JHEP is by now a leading
> > journal in the field. The number of consultations is outstanding
> > and the impact factor as high as the best in the field."
> >
> > http://library.cern.ch/HEPLW/1/papers/3/
> >
> >(3) It goes without saying that a refereed journal is a refereed
> >journal, regardless of the medium, on-line or on-paper. In some fields,
> >refereed conference proceedings are just about on a par with refereed
> >journals (but again, they need time to establish their quality levels
> >and impacts). What Dr. Rzepa neglects to indicate, however, is whether
> >the "'e-print' conferences in chemistry" of which he writes are (i)
> >refereed, and if so, (ii) whether they have yet established their
> >refereeing quality standards (and if so, (iii) where they stand in the
> >quality hierarchy).
> >
> >None of this is intended as advocacy of bean-counting. But if we are
> >going to count beans, there are factors that will invariably be taken
> >into account that prevent all publications from being treated on a par.
> >
> >The presence of page-numbers, however, will not be one of them.
> >
> >--------------------------------------------------------------------
> >Stevan Harnad harnad_at_cogsci.soton.ac.uk
> >Professor of Cognitive Science harnad_at_princeton.edu
> >Department of Electronics and phone: +44 23-80 592-582
> > Computer Science fax: +44 23-80 592-865
> >University of Southampton http://www.ecs.soton.ac.uk/~harnad/
> >Highfield, Southampton http://www.princeton.edu/~harnad/
> >SO17 1BJ UNITED KINGDOM
> >
> >NOTE: A complete archive of the ongoing discussion of providing free
> >access to the refereed journal literature online is available at the
> >American Scientist September Forum (98 & 99 & 00):
> >
> > http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
> >
> >You may join the list at the site above.
> >
> >Discussion can be posted to:
> >
> > american-scientist-open-access-forum_at_amsci.org
>
> **********************************************************
> Fytton Rowland, M.A., Ph.D., F.I.Inf.Sc., Lecturer,
> Deputy Director of Undergraduate Programmes and
> Programme Tutor for Publishing with English,
> Department of Information Science,
> Loughborough University,
> Loughborough, Leics LE11 3TU, UK.
>
> Phone +44 (0) 1509 223039 Fax +44 (0) 1509 223053
> E-mail: J.F.Rowland_at_lboro.ac.uk
> http://info.lboro.ac.uk/departments/ls/staff/frowland.html
> **********************************************************
Received on Mon Jan 24 2000 - 19:17:43 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:45:58 GMT