Re: The True Cost of the Essentials (Implementing Peer Review)

From: Andrew Odlyzko <odlyzko_at_DTC.UMN.EDU>
Date: Wed, 19 Dec 2001 23:04:57 -0600

A few very brief responses to Arthur Smith's comments on my posting:

> Arthur's comments
>> my original posting


>>
>> [On shifting costs back to authors' institutions]
>>
>> Bringing back secretaries to do basic typesetting does not make sense, as
>> almost all scholars find it easier to do this themselves. On the other hand,
>> I feel there will be increasing pressure to provide expert Web design as well
>> as editorial assistance to make articles easy to access and read. As papers are
>> increasingly accessed in their electronic preprint formats (as is documented
>> in various places, including my paper "The rapid evolution of scholarly
>> communication," which is available, along with other papers, at
>> <http://www.dtc.umn.edu/~odlyzko/doc/eworld.html>), the incentive for
>> scholars will be make those forms attractive for readers.

>But the reality is that we have an enormous range of authors who send papers,
>many of whom may have time and resources and capability to "make articles
>easy to access and read", but many of whom do not. A look at the statistics
>on articles we receive:

>http://ridge.aps.org/STATS/00geographic.html

>shows some of our journals have as little as 21% coming from US authors,
>less than 35% from authors in even nominally English-speaking countries
>(a good number of these come from India with rather variable quality of
>presentation). 15-20% or more come from Asia (mostly China and Japan).
>Even papers received from US institutions can vary quite widely
>in consistency. I don't know comparable statistics for arXiv.org, but
>you can see there quite a variety of presentation styles and skills
>(a sample paper I just brought up had all the figures upside down,
>for example) and the range of "raw materials" we receive seems to be
>even wider than is on display there.

>Now one of the things we try to do in copy-editing (along with bringing
>everything to a common tagged format) is to bring the articles
>we publish to some minimal "quality" level in the presentation,
>English/physics terminological usage, etc. I can't say this is
>done perfectly, but on the other hand I believe the consistency
>in format and presentation in the final published articles goes
>a long way to making sure that the relative merits of articles
>to the readers can be judged primarily on the content, not on
>enormous differences in presentation. As Andrew notes:

>> [...] Already [...] scholars in
>> some areas where getting a paper into a prestigioug conference was more
>> important than publishing it (theoretical computer science being the
>> prime example of that) were putting a lot of efforts into making their
>> submissions look nice.

>But is this a good thing for science? Should authors with the resources
>to do so be "selling" their research with flashy presentations, while other
>authors who invest their resources in actual research get ignored? We
>need to level the playing field somewhere; doing so at the point of
>publication through funds extracted from readers (or sponsors, no particular
>bias on my part there) ensures that authors from less privileged
>institutions are given equal billing, where the actual research
>performed warrants it.

Two points:

1. The conventional publication process does not "level the playing field"
all that much. It does, to some extent, after a paper passes through
peer review, but even there, in general the publisher does only a small
amount towards improving the presentation, basically just the provision
of what Arthur calls 'some minimal "quality" level.' Well-written papers
are easy to read after publication, while the terrible ones (often terrible
only because of the author(s) lack of knowledge of English require much
effort to understand. Further, there are many biases in the peer review
process. Well-known figures, or even not so well-known ones that come
from prestigious places, tend to do better than others. Quality of
presentation also appears to matter at that stage.

2. My comments were of the descriptive (and predictive) nature, not
prescriptive ones. I was referring to the incentives that influence
scholars, and are likely to shape evolution of research publications.
Free distribution of eprints has done much to level the playing field;
instead of a couple of dozen top experts from the most prestigious
schools getting a preprint, and everybody outside of that circle
having to wait a year for the paper to be published, now everyone has
access to the paper on arXiv (or similar server) at the same time.
However, that creates incentives for making the playing field less level.
(I wrote a paper back in 1996 entitled "The bumpy road of electronic commerce,"
which argued that we would not have that mythical 'frictionless capitalism'
as a result of the Internet, since there would be incentives to create
artificial barriers.) Remember that Harvard does not spend something
like $80 million per year on its libraries just out of concern for
preservation of the scholarly literature. Rather, Harvard uses the
quality of its library collection to attract faculty, students, and
donations. In the future, Harvard (and other institutions, I am
not picking on them) will have an incentive to provide their faculty
with help in making their papers more accessible. Yes, that will
make the field less level, but such is the world. I feel that
the field will still be more level than it was a decade ago, say.



>> In general, as we move towards a continuum of publication, it makes less
>> and less sense to concentrate the copyediting and other costs at the
>> formal publication stage. What I expect scholars will want is provision
>> of "clearly readable research" (in Arthur's words) from the very beginning.
>> It really is a "war for the eyeballs," in scholarly publishing as well as
>> in more commercially-oriented areas, as my papers and those of Steve
>> Lawrence demonstrage/

>My argument is simply that going in that direction is a bad idea for
>scholarly research, because it misdirects the resources and attention
>of scholars into issues of presentation, when their real focus should
>be the content of their scholarly research, and it penalizes researchers
>who focus on the latter at the expense of the former, or who may
>have no resources or skills to devote to it. Let a third party take
>care of the presentation aspects; perhaps not a publisher doing peer review,
>though peer review seems to me like an ideal way to judge whether
>an article warrants "equal billing" with other good research, or not.

>Now it can be argued how well we are actually doing in this area. Actual
>changes to the text of a manuscript are often very minimal. However,
>even steps such as getting the figures right-side up and positioning
>them logically among the text, making sure acronyms and uncommon terms
>are clearly spelled out somewhere, and of course our tagging efforts at
>linking citations etc., can make a huge difference to the reader, so
>time devoted to understanding the article is well-spent.

>Is this really something we want to lose, in favor of all-out
>"war for the eyeballs"? My imagination conjures up images of
>physicists plastering their results on billboards in an escalating
>war of presentation over content - but maybe there's an equilbrium
>"detente" point that doesn't actually take that much effort on the
>part of the author? The prospect does make me uncomfortable, but
>as Andrew points out, in some areas it seems to be already happening.
>What does experience teach us there? How is "science" actually
>faring under these conditions? Has anybody analyzed this sort of thing?

>More food for thought I hope. I've got a lot to read :-)

Well, scholars already devote much attention to presentation. I would
love to be able to toss out a few inarticulate remarks to a waiting
assistant, and to have them written up instantly in a smooth and
attractive style. Unfortunately, I haven't found a patron willing to
provide such an assistant! :-)

More seriously, a very large part of a scholar's life is devoted to
presentation (even aside from formal teaching), whether in written or
spoken form. Some type of balance between research and presentation
has always had to be struck.

Andrew




  -----Please note new address-----

  Andrew Odlyzko
  University of Minnesota
  Digital Technology Center
  1200 Washington Avenue South
  Minneapolis, MN 55415

  odlyzko_at_umn.edu email
  612-624-9510 voice phone
  612-625-2002 fax

  http://www.dtc.umn.edu/~odlyzko
Received on Thu Dec 20 2001 - 10:31:10 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:46:21 GMT