Open Access Archivangelism

---

SUMMARY: Data-mining robots like SciBorg can harvest Green OA full-texts, self-archived in their authors' Institutional Repositories (IRs) and “repurpose” them for better functionality. The postprint is the author’s own refereed, revised final draft. Green journal publishers endorse author posting of postprints in their own IR, free for all. The author can certainly revise that draft further, making additional corrections, updates and enhancements, including marking it up in XML and adding comments. Those corrections need not be done by the author's own hands: They could be done by a graduate student, a collaborator, a secretary, or a hired hand. The author could also have SciBorg “repurpose” his postprint -- under one trivial condition, easily fulfilled, which is that the locus of the enhanced postprint, the URL from which users must download it, remains the author’s own IR, not a 3rd-party website. It is not only unnecessary but would be highly inimical to the progress of Green OA mandates to insist instead that the Green publisher’s endorsement to self-archive the postprint in the author’s IR is "not enough" for full-blooded OA — that the author must also successfully negotiate with the publisher the retention of the right to assign to 3rd-party harvesters like SciBorg the right to publish a “derivative work” derived from the author’s postprint.

---

In "Why Green Open Access does not support text- and data-mining", Peter Murray-Rust wrote:

PM-R: "...the first thing to do is to gather a corpus of documents... any other scientist should be able to have access to it. It therefore has to be freely distributable..."

Agreed. So far this is just bog-standard OA. If the original documents are self-archived as Green OA postprints in their authors' Institutional Repositories (IRs), your SciBorg robot can harvest them and data-mine them, and make the results freely accessible (but linking back to the postprint in the author's IR whenever the full-text needs to be downloaded).

PM-R: "[At SciBorg] we are interested in machines understanding science..."

Fine. Let your SciBorg machines harvest the Green OA full-texts and "repurpose" them as they see fit.

PM-R: "almost all articles are copyrighted and non-distributable. Publisher Copyright is a major barrier... you can’t just go out and compile a wordlist or whatever as you may infringe copyright or invisible publisher contracts (we found that out the hard way)..."

You can't do that if you are harvesting the publisher's proprietary text, but you can certainly do that if you are harvesting the author's Green OA postprints.

PM-R: "PDFs are so awful... we have to repurpose them by converting to HTML, XML and so on..."

Fine.

PM-R: "Now the corpus is annotated. Expert humans go through line by line...It is this annotated corpus which is of most use to the scientific community..."

Fine.

PM-R: "So suppose I find 50 articles in 50 different repositories, all of which claim to be Green Open Access. I now download them, aggregate them and [SciBorg] repurpose[s] them. What is the likelihood that some publisher will complain? I would guess very high..."

Complain about what, and to whom? A Green publisher has endorsed the author's posting of his own Green OA postprint in his own IR, free for all. The postprint is the author's own refereed, revised final draft. Now follow me: Having endorsed the posting of that draft, does anyone imagine that the publisher would have any grounds for objection if the author revised the draft further, making additional corrections and enhancements? Of course not. It's exactly the same thing: the author's Green OA postprint.

So what if the author decides to mark it up as XML and add comments? Any grounds for objections? Again, no. Corrections, updates and enhancements of the author's postprint are in complete conformity with posting his postprint.

Suppose the author did not do those corrections with his own hands, but had a colleague, graduate student, a secretary, or a hired hand do them for him, and then posted the corrected postprint? Still perfectly fine.

Now suppose the author had your SciBorg "repurpose" his postprint: Any difference? None -- except a trivial condition, easily fulfilled, which is that the locus of the enhanced postprint, the URL from which users can download it, should again be the author's IR, not a 3rd-party website (which the publisher could then legitimately regard as a rival publisher -- especially if it was selling access to the "repurposed" text).

So the solution is quite obvious and quite trivial: It is fine for the SciBorg harvester to be the locus of the data-mining and enhancement of each Green OA postprint. It can also be the means by which users search and navigate the corpus. But SciBorg must not be the locus from which the user accesses the full-text: The "repurposed" full-text must be parked in the author's own IR, and retrieved from there whenever a user wants to read and download it (rather than just to search and surf the entire corpus via SciBorg).

Not only does this all sound silly: it really is silly. In the online age, it makes no functional difference at all where a document is actually physically located, especially if the document is OA! But we are still at the confused interface between the paper age and the OA era. So we have to be prepared to go through a few silly rituals, to forestall any needless fits of apoplexy, which would otherwise mean further dysfunctional delay (for OA).

So the ritual is this: It would be highly inimical to the progress of Green OA mandates to insist that the publisher's endorsement to self-archive the postprint in the author's IR is "not enough" -- that the author must also successfully negotiate with the publisher the retention of the right to assign to 3rd-party harvesters like SciBorg the right to publish a "derivative work" derived from the author's postprint. That would definitely be the tail wagging the dog, insofar as OA is concerned, and it would put authors off providing Green OA (and hence put their institutions off mandating it) for a long time to come.

Instead, when SciBorg harvests a document from a Green OA IR, SciBorg must make an arrangement with the author that the resultant "repurposed" draft will be deposited by the author in the author's own IR as an update of the postprint. Then, whenever a user of SciBorg wishes to retrieve the "repurposed" draft, the downloading site must always be the author's IR: no direct retrieval from the SciBorg site.

This ritual is ridiculous, and of course it is functionally unnecessary, but it is pseudo-juridically necessary, during this imbecilic interregnum, to keep all parties (publishers, lawyers, IP specialists, institutions, authors) calm and happy -- or at least mutely resigned -- about the transition to the optimal and inevitable that is currently taking place. Once it's over, and we have 100% Green OA, all this papyrophrenic horseplay can be well-deservedly dropped for the nonsense it is.

Please, Peter, be prepared to adapt SciBorg to the exigencies of this all-important (and all too slow-footed) transitional phase, rather than trying to force-fit the status quo to SciBorg, at the cost of still more delays to OA.

PM-R: "Only a rights statement actually on each document would allow us to create a corpus for NLP without fear of being asked to take it down..."

No. Green OA authors with standard copyright agreements are not in a position to license republication rights to SciBorg or any other 3rd party. Let us be happy that they have provided Green OA at all, and let SciBorg be the one to adapt to it for now, rather than vice versa.

Brody, T., Carr, L., Gingras, Y., Hajjem, C., Harnad, S. and Swan, A. (2007) Incentivizing the Open Access Research Web: Publication-Archiving, Data-Archiving and Scientometrics. CTWatch Quarterly 3(3).

Stevan Harnad
American Scientist Open Access Forum

---

SUMMARY: Green OA self-archiving removes not just "most" but all the "permission barriers" pertinent to research use.
   If "naked" (unlicensed) content on the web were really a barrier to use, we would be hearing about an urgent need to license all web content (e.g. advertisements, blogs). But the de facto default assumption since the advent of the web has been that if something is freely accessible online then you can download, print, store and otherwise "re-use" it.
   Far from perceiving themselves as being stuck behind "permission barriers" when they surf the web except when explicitly licensed to do otherwise, web users usually (wrongly) assume that they have even more rights than what comes with the free online territory: They will, for example, not only read, download, store, and print-off copyrighted web images, but also re-post them online, identically or in "derivative works," and sometimes even re-publish them in print publications. (You actually have to have shrill "You may NOT" notices to try to discourage users from going overboard like that!)
   Hence the "problem", if any, is precisely the reverse of what is being imagined by those who think that self-archived OA content risks not being used beyond on-screen reading unless it provides a formal permissive license over and above just being there, free for all on the web.
   The right to re-publish the verbatim text, online or on-paper, is not part of OA and never was, just as plagiarism or re-publishing a corrupted "derivative" version of the text is not and never was a part of OA, online or on-paper. OA's "unprecedented public good" is: free online access to what used to require paid on-paper access. This does not license re-publication or "derivative works" (collected or cut-pasted from the verbatim text), online or on paper (any more than it licenses the re-publication of copyrighted images). But the remarkable property of the new technology is that, for OA, such rights are not needed! For anyone, anywhere, can access (and print off) the online text, any time. Re-publication is not licensed, it is mooted by the properties of the new medium.
   What is left is (1) harvesting, (2) data-mining and (3) "derivative works." Harvesting likewise comes with the territory. (Google did not have to be licensed to harvest the rest of the web, hence needs no special license for OA content.) The same is true for other forms of data-mining by harvesting robots. For the publication of "derivative works," the rule is much the same as for copyrighted web images: Re-publication of verbatim OA texts, online or on paper, is not allowed without permission -- but it is not needed either, because a collection of URLs pointing to the OA URL serves the same function as re-publication.
   Corrupted, improved or otherwise altered versions of the OA texts? Apart from attributed fair-use excerpts, that is not allowed without the author's permission either. But what would ever have made anyone think that the invention of a new technology that would allow free and unlimited access to authors' give-away texts would mean that authors would all want to license that those texts, besides being freely accessible to all users, should be alterable or corruptible, ad lib? Surely we are happier with requiring specific case-by-case permission for such further uses, rather than a blanket license under the guise of "OA"?
   "Fair use" (like "copying") was a paper-based notion. In the case of the online medium, "fair use" quite naturally, indeed unavoidably, expands to include everything else that comes with the freely accessible online territory -- but that only means access and personal use, not re-posting (online), re-publishing (on paper), or perturbing that freely accessible content (online or on paper) without the author's permission -- apart from freely re-distributing its URL.

---

In Open Access News, Peter Suber wrote (by way of reply to my posting):

PS: "Comments. I hope no one minds if I reprint my comments from June 12 2007 in which I responded in detail to a very similar post by Stevan:

"Stevan isn't saying that OA doesn't or shouldn't remove permission barriers. He's saying that removing price barriers (making work accessible online free of charge) already does most or all of the work of removing permission barriers and therefore that no extra steps are needed."

So far this is exactly correct -- except I would definitely say that Green OA self-archiving removes not just "most" but all the "permission barriers" pertinent to research use, which is what OA is all about. Remember that there is also a "permission barrier" to re-publishing in print, but OA is not and never was intended to address that!

PS: "The chief problem with this view is the law. If a work is online without a special license or permission statement, then either it stands or appears to stand under an all-rights-reserved copyright. The only assured rights for users are those collected under fair use or fair dealing. These rights are far fewer and less adequate than OA contemplates, and in any case the boundaries of fair use and fair dealing are vague and contestable."

If "naked" (unlicensed) content on the web is really a barrier to use, how come we are not hearing about the need to license all web content (e.g. advertisements, blogs) because people are otherwise afraid to download, print, store and otherwise "re-use" them? (Answer: Because people are doing all those things, without hesitation.)

I think the truth is the exact opposite! That the default option, if something is freely accessible on the web, is that it's fine to do all those other things that come with it, and then some. (You can't even view web content without downloading and "storing" at least for long enough to read, yet that downloading and storing are not explicitly licensed!)

Far from perceiving themselves as being stuck behind "permission barriers" when they surf the web except when explicitly license to do otherwise, web users usually (wrongly) assume that they have even more rights than what comes with the territory: They will, for example, not only read, download, store, and print copyrighted web images, but also re-post them online, identically or in "derivative" form, and sometimes even re-publish them in print publications, identically or in "derivative" form. You actually have to have shrill "you may not" notices to try to discourage them from going overboard like that!

Now I have no particular interest in these excesses one way or the other (either to cultivate them or to curb them). But I think that they clearly illustrate that the "problem", if any, is precisely the reverse of what is being imagined by those who think that self-archived OA content needs a formal permissive license over and above just being there, free for all on the web, otherwise it risks not being used beyond on-screen reading! All seven individual uses I described in my earlier posting (as well as the two robotic ones) can be and are being fully exercised for all Green OA content today -- and there just aren't any further uses to which OA can justifiably lay claim (as OA).

PS: "This legal problem leads to a practical problem: conscientious users will feel obliged to err on the side of asking permission and sometimes even paying permission fees (hurdles that OA is designed to remove) or to err on the side of non-use (further damaging research and scholarship). Either that, or conscientious users will feel pressure to become less conscientious. This may be happening, but it cannot be a strategy for a movement which claims that its central practices are lawful."

Paying permission fees for Green OA content? Paying whom?

I honestly cannot imagine who or what you have in mind here, Peter! I am conscientious about not re-using web-accessible images in re-postings or publications unless I know they are public domain or I have permission. But I (and every other researcher on the planet) don't give a second thought as to whether I may read, download, store, print-off, and re-use the contents -- but not re-post or re-publish the verbatim text (which is like the image) -- of journal articles we can access freely on the web.

I am not feigning puzzlement: I am truly baffled about why, when the reality is the exact opposite, OA advocates, of all people, would worry that web users might be too coy (or "conscientious") to do with OA texts exactly the same things that we all do with all other free web content -- and too coy or "conscientious" to do so specifically in the case OA texts, of all things, because they lack a formal license to do it (exactly as virtually all other web content lacks such a license!).

[Could it be, Peter, that when you think of "unlicensed OA content" you are thinking of hybrid Gray/Gold publishers, who take an author's money, but don't adopt the full Creative Commons License that that money has surely paid for? I would probably agree with you on such cases, but my paradigmatic case is not paid Gray/Gold OA but Green OA, where it does not matter what copyright transfer agreement an author has signed as long the journal endorses immediate Green OA self-archiving, as 62% already do. All the rest comes with the Green OA territory.]

I think part of the problem is that (some) OA advocates may indeed be over-reaching in what they are taking to be licensed by "OA". I make absolutely no bones about the fact that the right to re-publish the verbatim text, online or on-paper, is not part of OA and never was, just as plagiarism or re-publishing a corrupted "derivative" version of the text is not and never was a part of OA, online or on-paper. OA is a new capability opened up by a new medium. I quote your own stirring words:

"An old tradition and a new technology have converged to make possible an unprecedented public good. The old tradition is the willingness of scientists and scholars to publish the fruits of their research in scholarly journals without payment, for the sake of inquiry and knowledge. The new technology is the internet. The public good they make possible is the world-wide electronic distribution of the peer-reviewed journal literature and completely free and unrestricted access to it by all scientists, scholars, teachers, students, and other curious minds."

The unprecedented public good is free online access to what used to require paid on-paper access. This does not license re-publication or "derivative works" (cut-pasted from the verbatim text), online or on paper, but the remarkable property of the new technology is that it does not need to! For one of the other things that "comes with the [Green OA] territory" is that anyone, anywhere, can access (and print off) the online text, any time. Re-publication is not licensed, it is mooted. There is simply no need for it. (The worriers about licensing content for "course-packs" are still thinking the old way: OA content does not need a license to be put in a course-pack: The text does not need to be put in a course-pack! Only its URL does.)

So, having disposed of the red herring of a special license that is supposedly needed to allow downloading, storing, and printing-off of freely available web content, we now see that there is no need to license re-publishing either, either online or on paper. What is left? Harvesting, data-mining and derivative works. Harvesting, as noted, likewise comes with the territory. If google did not have to be licensed to harvest the rest of the web, why on earth would we imagine that it needed to be licensed to harvest OA content, of all things? Ditto for the data-mining by harvesting robots. (Individual data-mining on your downloaded copy is not even at issue.)

Now what about "derivative works"? Let's be specific: re-publication of verbatim OA texts, online or on paper? That is not allowed without permission -- but nor is it needed, because a collection of URLs does the trick just as well.

Altered or corrupted versions of the OA texts? Apart from attributed fair-use excerpts, that is not allowed without permission either. But what would ever have made anyone think that the invention of a new technology that would allow unlimited access to authors' give-away texts would mean that authors would all want to license that those texts, besides being accessible, should be alterable or corruptible, ad lib? Surely we are happier with requiring specific case-by-case permission for such further uses, rather than a blanket license under the guise of "OA"?

What uses are left that research or researchers could possibly want, as a general rule: Harvestability into a commercial, pay-to-use database? It seems to me that that is no longer an OA matter. Indeed, authors might prefer to license their content to free database providers in preference to commercial ones. But either way, that is not part of OA, which was about "completely free and unrestricted access to it by all scientists, scholars, teachers, students, and other curious minds." Nothing there about commercial interests.

PS: "This doesn't mean that articles in OA repositories without special licenses or permission statements may not be read or used. It means that users have access free of charge (a significant breakthrough) but are limited to fair use."

"Fair use" was a paper-based notion. In the case of the online medium, "fair use" quite naturally, indeed unavoidably, expands to include everything else that comes with the online territory. In the case of freely accessible web documents, that "fair use" simply includes downloadability, storeability, printability, and data-minability, for individuals; and, for harvesters: robotic harvestability, data-minability, and certain derivative services (though I would not venture to specify which, though they certainly include free boolean searchability). With the Green OA territory comes also the accessibility online to everyone everywhere, mooting forever all need for collections, course-packs, re-publications, or other such "derivative works," online or on paper. For individual "derivative works," some form of "fair use" criterion still has to apply to determine how much verbatim content is permissible without the original author's permission.

Stevan Harnad
American Scientist Open Access Forum