The Open Access (OA) citation advantage has been
repeatedly demonstrated for Green OA, that is, articles published in any journal at all, but made open access by their authors by self-archiving them free for all online.
But the significant citation advantage for OA articles over non-OA articles -- which has been found in every field tested -- is based on comparing like with like: journal articles appearing in the same journal and year, and even sometimes matched for topic via title words.
Testing for a citation advantage of OA journals (Gold OA) over non-OA journals ("
EC study finds low citation gains for gold open access") requires comparisons between journals, instead of between articles within the same journal. As a consequence, even if efforts are made to compare journals within the same field, there is no way to ensure that the journals cover the same subject matter, nor, perhaps even more important, to ensure that they are of the same quality. For journals do differ not only in subject matter but in the quality of their content.
As Eric Archambault notes, Gold OA journals are handicapped by the fact that they tend to be younger, and hence have not had a chance to establish a track record for either subject matter or quality.
But even for journals of the same age, and even if they are closely matched for subject matter, it is impossible to match them for quality. And to make it even worse, journal average citation counts ("journal impact factors") are sometimes taken as a proxy for quality! Hence equating journals for quality that way would guarantee that there could be no citation advantage between matched OA and non-OA journals!
The good news is that there is no reason to believe that the OA citation advantage that has been repeatedly demonstrated by within-journal comparisons using Green OA should not also generalize to OA provided by Gold OA -- for articles of comparable quality.
One last point: Our studies have found that the size of the OA advantage is itself correlated with quality (or at least with quality as measured by citation counts): The size of the Green OA advantage is greater in journals with higher average citation counts. We tentatively conclude that the citation advantage is greater for "more citable" articles. A lower quality article will not gain as much as a higher quality article from being made more accessible. OA may even lower the citation counts of low quality articles by levelling the playing field, making all articles accessible, and hence making it possible for authors to access, use and cite the best and most relevant articles, rather than being limited to the articles that their institutions can afford to access via institutional journal subscriptions or licenses.
Gargouri, Y., Hajjem, C., Lariviere, V., Gingras, Y., Brody, T., Carr, L. and Harnad, S. (2010)
Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research.
PLOS ONE 5 (10) e13636