Citation Validation

The output from Spotcites

Comparing "orange" and "red" links with the total number of citations. For the opcit project certain classes of identifiable citations have been colour coded (for development purposes):

Using the output from Spotcites red links can be identified by lines that contain a arXiv reference more than once. A table can be built of paper * cited papers * orange|red. This table can then be used to build the number of orange/red links for a given month.

This graph shows how many citations have been identified, retrospectively, for papers deposited each month. The total number of "lines of citations" is shown in blue (not the total of red and orange), i.e. the background population from which citations are being identified from. The blue line corresponds to the right-hand y-axis and is scaled down.

Excel Spreadsheet Source

The numbers of red and orange citations found can be compared against the actual number of citations in the following graph. The green line corresponds to the sum of red and orange links scaled for comparison to the total number of citations.

The total number of citations can be factored out by dividing the red, orange and sum by the total citations for the equivalent month. This shows how increasingly spotcites is more effective at identifying citations, and that more citations are to papers in arXiv.

A comparison can be made between red and orange links, i.e. the changing behaviour of authors from citing post-print articles to citing e-print articles. The following graph shows how the number of direct citations has increased at a greater rate than the number of identifiable citations to published articles.

The graphs so far have covered the entire archive's deposits. It would be useful to identify how well citations are being identified within certain subject areas.

hep-th

astro-ph

Why are 40% of citations red-links - citations to LANL e-prints?

cond-mat

hep-ph

In addition to analysing citation identification we can also add the ratio of citations to "antique" papers, citations to papers that were published before the start of the archive. As would be expected the percentage of citations to antique papers reduces over time (i.e. currently a paper would have to be more than 10 years old to be antique").

When comparing the trend of antique citations to the increasing percentage of successfully identified red and orange citations we can see that citations to papers that are potentially not in the archive has remained relatively constant (from 50% to 30%), in contrast to the increase of identifiable citations to papers in the archive (from 0% to 50%). If we assume that spotcites is capable of identifying 100% of citations to articles in arXiv this suggests that the ratio of citations to articles not in arXiv has remained relatively constant - this could be because:

1. these are citations to non-arXiv media (e.g. books)
2. the cited author has decided not to deposit their articles in arXiv
3. the numbers of articles outside of arXiv are constantly being added to because antiques are reducing

Negative Latency Citations

There are 6455 instances of citations having a negative latency (that is that a paper cites another paper before the cited paper is deposited in the archive). These are from 4259 papers.

This number can be seperated into those that are citations to LANL preprints (3479) and those that are citations to published articles (2976), but the published article has been deposited in arXiv. Negative latency citations to LANL articles can only occur if an author updates the paper, and adds additional references. The total number of papers that this has occured in is 2095 (1.58% of 132219 papers).

Checking a sample of citations

A sample of 100 citations (the last 100) were taken from papers deposited in 1999, 1997 and 1995. For each citation the relevent line was found from the output of spotcites (the source citation text) and compared to the abstract of the cited paper. Primarily the author's names were used for verification, as spotcites is based on matching publication data.