Wellcome Trust and COAF Open Access Spend, 2014-15

In the “Reckoning” blog post last year – which provided data on how much the Trust spent on open access (OA) publishing and to what extent publishers provided the service we paid for – Robert Kiley, Head of Digital Services at the Wellcome Library concluded that “there are significant problems –in terms of depositing content in PMC and licensing this in accordance with our requirements – and the first class service called for still seems to be some way off”.

One year on Robert Kiley and Cecy Marden provide an analysis on what the 2014-15 data reveals…

Once a year we ask all those institutions in receipt of an open access (OA) grant from the Trust to provide details on how the grant has been spent.  In 2014 six research funders established the Charity Open Access Fund (COAF) in order to provide a single funding mechanism to cover Article Processing Charges (APCs).  The data supplied by institutions for the 2014-15 report therefore covers APC spend on articles arising from research funded by Arthritis Research UK, Breast Cancer Now, Bloodwise, British Heart Foundation, Cancer Research UK and the Wellcome Trust.

Following the approach we adopted last year, this blog post provides details of the total spend on OA publishing and considers to what extent papers are freely available through the Europe PMC repository and licenced in accordance with our requirements (i.e. made available under a Creative Commons Attribution License (CC-BY)).  In summary, we find that hybrid open access continues to be significantly more expensive than fully open access journals and that as a whole the level of service provided by hybrid publishers is poor and is not delivering what we are paying for.  The post concludes with a set of action items aimed to rectify these problems.

Cost data

In the 2014-15 reporting period, some 2942 articles were published under an APC model using COAF grants.  This represents a 15% increase compared with the previous year.

The total COAF spend for the year 2014-15 was around £5 million.  This equates to a 14% increase on what the Trust spent in the previous year.  The average APC was £1914, whilst the median cost was £1834.

Table 1 shows how these figures compare with previous years.

Table 1: APC spend for the years 2012-13, 2013-14 and 2014-15

Although the total COAF spend has risen proportionately (as compared with Wellcome Trust spend in previous years), the average APC has increased by 4% over the past 12 months.  In the previous year, the average APC increase was less than 1%.

Table 2 provides a breakdown of the type of journal where the articles were published, along with the average APC for each type.  A fully OA journal is one in which every article is made OA (e.g. BMC Bioinformatics, PLOS One etc.), whereas a hybrid journal is one which is still published under a subscription model, but where individual articles can be made OA.  The ‘unknowns’ are those where we were unable (programmatically) to determine if the article was published in a fully OA or a hybrid journal.

Table 2: Fully OA vs hybrid OA – 2013-14 and 2014-15

There has been a slight increase in the proportion of papers being published in fully OA journals (from 24% to 26%) but the majority of authors continue to utilise the open access, hybrid option within traditional, subscription-based journals.

In line with last year’s findings, we note that the average APCs levied by hybrid journals are significantly higher (51%) than the average APCs charged by fully OA titles.

Less expected however, was the finding that the average APC for fully OA journals has increased more than the average APC for hybrid journals over the past year (12.5% as compared to 3.7%).  A preliminary analysis of the data suggests that this is a consequence of the fact that authors are publishing more articles in fully OA journals published by subscription publishers than hitherto, and that OA journals published by subscription publishers tend to have higher APCs then their “born digital” counterparts.  By way of example, in 2013-14 there were just 14 Wellcome-funded papers published in Elsevier’s Cell Reports (which has an APC of $5k); in 2014-15 this number had increased to 22.  To determine how Elsevier’s average APC for their fully OA journals was affecting the overall average APC, we removed Elsevier’s articles from the dataset.  This resulted in the average APC for fully OA journals to fall from £1396 (see Table 2) to £1329.

Table 3 breaks down the costs for the 5 publishers which published the most COAF-funded articles in 2014-2015.

Table 3: Top 5 publishers (by volume of COAF-supported research) and APC spend

Last year we noted that the average APCs for the fully OA journals published by Elsevier and Wiley were significantly higher than those charged by competitors at PLOS, BMC and OUP.  Interestingly this year Wiley’s average APC for fully OA journals is much more in line with its competitors.  However, though Elsevier’s average APC for fully OA journals has also reduced (from £2930 to £2447) it remains more expensive than its competitors.

Compliance data

In addition to understanding how much open access is costing us, we also want to know whether publishers are providing the service we require.  Specifically, if COAF funding is used to cover an APC, the publisher must deposit the final version of the article in PubMed Central (PMC) immediately upon publication (this content is then mirrored to Europe PMC), and ensure the article is clearly licenced CC-BY on their own site and in PMC.

Once again we used our Cottage Labs compliance-checking tool to programmatically determine whether the paper is in the Europe PMC repository and what licence (if any) is attached (see Table 4).  This analysis was run on the 6th January 2016.

Table 4: APC content: compliance with COAF OA policy: 2013-14 vs 2014-15

The good news is that we have seen an improvement in correct and programmatically identifiable licences (from 61% of papers in ’13-‘14, to 70% in ’14-‘15) and a similar increase in overall compliance from 61% to 70%.  The bad news, however, is that in 30% of cases we are not getting what we are paying for.

Last year we were so disappointed by the compliance data that we undertook some further investigation to try to break down what was happening.  We found that roughly a quarter of the articles which were not in Europe PMC had not yet been assigned to a final issue – they still had “ahead of print” status.  This is important as the PMC repository only accepts the final version of the article, and thus it is not possible for a publisher to deposit a paper that is released online “ahead of print”.  This year we performed the same analysis and got consistent results – just over a quarter of the articles not in Europe PMC are still “ahead of print” and we should therefore expect to see them deposited in PMC (and mirrored to Europe PMC) over the coming months (see Table 5).

Table 5: Final published version vs ahead of print 2014-15

However, even after removing the 142 “early view” papers from the cohort, there are still 392 articles for which we paid an APC which are not available in PMC or Europe PMC. In financial terms this equates to around £765,000.  Spending this level of money – and not having access to the article in the designated repository – is clearly unacceptable.

Source of non-compliance?

To help understand the source of non-compliance – and determine where we should focus our efforts in terms of rectifying this – we carried out two further bits of analysis.  Firstly, we looked at all the articles for which an OA fee had been paid to determine if the non-compliance was more prevalent in hybrid journals.  And, in line with the findings of previous years, non-compliance is far more problematic in hybrid journals (see Table 6).

Table 6: Non-compliance: Fully OA journals vs hybrid journals

Secondly, using our “Top 5 publishers” by volume (see Table 3, above) we looked at their respective levels of compliance (see Table 7).

Table 7: Non-compliance – the top 5 publishers by volume

What is striking is that half of the articles published by Wiley were non-compliant, and that a quarter of the articles published by Elsevier in fully OA journals were also non-compliant.  In contrast, 100% of the articles published by PLOS were problem-free.

Action items

Given the above, we believe there are a number of actions we can take to ensure that COAF funders receive the service we are paying for.

Retrospective actions

To try to correct the problems associated with articles detailed in the 2014-15 APC spend data we have reached out to all institutions in receipt of a COAF grant and asked them to follow up with the relevant publishers regarding non-compliant papers.  We will re-run the analysis in May ‘16 and if papers are still non-compliant we will need to re-consider the ways in which we make payments for APCs.

We have also had meetings with the two biggest publishers (Elsevier and Wiley) to make them aware of these issues and, more importantly, to put measures in place to ensure these problems do not arise in the future.  Both publishers have retrospectively deposited missing (or incorrectly licenced) content to PMC.

Prospective actions

Perhaps what is most evident from our recent conversations with stakeholders is a misunderstanding of what we actually require when we fund an APC, and how we measure compliance.  By way of example, a number of institutions check licence compliance by looking at  content on publisher web sites (rather than PMC/ Europe PMC) and, if a paper is licensed correctly this was deemed to be compliant. On the face of it this seems a reasonable position.

However, we check compliance by looking at the content available in Europe PMC and, as this post has made clear, in so doing we have found numerous examples of papers not being deposited to PMC, even though the article has been made OA on the publisher web site and is clearly COAF funded.  It is worth stressing that we require content to be available through Europe PMC (and not just OA on the publisher site) as it provides funders with a single, archival repository where all their research can be found and researchers with a large corpus of content, which can be searched and mined and which is fully integrated with the underlying biological databases.

Our analysis has also uncovered many examples of papers where the licence cited on the PMC article is different to the licence cited on the publisher web site.  Given that the publisher is being paid to deposit the final version of the article this issue seems particularly perplexing until you realise that publishers are changing the licence (post deposit to PMC) – typically in response to requests from repository managers who realise that the author has selected a non-compliant licence – but are not sending the updated paper to PMC.

Consequently, to address such issues we plan to develop a more detailed set of principles and requirements which have to be met before we regard a journal to be compliant.  Journals which confirm that they can meet these will be compliant with our policy; those which don’t, will not.

In last year’s blog we singled out Wiley as the publisher who was having most problems meeting our requirements.  This year Wiley’s performance is still poor – for example, 115 papers for which an APC was paid had not been deposited to PMC.  Wiley have recognised this and have already begun making changes to their processes, aiming to significantly increase compliance.  If we do not see a demonstrable improvement (based on content published under the APC route in the first six months of 2016) then we will consider additional steps.  These could include moving to a system whereby an invoice will only be paid when the article is available in PMC, seeking compensation for non-compliant papers, and/or declaring Wiley journals to be non-compliant with our OA policy.

Of course, quality of service is only one dimension of the problem – cost is the other.  We need to understand why publishing in hybrid OA journals is so much more expensive than fully OA journals, and why fully OA journals, published by the traditional subscription publishers, tend to be more expensive than the OA journals published by “born digital OA publishers”.  We are actively engaging with the community to explore options for addressing this issue, and found the recent Jisc paper on this topic extremely helpful.

Conclusion

Given the well documented problems associated with hybrid OA –  most notably around high prices and poor levels of service –  a number of research funders, including the German Research Foundation (DFG) and the Norwegian Research Council, have deemed that that their funds cannot be used to support this type of OA publishing.

Wellcome, however, has not yet reached this position.  We are mindful of the fact that our researchers want to publish their research in a range of journal outlets, many of which are traditional, subscription based titles.  We believe declaring that Wellcome funds cannot be used to pay for hybrid OA is too blunt an instrument, unfairly penalising those publishers which provide a good service at a reasonable price, and that it would slow down the transition to a fully OA world – the position we ultimately want to get to.

However, doing nothing is no longer a valid option.  If hybrid publishers are unable to commit to the Wellcome Trust’s set of requirements and do not significantly improve the quality of the service, then classifying those hybrid journals as “non-compliant” will be an inevitable next step.

The data underpinning the findings presented above are available on Figshare.

Image credits: All illustrations by Ed Grace, CC-BY-NC-SA