Blog

What happens to a research work outside of the formal literature? That’s what Event Data will aim to answer when the service launches later this year.

Following the successful DOI Event Tracker pilot in Spring 2014, development has been underway to build our new service, newly re-named Crossref Event Data. It’s an open data service that registers online activity (specifically, events) associated with Crossref metadata. Event Data will collect and store a record of any activity surrounding a research work from a defined set of web sources. The data will be made available as part of our metadata search service or via our Metadata API and normalised across a diverse set of sources. Data will be open, audit-able and replicable.

We will shortly be adding a new feature to Crossmark. In a section called “Clinical Trials” we will be using new metadata fields to link together all of the publications we know about that reference a particular clinical trial.

Most medical journals make clinical trial registration a prerequisite for publication. Trials should be registered with one of the fifteen WHO-approved public trial registries , or with clinicaltrials.gov which is run by the US National Library of Medicine. Once registered, a trial is assigned a clinical trial number (CTN) which is subsequently used to identify that trial in any publications that report on it.

In the 2015 Crossref Annual Meeting, I introduced a metaphor for the work that we do at Crossref. I re-present it here for broader discussion as this narrative continues to play a guiding role in the development of products and services this year.

Metadata enable connections

At Crossref, we make research outputs easy to find, cite, link, and assess through DOIs. Publishers register their publications and deposit metadata through a variety of channels (XML, CSV, PDF, manual entry), which we process and transform into Crossref XML for inclusion into our corpus. This data infrastructure which makes possible scholarly communications without restrictions on publisher, subject area, geography, etc. is far more than a reference list, index or directory.

Forty wire telephone switchboard, 1907, Author unknown, Popular Science Monthly Vol 70, Wikimedia Commons.

A few months ago Crossref announced that we will be launching a new service for the community in 2016 that tracks activities around DOIs recording user content interactions. These “events” cover a broad spectrum of online activities including publication usage, links to datasets, social bookmarks, blog mentions, social shares, comments, recommendations, etc. The Event Data service collects the data and make it available to all in an open clearinghouse so that data are open, comparable, audit-able, and portable. These data are all publicly available from external platform partners, and they meet the terms of distribution from each partner.

Crossref goes live in tandem with DataCite to push both publication and dataset information to ORCID profiles automatically. All organisations that deposit ORCID iDs with Crossref and/or DataCite will see this information going further, automatically updating author records. 

Skimming the headlines on Hacker News yesterday morning, I noticed something exciting. A dump of all the submissions to Reddit since 2006. “How many of those are DOIs?”, I thought. Reddit is a very broad community, but has some very interesting parts, including some great science communication. How much are DOIs used in Reddit?

(There has since been a discussion about this blog post on Hacker News)

We have a whole strategy for DOI Event Tracking, but nothing beats a quick hack or is more irresistible than a data dump.

Publishers, researchers, funders, institutions and technology providers are all interested in better understanding how scholarly research is used. Scholarly content has always been discussed by scholars outside the formal literature and by others beyond the academic community. We need a way to monitor and distribute this valuable information.

We regularly see developers using regular expressions to validate or scrape for DOIs. For modern Crossref DOIs the regular expression is short

/^10.\d{4,9}/[-._;()/:A-Z0-9]+$/i

For the 74.9M DOIs we have seen this matches 74.4M of them. If you need to use only one pattern then use this one.

Anybody who knows me or reads this blog is probably aware that I don’t exactly hold back when discussing problems with the DOI system. But just occasionally I find myself actually defending the thing…

Background

On January 20th, 2015 the main DOI HTTP proxy at doi.org experienced a partial, rolling global outage. The system was never completely down, but for at least part of the subsequent 48 hours, up to 50% of DOI resolution traffic was effectively broken. This was true for almost all DOI registration agencies, including Crossref, DataCite and mEDRA.

At the time we kept people updated on what we knew via Twitter, mailing lists and our technical blog at CrossTech. We also promised that, once we’d done a thorough investigation, we’d report back. Well, we haven’t finished investigating all implications of the outage. There are both substantial technical and governance issues to investigate. But last week we provided a preliminary report to the Crossref board on the basic technical issues, and we thought we’d share that publicly now.