5 minute read.Announcing changes to REST API rate limits
Our REST API makes all of the metadata we hold publicly available. It receives the majority of our API traffic, with around 1 billion hits per month. It’s one of the key ways that we fulfil our mission to make research objects easy to find, cite, link, assess, and reuse. From 1 December 2025, we will be revising the rate limits for the public and polite pools of the REST API to ensure that we can maintain a stable and reliable system, and that metadata is freely available to everyone.
We haven’t changed the rate limits since the REST API was launched in 2013. In the past five years, the number of requests to the REST API has tripled and the number of metadata records has increased by a third, from 120 million to around 180 million. This means an increase in the resources needed to run it, and we’ve seen periods of instability where we haven’t been able to keep the API available for all users. We have decided that it is the right time to revisit rate limits to check that they’re in line with what our technology can provide and what our community needs. As a result, we will apply the following for the public and polite pools:
Public pool:
| Request type | Rate limit | Concurrency limit |
|---|
| Single record | 5 | 1 |
| List of records (queries, filters, etc.) | 1 | 1 |
Polite pool:
| Request type | Rate limit | Concurrency limit |
|---|
| Single DOI record | 10 | 3 |
| List of records (queries, filters, etc.) | 3 | 3 |
The rate limit is the number of total requests that can be made per second. The concurrency limit is how many requests can be running at the same time. This means that for longer-running requests you may need to wait for previous requests to finish before you can make a new one.
Here are some examples of single records requests:
The second case here will be directed to the polite pool because an email is included using the ‘mailto’ parameter. And here are examples of requests that return lists of records:
The second and third examples here will use the polite pool.
Our guiding principle in making these changes is to keep all of the metadata available to everyone, all of the time. These changes to rate limits won’t restrict current users from accessing the metadata they want to retrieve, but it will make it easier for us to maintain the system now and in the future.
Which use cases do we support?
Our metadata has a broad range of applications. If you’re someone who uses the REST API, we’re glad that you are part of our community! Our mission includes making it easier to find, reuse, and assess scholarly research outputs. By using metadata, you’re helping us to fulfil that goal.
The main uses of the REST API fit into several categories. The new rate limits will continue to support these, among many others:
- I have some metadata, what is the DOI?
- I have a DOI, what is its metadata?
- I want all of the metadata, just give me everything.
- Research on a specific topic or subset of metadata, often refreshing the results every few weeks or months.
Rate limits can encourage responsible usage. The majority of API users make requests at a low rate and will not need to make any changes, however a few send spikes of large numbers of requests in a short space of time, sometimes making it difficult for others to access the service. These can be smoothed out by lower rate limits. Complex requests that search across large numbers of items put more pressure on our systems than requests for a single content item, so we have decided to set different rate limits for different types of request.
Who will be affected?
We estimate that the changes might affect around 40 users per week across the public and polite pools, and this is only for some of their requests. In all of the cases we’ve seen, the rate of requests could be slowed down and users would still be able to get the same results. In other words, the aim of these changes is to make the load on the API more predictable, not to reduce the total number of requests or amount of metadata transferred. No changes are being made to the Metadata Plus service or other APIs, such as the XML API and OAI-PMH endpoint.
Do I need to change how I use the API?
If you’re reading this, thank you! It’s clear that you want to be a considerate user of our services. Almost all users can continue to use the REST API in exactly the same way, you won’t need to change anything. Here is some general advice that will help you make the most of the service and ensure that you won’t encounter issues.
- Use a mailto parameter. This gives you access to the polite pool meaning higher rate limits and meaning we can get in touch with you if needed. We’ll only use your address to contact you about your API requests.
- Check the HTTP response status for your requests. This is always good practice and can help you identify malformed requests and where you reach rate limits.
- Cache results to avoid repeatedly making the same requests. Most records don’t change on a regular basis. How often you update the cache will depend on what you are interested in, but most metadata fields rarely change.
- If you are making a very high volume of requests or have very complex analysis to carry out, consider downloading the public data file which is made available once a year and contains all of our metadata. You can update it with recent additions using the REST API.
- If you are relying on our metadata in a production service, Metadata Plus can provide more stability, support, and access to monthly snapshots of our entire database.
We have more tips and tricks for the REST API in our documentation. If you have questions, please join the conversation on our Community Forum.
