[FOLIO-2820] SPIKE: Docker Hub download rate limiting Created: 06/Oct/20  Updated: 03/Nov/20  Resolved: 02/Nov/20

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Story Priority: P2
Reporter: Wayne Schneider Assignee: John Malconian
Resolution: Done Votes: 0
Labels: devops-backlog
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
is blocked by FOLIO-2844 Create a Docker Hub account for the `... Closed
is blocked by FOLIO-2845 update FOLIO ansible to use Okapi reg... Closed
is blocked by FOLIO-2846 update Jenkins pipelines for Docker auth Closed
Relates
relates to OKAPI-912 Docker pull with authenticated user (... Closed
relates to FOLIO-2722 Set up Docker Hub organization beyond... Closed
relates to FOLIO-2860 Add support for authenticated docker ... Closed
Sprint: DevOps: Sprint 99, DevOps: Sprint 100
Development Team: FOLIO DevOps

 Description   

Starting this month, with full enforcement by Nov 1, Docker Hub will begin enforcing download rate limits:

https://docs.docker.com/docker-hub/download-rate-limit

The practical upshot:

  • Unauthenticated docker pull requests are limited to 100/6 hrs from any one IP
  • Authenticated docker pull requests are limited to 200/6 hrs for free accounts
  • Authenticated docker pull requests for paid accounts are unlimited, but note: Limits are applied based on the user doing the pull, and not based on the image being pulled or its owner.

Since standing up a full FOLIO environment involves about 60 docker pull requests, this has imminent implications for our CI builds. In addition, other implementers (hosting providers, etc.) will start to run into issues.

Things to consider:

  • Do we need to set up a paid account for our CI to avoid any possibility of blocking based on rate limits?
  • How do we authenticate docker pull requests in the context of Okapi-based module orchestration? OKAPI-912 Closed
  • What other contexts for docker pull need to be considered, and how can authentication be integrated in those context?
  • Does it make sense to move the FOLIO container registries to another service, e.g. GitHub Container Registry or in our own Nexus repository?


 Comments   
Comment by John Malconian [ 06/Oct/20 ]

One idea may be to use the FOLIO Nexus repository as a caching Docker proxy to Docker Hub similar to the way it's used for NPM and Maven Central. This potential option is really only a viable for FOLIO project CI infrastructure and not for general use by the community, however. I'm not sure there is a way to make the proxy transparent, but we can look into that. Otherwise, we'd have to update various build pipelines to use the Nexus Docker proxy registry. In this scenario, the authenticated Docker user would be Nexus.

Comment by Jakub Skoczen [ 13/Oct/20 ]

John Malconian Wayne Schneider we've discussed that the least intrusive way is to wait for OKAPI-912 Closed and authenticate with DockerHub through a non-limited account (Peter Murray is trying to get a non-limited account for FOLIO).

Comment by Peter Murray [ 13/Oct/20 ]

I'm pursuing two tracks. First is to get us recognized as an open source project ( FOLIO-2722 Closed ) and in the course of doing so ask if we can have an account for the CI that wouldn't be affected by the rate-limiting. Second—if needed—I'll work with Scott Anderson (OLF Treasurer) to get a Docker Hub account paid for through the Foundation.

Comment by Jakub Skoczen [ 20/Oct/20 ]

with OKAPI-912 Closed implemented and released we still need the following things:

1. update ansible to make use of the new Okapi feature
2. update Jenkins pipelines to allow specifying DH auth and registry (to use CI's Nexus)
3. Helm charts used for scratch env deployment

Stanislav Miroshnichenko Any ideas about the 3. thing?

Comment by Stanislav Miroshnichenko [ 20/Oct/20 ]

Jakub Skoczen Rancher can store auth. credentials for Docker Registry, such as DockerHub. But these credentials will be available for team's members in Rancher for reading.

Comment by Peter Murray [ 02/Nov/20 ]

Note that this may turn out to be a non-issue, as the `folioorg` and `folioci` repos would be exempted from the rate limits when the project is recognized as an open source project. See this comment on FOLIO-2722 for more details.

Comment by Jakub Skoczen [ 03/Nov/20 ]

Stanislav Miroshnichenko Wayne Schneider We talked about using the FOLIO project Nexus proxy for pulling images from DockerHub. Wayne is going to provide Nexus access credentials to you.

Comment by Jakub Skoczen [ 03/Nov/20 ]

Nexus repo for pushing scratch env images: docker.dev.folio.org
Nexus proxy: docker.ci.folio.org

Generated at Thu Feb 08 23:23:29 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.