Shared storage for Docker images with AWS EC2

GitLab CI/CD
, , ,
1

What is the recommended way to “cache” Docker images used by autoscaling GitLab runners on AWS EC2?

Scenario

  • You have multiple (spot) instances with multiple runners on AWS.
  • Runners pull Docker images from public and private registries.
  • Some images may be really large (4GB or more).

Problem
Repeatedly pulling the same images across all your runners is costly - both in terms of performance and money as you need large enough block storage to host all images.

Options (I see)

  • Amazon ECR offers pull-through caching but only for pulling images from public registries.
  • Amazon Elastic File System?
  • EBS Multi-attach?
1 Like
2

The current plan is deploy a Nexus Repository Manager on AWS and configure it as a Docker proxy / pull-through cache.