What is the recommended way to “cache” Docker images used by autoscaling GitLab runners on AWS EC2?
Scenario
- You have multiple (spot) instances with multiple runners on AWS.
- Runners pull Docker images from public and private registries.
- Some images may be really large (4GB or more).
Problem
Repeatedly pulling the same images across all your runners is costly - both in terms of performance and money as you need large enough block storage to host all images.
Options (I see)
- Amazon ECR offers pull-through caching but only for pulling images from public registries.
- Amazon Elastic File System?
- EBS Multi-attach?