Can we use AWS EFS or AWS S3 to store/upload Gitlab Job artifacts?

How to Use GitLab
1

In AWS, we use Gitlab CE with ECS, with AWS EFS storing configuration and other items.
We are currently using S3 for Giltab Job Artifacts and will be uploading Job Artifacts there.

I’d like to ask you a few questions about the Job Artifacts.

  • What is the advantage of uploading Job artefacts to S3 rather than storing them in AWS EFS?

  • The advantages and disadvantages of AWS s3 and EFS for job artifacts.

It would be wonderful if someone could help with these details.

2

Hi @praseebkdas
you can look at official docs Jobs artifacts administration | GitLab whats the options and caveats.

As for the advantages of storing Job Artifacts in S3 it really is mostly about having ephemeral GitLab in containerized envirornments like k8s, where you don’t want to store anything locally in each stateless pod. So you would end up storing everything in external storage.

Another use-case is if you have limited storage on the server where GitLab is running and Job artifacts could fill it up.

Other than this 2 cases I can’t really think of why you would want to store Job Artifacts in S3 or other external storage.

3

@balonik I strongly agree with your points if we are storing in a local environment, but in this case we will be storing in AWS EFS where we will be mounting the volume to store any gitlab configuration or logs for containerized environments such as k8,ECS.

4

I see two advantages:

  • S3 is way cheaper than EFS, especially if you use IA.
  • You can upload directly from the runners to S3, without proxying through GitLab. Performances are better, and the load on GitLab is way smaller. With EFS, you cannot write directly to it
1 Like
5

Thanks for the details.