The GitLab instance I am using is a corporate on-premise solution. I am working with auto generating a yaml file to use as an input to trigger a child pipeline. In one scenario, I have 8 needs, 2 of which are local jobs from the same pipeline, and 6 of which are pipeline/jobs from the parent pipeline.
jobs:deploy_package:needs config must be less than or equal to 5
So when I remove one of those pipeline/job needs leaving only 7 needs in total (2 local jobs, 5 pipeline/jobs) it works fine. This leads me to believe that there is a limitation specifically on the pipeline/job needs and not the normal local jobs needs. I have spoken to those who manage our GitLab server and they have confirmed that the default job limit is 50 needs jobs.
My question is there a separate pipeline/jobs needs limitation configuration?
Yes, I understand the limit can be changed. However, I have confirmed that the needs limit for my company’s GitLab server is in fact 50 needs. So why am I still getting a limit exceeded error when I have more than 5 Pipeline/Job needs referenced?
My inquiry is not if the limit of needs is configurable, but specifically which configuration do I need to change to increase the Pipeline Job needs. Not all needs. I know that the default limit is 50, however it appears that the Pipeline Job needs are still limited to 5.
For anyone else who might be encountering the same thing, I wanted to share what I discovered: there is a separate limit for cross-project needs, set to a limit of 5 jobsThe limit is documented here:
Use needs:project to download artifacts from up to five jobs in other pipelines
The limit is set on purpose and cannot be configured because querying for the correct instances of jobs based on cross project needs requires an expensive and complex SQL query.
Another possible idea might be to gather up the artifacts in one job in the Parent pipeline with “needs” and then make that job needed in the child pipeline.