How to handle deployment tasks to a massive amount of hosts?

How to Use GitLab
1

Hello,

I have a massive amount (100+) different hosts that i can deploy my product to. I have to be able to deploy the product to each one of them separately.
Creating different tasks for all of these with only different host seems unmanagable. One possiblity is to have one task and while running the task i can add variable host=hostname, but this is also unmanageable because i cant memorize all different possible hostnames.
What is the recommended solution for this in GitLab CI?

2

Hi,

are you using for example Ansible or Terraform to deploy your application in this regard? With that in mind, you can define a host list as variable and let the deployment job just trigger one deployment - being then run for the defined host list.

Besides, how do you manage these hosts, are they auto-discovered or statically the same?

Cheers,
Michael

3

Hi. I have a static list of hosts. Currently im not using ansible, but I would be open to it if it helps me with my problem. The main problem is that i want to be able to deploy the protduct to each host seperatly, but i dont want a task for each one. Ideally I imagine I would have one deploy task and i could somehow select a variable from a dropdown where is it deployd. Is something like this possible?

4

Hi,

the dropdown will be hard I’d say, the CI web forms are not that generic. One thing you can do is to pass specific variables to a CI pipeline with triggering this on demand. If you fill in a list of names with being parsed in the job, this could work.

One thing I would take into account are dynamic environments. You could have an environment for each host, or a group of hosts where you deploy manually then.

The docs also mention incremental rollouts for Kubernetes, which could be an idea here.

I’m not sure if that fits with Ansible deployments though, but at least worth a try :slight_smile: CI could also trigger an external “daemon” which does the incremental rollout then.

Cheers,
Michael

5

For me that sounds more like something puppet could be useful for. You can define groups of hosts (roles) where configurations/programs are deployed then - also possible in combination with gitlab and ci… but maybe I’m too much puppet-branded :wink:

1 Like
6

Hey Guys,

I would suggest using Salt(salt-api) to achieve managing tons of hosts. We use saltstack and now i’m thinking on how to use curl to grab salt-api credentials from gitlab variables and do remote execution.
If you have any ideas about this, please share :slight_smile:

7

Hi,

please open a new topic and add some more examples on your current Saltstack implementation, as well as where you’d want to fetch the credentials exactly from. Keep in mind that not everyone knows Saltstack and its behaviour, so a little explanation will help in this case.

Cheers,
Michael