Rolling back multi stage deployments to an environment


#1

When deploying to environments using GitLab CI, is there a way to have a multiple step environment. What I am looking for is the ability to run multiple job steps during a roll-back.

Currently, a roll-back will only run the stage that did the final deployment step. I currently build docker images for deployment but I do not want to keep every one. What I would like to do is run the build steps and the deployment steps when a roll-back is triggered. Is this possible?


#2

We have a similar setup. Broadly speaking our build steps are 1) Build; 2) Test; 3) Build Docker Image; and 4) Deploy Docker Image.

First, I think it’s a good “best practice” to keep at least some docker images around, particularly for production. If production goes down (say, from a bad merge that wasn’t caught in testing, for example), I can think of nothing I want to be doing less than waiting for a build to complete in that situation. Personally, we keep the most recent hot-fix of every major release and all production deployments for the last month. This may avoid your issue straightaway.

Second, if keeping images around is not possible to do, (and I’m in speculation territory here because we never need to do this because of the above) how does gitlab respond in a redeploy if there is a dependency on a previous build step? I would assume that it would be forced to re-execute the deploy and all previous dependent steps if it/they is/are not available in its cache. If that is the case, as long as you have your deploy step dependent on the relevant build steps, I would expect that it would rebuild. The other way I could see it working is that it might just not let you roll back to that version at all.

Do you have dependencies setup between the CI build stages?

Dave


#3

I do have dependencies between build steps but this does not trigger the entire chain.

I believe my best solution is to keep all production builds (removing after a certain timeframe if necessary) for the reasons you mentioned. I did not like the idea of waiting for a rebuild because that is what we are doing now.

How are you discarding images that are over the month threshold? I don’t believe this is provided in the API. I had a thought of nesting since GitLab will allow up to three levels in docker naming and then groups them in the UI. So something like following, where the images are grouped by the month. The UI will then allow you to delete all tags for a grouped image.

docker.example.com/group/project/201708:AD23FCB