Should auto devops be disabled by default?

I just thought of an additional way this hurt. The extra CPU load it caused! Looking at some of jobs that failed, some of them took almost a minute, even though whatever gitlab tried didn’t make sense.

With a lot of forks of the some of the projects (I think some of the projects not containing code might the the most forked) and gitlab having to try this for every fork, it wasted quite a few CPU cycles on this, and as our gitlab server is heavily loaded (that’s one of the reasons why I have to set up a another one, security being another) this wasn’t good.

And by chance I just stumbled upon a fork where autodevops apparently wasn’t disabled after the first failure (we have 4 failed jobs for the same commit with the same pipeline id, but different job ids, over a period of 52 minutes). You can have screenshots of the job list and job outputs, but they are not really interesting, all the failed jobs have “build” in both the “stage” and “name” columns, while the “test” stage jobs were skipped, and except for some git differences in the job outputs they are all alike, ending with:

Checking out 56f382d7 as master…
Skipping Git submodules setup
$ # Auto DevOps variables and functions # collapsed multi-line command
$ setup_docker
$ build
Logging to GitLab Container Registry with CI credentials…
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Login Succeeded

Building Heroku-based application using gliderlabs/herokuish docker image…
-----> Unable to select a buildpack
ERROR: Job failed: exit status 1

I maintain the belief that it was a terrible idea to enable this be default, and the belief that only being able to disable it for all projects from the webinterface is terrible lack of configurability.

I’m going to add to what everybody else has already said… Auto Devops shouldn’t be getting enabled without the user requesting it. We had existing projects which auto-deply after a successful merqe request but were all of a sudden not deploying anymore. We realised they were being held up because Auto Devops had been switched on and was blocking the merqe request from completing.

This reflects really poorly on Gitlab. Please strongly consider reversing your decision.

2 Likes

In order to test your beta service, you will force your happy (paying) customers to unknowingly have their private repos practically stolen and processed on some public runner. Let’s see how many will be happy once they found out what you are doing. This is really sad.

HI Daniel. The issue I’m having is that we have hundreds of repos on gitlab.com and on some of the repos these autodevops jobs are now running when we push any commits and all failing. I have found documentation on how to turn this off globally (https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#auto-devops) but these directions do not seem to apply to gitlab.com. If they do, then I’ve got other issues because I have no “admin area” little wrench icon in my toolbar. If we’re going to have to go into the settings of every one of these hundreds of repos, we’re in trouble.

We had existing projects which auto-deply after a successful merqe request but were all of a sudden not deploying anymore. We realised they were being held up because Auto Devops had been switched on and was blocking the merqe request from completing.

@jspurling Interesting. I’d like to understand your usage better. Were you automating deployments using something outside .gitlab-ci.yml? If so, is the problem that we were blocking the MR from being merged at all, because of the failed pipeline?

Thanks for the feedback @grove. What sort of your problems did your team come across?

We try to minimize time spent on projects that are not compatible by automatically disabling Auto DevOps on projects where its pipeline fails the first time.

Thanks for the feedback @pestophagous, will consider this in future features.

@andi You private repo remains private and the applications only get pushed to you know own cluster (when configured). If no cluster is present your application is not deployed.

@bfredtim the directions to disable at the instance-level are for self-managed instances for GitLab only. For GitLab.com there is no need to disable your project after an Auto DevOps pipeline fails; Auto DevOps will be automatically disabled after a failure.

11.4 was released yesterday, and the terrible decision to enable autodevops by default has not been reverted :cry:

I have checked with one of the colleagues who encountered the problems, and it seems they were “just” what has already been reported:

  • confusing mails
  • blocked merges
  • autodevops NOT automatically being disabled
  • doesn’t make sense

As we pay for gitlab, and read the release logs, we felt justified in believing gitlab wouldn’t disturb our work, but it did! You might feel autodevops is smart, and offers value to some of your customers, but it also cost a lot for some of your customers! Enabling it by default was a TERRIBLE idea. I think gitlab should be glad you don’t have much competition.

It’s also somewhat interesting to realise that although many users visit this forum every day, this thread has now existed for 20 days without anyone (except for three gitlab employees) saying anything positive about this “feature”, so several people hate it and point to very specific problems and noone likes it enough to defend it, except employees who more or less have to.

And I still would like to know how I can get it disabled instance wide (from a configuration file, so it can be automated) on the new instance I’m going to set up soon.

1 Like

Agree with this. Confusing to Developers and resulted in emails to me and therefore extra work. Bad decision I’m afraid.

1 Like

As a brand new user to GitLab trying out the service, I was looking forward to moving my projects from BitBucket. I wrote a little bash script to loop through all my private repos, add a second pushurl to them to hit GitLab, and pushed. GitLab made new private repos for me which was awesome! Great first impression. But then my inbox filled with like 50+ auto devops failures. Since I had no intention of wasting hours clicking through to make new repos in the UI, this always-on-at-first auto devops setting is a real poor initial user experience with GitLab. The documentation references disabling via “Admin area > Settings > Continuous Integration and Deployment”, but I can’t find any “Admin area”, and the docs don’t link to a visual (https://docs.gitlab.com/ee/topics/autodevops/).

I think I understand what the intent of opt-out auto devops is and I might support that if the implementation was good (success by default when non-configured would go a long way). But as it stands, it is not working and the user feedback here seems to support that. I want to like and use GitLab, but this was a pretty rough first impression. One other oddity - not all my repos had auto-devops on, and by deleting the “failed” repos and re-adding them the same way (by adding a second remote from BitBucket), some of them successfully made it in without auto-devops enabled. Weird.

I dunno - maybe the issue is merely cosmetic (aside from all the email spam), but some of my older repos may never see another commit, and sitting there in a failed state forever is intolerable. Picky, I know. Anyway, hope this helps and that users are being heard.

Hey, we are running into problems with this, too, and are not happy with it. We need a way to globally disable this feature on gitlab.com. The superfluous error messages are confusing and it seems that even the failed Auto DevOps runs have eaten up quota. Please disable this by default or tell us where exactly can we disable the automatically enabled Auto DevOps “globally” for all new (and existing) projects in a group on gitlab.com.

Best regards.

The Application settings API lists auto_devops_enabled as a setting that can be accessed (and hopefully changed by the method shown, but I won’t test that on our production server during working hours). So now I’ve added a task for myself in seeing what we can control through this part of the API, and make a script to check/set those settings that matter.

It’s amazing that none of the gitlab employees that have visited this thread knew that and bothered to write so, the most plausible (to me at least) explanation is that gitlab employees are more concerned with making this mis-feature work (possibly because it makes users of the hosted solution pay for more CPU?) than helping those of us with our own installations manage those effectively. When searching for something completely different I just stumbled upon what I think is the answer to the question I asked in this thread two months ago.

Thanks for following up on this.

Update: I’ve just been reminded that the Web UI and that API setting have equivalent functionality (thanks @axil!)

https://docs.gitlab.com/ee/topics/autodevops/#enablingdisabling-auto-devops-at-the-instance-level-administrators-only

I cannot answer for the other GitLabbers in the thread, but I’m fairly certain that if others would have known the answer, they would have pointed to it.

New features land every month, and it’s a vast and fast-paced project. It’s often the case that unless you are involved on a particular stage you might not be aware of one particular feature or how that part of GitLab you haven’t yet been exposed to works. In general I tend to assume good intentions.

The discussion on this thread is open and so is the documentation, although it seems we could do a better job in terms of discoverability!

1 Like

I knew about that Web UI setting, that’s what we’ve gotten by with so far, but when you care about your set up, you prefer settings that can be automated, the API setting hopefully falls in that category (I’ve been busy with other stuff - e.g. a SSL certificate renewal that broke our GitLab, but that’s not for here - so I haven’t gotten around to try), but without any hints it was there… Well it took two months.

I also prefer to assume people have good intentions, but having seen this, and requests for very necessary features like:


I don’t see how to draw other conclusions than GitLab not being interested in “helping those of us with our own installations manage those effectively”

And yet another example of the poor user treatment in this thread: nobody cared to point @sebastian.meyer at:


it might not have helped him directly, but knowing the issue exists might at least make him feel a bit comforted and less alone.

1 Like

I have a problem with auto devops, can somebody help me?

I just found


where several concerns were raised before enabling autodevops by default. That it still was done is really depressing.

It’s also depressing that I’ve heard absoluting nothing in response to the support request to GitLab I made (encouraged by @danielgruesso) reporting that AutoDevops wasn’t automatically disabled.

Hi @grove. Sorry to hear you have not received a response. Could you link me to the ticket you’ve filed? I’ll be glad to help there. You can send me a message via email (first name at gitlab dot com).

We thought long and hard before making this decision. We understand Auto DevOps does not cover all use cases yet and still has room for improvement. We’re working hard to make that happen. Thanks for caring and taking the time to write this.

And writing @danielgruesso gave absolutely nothing. I’ve still not heard anything about why (or just an apology for the trouble it caused) autodevops didn’t automatically disable.