Reset CI/CD runners using gitlab free shared runners

How do I completely reset the CI/CD pipeline for a repo

I have what I would call a head scratch-er. I have a project in which I’m running a couple of pipelines. Recently I added one test case and I keep on getting a 400 error from the API it’s calling. This seemed weird since it worked on my machine and multiple others machines; there was no diff in payload or payload parameters.

To add to the confusion someone cloned the repo into a new one and in the new repo the pipeline did not receive a 400 error. With no code changes, the runner seemed to execute the code just fine. I am wondering if there is a way to fully reset the CI/CD pipeline for a project? I am using the free runners provided by gitlab to execute the pipelines and have already tried to clear their cache’s but that did not solve the issue. So why would an export and new repo allow the code to work but it would not work in the original

Thanks for taking the time, it really helps! :blush:

Hi @Arjunica

If you go to https://gitlab.com/GROUP/.../PROJECT/-/pipelines and look at your pipelines, what happens if you press the Clear runner cache button, and re-run the pipeline?

Cheers,

Sarah

Hi @snim2 !

Thanks for the response. Yes that is a solution I have tried; but the pipeline still returns a 400 in that case. Should I wait a couple minutes between clearing and retrying or is immediately fine?

Thanks

This is really strange. 400 is a Bad Request error, but it’s not clear to me which part of the code is getting that response.

Is the project public? Can I fork it and have a look?

Sarah

Hey, The project is not public but I can include some code snippets to give more context. the post call is where the error is happening. when I run this exact case is a brand new repo the pipeline gets a 200 return from Datadog which leads me to believe there is something wrong in my current

In your original post, you say that python3 lint.py returned the 400 response. I guess that is coming from if response.status_code == possible_code_response.base_request_client_code: ...?

I’m guessing also that that response object comes from the method below def send_log_datalog(self, message): ... where you send the HTTP response to __handle_response().

I notice that you have a call os.getenv('DATADOG_API_KEY) where you get an API key from the operating system environment. That API key needs to be in your CI/CD env vars, which would normally be set in the GitLab ui.

So, if your API key is not set, that might cause your problem. Equally, if the URL you are requesting with the requests library does not conform to the expectations of the API, then you might get that error (you might be able to check this with curl or with a browser).

Good luck!

Sarah

Hey Sarah –

Thank you for the help! you are right that the original post talked about linting.py. It is happening in two different repos using the same POST call so the issue seems to be wide spread. I have set the var DATADOG_API_KEY in the CI/CD settings. I know it’s returning a 400 since the line response_body = json.loads(response.text) blows up since the text returned is not a json body. I believe invalid API key would be a 403 error but you are right that I should check that.

Here is the code from the lint.py:

So it looks like the only lines there that might send an HTTP request are 70 and 76. If i were you, I’d find out exactly what payload those two lines are sending (with print() statements) and then try to run the same HTTP requests with curl on the command line, using the API key set in your CI/CD configuration.

Good luck!

Sarah

Hey –

Thanks for the idea @snim2!

any ideas for why it works perfectly well in a new repo? Do you think there could be some artifact not being cleared even though the cache is being cleared?

I doubt it, but you can always delete the artifacts and your previous pipelines, as well as the cache. It sounds more likely to be an issue with your CI variables, but without trying to replicate the problem from the command line, or another branch or something, it’s hard to say more.

Sarah

Hey @snim2 I tried it with again with using cURL but no luck. It works in the brand new repo and fails in the original. The variables are set properly so I’m out of ideas. Wondering if you had any more

Weird! Just to check - both your repo and the forks are using GitLab SASS shared runners? Do you have your own Docker images? You might want to add a call to the API via curl in your pipeline config (.gitlab-ci.yml) and see what happens, that would give you some indication of whether the issue is related to your Python code.

If it is not that, and there is nothing relevant on your project or CI settings, then I guess you have triggered a bug…

Hey @snim2 ! Yes both repos are using the gitlab SaaS shared runners. I moved the curl call into the actual pipeline( .gitlab-ci.yml) rather than the python script and the same behavior happened. So I think I can be fairly certain that the issue is not within my code or pipeline but rather a runner or set of runners that one repo is running on. Thank you for all your suggestions and guidance!! :smiley: