Adding CI/CD Schedules using cURL. Variables are not getting added

I Wrote a Python script to add 62 schedules for my pipeline. These schedules get added but the pipeline variables are not getting added.

Here is how I attempted to do this:

command = f'curl -k --request POST --header "PRIVATE-TOKEN: {private_token}" --form description="{description}" --form ref="main" --form cron="{cron_fields[0]} {cron_fields[1]} {cron_fields[2]} {cron_fields[3]} {cron_fields[4]}" --form cron_timezone="America/Chicago" --form active="true" --form "variables[mac]={mac_address}" --form "variables[Check]=False" --form "variables[Package]=all" --form "variables[Iterations]=3" --form "variables[Ticket]={ticket}" "{gitlab_url}"'

As I mentioned, the schedules are added with the proper description and cron attributes. The only thing not getting added are those variables.

I could not find documentation on how to add the variables but I did run across a suggestion which I tried.

Would anyone here be able to point me in the right direction.

Thank you for taking the time in reading this request.

I have not tried it, but reading the docs, it seems that the initial schedule create API event does not support setting variables.

This is documented as a separate action after the schedule has been created.

Thank you dnsmichi. This makes things a bit more cumbersome to automate adding schedules that have pipeline parameters that need to be filled out to prior to running.

I’ll play around a bit but in the end I might just have to bite the bullet and add all 62 manually.

I’ll update if I get this working.

Unfortunately that did not work for me.

curl --request POST --header "PRIVATE-TOKEN: <my token>" --form "key=mac"  --form "value=e4:bf:fa:fd:88:68" "https://<my gitlab>/api/v4/projects/526/pipeline_schedules/100/variables"

failed with the following: curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it.

I have a total of 5 pipeline parameters I need to add. I just attempted with one.

I used that same token as I did to add the schedule and the same Gitlab URL with the addition of the schedule number and the word variables.

Spoke too soon. I just needed to add a -k to the curl line. Now to see if I can get multiple parameters then automate all this.

Thank you for the help. I greatly appreciate it.

1 Like

Glad you could solve it yourself.

If curl does not allow for chaining requests in an efficient way, you might want to look into programmatic libraries, such as python-gitlab.

I wrote a longer log post about the python-gitlab library with use case example code that can be helpful too (I never looked into pipeline schedules though - the python-gitlab docs look promising.)

Here is how I did it. I wanted to automate the entire thing, pulling the existing schedules and looping through a file I have. It might be kludgy but it worked:

import argparse
import re
import time

import requests

# Setting command line options
if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument("-v", "--version", required=True, metavar="dev/prod", help="Which version of Gitlab do you want to run against, dev or prod")
    args = parser.parse_args()
    if(args.version):
        gitlab_version = args.version

def get_pipeline_schedules(url, token):
    headers = {
        "PRIVATE-TOKEN": token
    }

    response = requests.get(url, headers=headers, verify=False)

    if response.status_code == 200:
        data = response.json()
        return data
    else:
        print(f"Failed to retrieve pipeline schedules. Status code: {response.status_code}")
        return None

def extract_mac_address(description):
    pattern = r'([0-9A-Fa-f]{2}(?::[0-9A-Fa-f]{2}){5})'
    match = re.search(pattern, description)
    if match:
        return match.group(0)
    else:
        return None

def read_teqa_file(file_path):
    teqa_data = {}
    with open(file_path, 'r') as file:
        for line in file:
            match = re.search(r'-m\s+([\w:]+)\s+-x\s+(\S+)', line)
            if match:
                mac_address = match.group(1)
                teqa_id = match.group(2)
                teqa_data[mac_address] = teqa_id.strip()
    return teqa_data

def main():
    if gitlab_version == "dev":
        my_url = "<dev_url"
        my_token = "<dev_token>"
        project_id = 27
        text_file = "teqa_data.txt"
    elif gitlab_version == "prod":
        my_url = "<prod_url>"
        my_token = "<prod_token>"
        project_id = 526
        text_file = "cron_jobs.txt"
    endpoint = f"/api/v4/projects/{project_id}/pipeline_schedules"

    full_url = my_url + endpoint

    data = get_pipeline_schedules(full_url, my_token)
    
    if data:
        teqa_data = read_teqa_file(text_file)

        for schedule in data:
            top_level_id = schedule["id"]
            description = schedule["description"]
            mac_address = extract_mac_address(description)

            if mac_address in teqa_data:
                teqa_id = teqa_data[mac_address]

                variables_endpoint = f"{my_url}/api/v4/projects/{project_id}/pipeline_schedules/{top_level_id}/variables"
                headers = {
                    "PRIVATE-TOKEN": my_token
                }

                data_to_post = [
                    {"key": "mac", "value": mac_address},
                    {"key": "FW_Check", "value": "False"},
                    {"key": "DataPackage", "value": "all"},
                    {"key": "Iterations", "value": "3"},
                    {"key": "XrayTicket", "value": teqa_id}
                ]
                
                print("\n")

                for item in data_to_post:
                    print(f"curl -k --request POST --header \"PRIVATE-TOKEN: {my_token}\" --form \"key={item['key']}\" --form \"value={item['value']}\" \"{variables_endpoint}\"\n")
                    time.sleep(1)
                    response = requests.post(variables_endpoint, headers=headers, verify=False, data=item)
                    if response.status_code == 201:
                        print(f"Variable '{item['key']}' added for top-level ID {top_level_id}")
                    else:
                        print(f"Failed to add variable '{item['key']}' for top-level ID {top_level_id}. Status code: {response.status_code}")
                print("\n")

if __name__ == "__main__":
    main()

My issue now, which I opened another chat for, is I’m only getting the first 20 schedules and not all 62.

1 Like