First CI/CD Pipeline Won't deploy

I am trying to create my first GitLab CI/CD pipeline. The goal is to automatically build and deploy a new project version with updated semantic version number based on the end of the commit message Sem-Ver: bugfix/feature/etc. However, I can’t even get the initial pipeline to deploy a build. I have the following project .gitlab-ci.yml file (located at .gitlab-ci.yml · glicka-setup-pipeline · Adam Glick / Merlot · GitLab):

# This file is a template, and might need editing before it works on your project.
# This is a sample GitLab CI/CD configuration file that should run without any modifications.
# It demonstrates a basic 3 stage CI/CD pipeline. Instead of real tests or scripts,
# it uses echo commands to simulate the pipeline execution.
#
# A pipeline is composed of independent jobs that run scripts, grouped into stages.
# Stages run in sequential order, but jobs within stages run in parallel.
#
# For more information, see: https://docs.gitlab.com/ee/ci/yaml/index.html#stages
#
# You can copy and paste this template into a new `.gitlab-ci.yml` file.
# You should not add this template to an existing `.gitlab-ci.yml` file by using the `include:` keyword.
#
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Getting-Started.gitlab-ci.yml
variables:
  TWINE_PASSWORD: "#MerlotCICDPW1"
  TWINE_REPOSITORY_URL: "https://gitlab.com/glicka/merlot/-/packages"
  TWINE_USERNAME: "merlot"

stages:          # List of stages for jobs, and their order of execution
  - build
  - test
  - deploy

build-job:       # This job runs in the build stage, which runs first.
  stage: build
  image: python:3.11-bullseye
  # only:
  #   - tags
  script:
    - echo "Compiling the code..."
    - pip install twine wheel
    - python setup.py sdist bdist_wheel
    - twine upload dist/*
    - echo "Compile complete."

unit-test-job:   # This job runs in the test stage.
  stage: test    # It only starts when the job in the build stage completes successfully.
  script:
    - echo "Running unit tests... This will take about 60 seconds."
    - sleep 60
    - echo "Code coverage is 90%"

lint-test-job:   # This job also runs in the test stage.
  stage: test    # It can run at the same time as unit-test-job (in parallel).
  script:
    - echo "Linting code... This will take about 10 seconds."
    - sleep 10
    - echo "No lint issues found."

deploy-job:      # This job runs in the deploy stage.
  stage: deploy  # It only runs when *both* jobs in the test stage complete successfully.
  environment: production
  script:
    - echo "Deploying application..."
    - echo "Application successfully deployed."

and the pipeline is failing with

$ twine upload dist/*
Uploading distributions to https://gitlab.com/glicka/merlot/-/packages
Uploading merlot-0.0.1-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 kB • 00:00 • ?
WARNING  Received "500: Internal Server Error"                                  
         Package upload appears to have failed. Retry 1 of 5.                   
Uploading merlot-0.0.1-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 kB • 00:00 • ?
WARNING  Received "500: Internal Server Error"                                  
         Package upload appears to have failed. Retry 2 of 5.                   
Uploading merlot-0.0.1-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 kB • 00:00 • ?
WARNING  Received "500: Internal Server Error"                                  
         Package upload appears to have failed. Retry 3 of 5.                   
Uploading merlot-0.0.1-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 kB • 00:00 • ?
WARNING  Received "500: Internal Server Error"                                  
         Package upload appears to have failed. Retry 4 of 5.                   
Uploading merlot-0.0.1-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 kB • 00:00 • ?
WARNING  Received "500: Internal Server Error"                                  
         Package upload appears to have failed. Retry 5 of 5.                   
WARNING  Error during upload. Retry with the --verbose option for more details. 
ERROR    HTTPError: 500 Internal Server Error from                              
         https://gitlab.com/glicka/merlot/-/packages                            
         Internal Server Error                                                  
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1

Can you please help me set up the .gitlab-ci.yml file so that it builds and deploys the project correctly with the correct semantic version number?

Thank you!

Hey,

I believe your TWINE URL is wrong. Could you try pointing it to the API endpoint for PYPI package:

${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi
1 Like

I tried to do that, but got the same error:

$ twine upload dist/*
Uploading distributions to https://gitlab.com/glicka/merlot/-/packages
Uploading merlot-0.0.2-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.3/9.3 kB • 00:00 • ?
WARNING  Received "500: Internal Server Error"                                  
         Package upload appears to have failed. Retry 1 of 5.                   
Uploading merlot-0.0.2-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.3/9.3 kB • 00:00 • ?
WARNING  Received "500: Internal Server Error"                                  
         Package upload appears to have failed. Retry 2 of 5.                   
Uploading merlot-0.0.2-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.3/9.3 kB • 00:00 • ?
WARNING  Received "500: Internal Server Error"                                  
         Package upload appears to have failed. Retry 3 of 5.                   
Uploading merlot-0.0.2-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.3/9.3 kB • 00:00 • ?
WARNING  Received "500: Internal Server Error"                                  
         Package upload appears to have failed. Retry 4 of 5.                   
Uploading merlot-0.0.2-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.3/9.3 kB • 00:00 • ?
WARNING  Received "500: Internal Server Error"                                  
         Package upload appears to have failed. Retry 5 of 5.                   
WARNING  Error during upload. Retry with the --verbose option for more details. 
ERROR    HTTPError: 500 Internal Server Error from                              
         https://gitlab.com/glicka/merlot/-/packages                            
         Internal Server Error                                                  
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1

you can see the change here: .gitlab-ci.yml · glicka-setup-pipeline · Adam Glick / Merlot · GitLab

Hey,

I looked at your latest job, twine still seems to be using wrong URL:

Weird.

I don’t know if this will help, but I would try moving those variables inside the build-job and making sure twine reads from them.

P.S. This is probably not the reason, but I’d still suggest you to remove those credentials from .gitlab-ci.yml file. It’s a security issue commiting secrets and additionally using them here.
Using CI_JOB_TOKEN will do the job as well. Otherwise you can save your credentials in project’s CI/CD variables. Here’s my working sample of upload package job:

upload package:
  stage: release
  image: python:3.10-alpine
  dependencies:
    - "build package"
  variables:
    TWINE_USERNAME: 'gitlab-ci-token'
    TWINE_PASSWORD: '${CI_JOB_TOKEN}'
    TWINE_REPOSITORY_URL: '${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi'
  before_script:
    - pip install twine
  script:
    - python -m twine upload --cert $CA_CERT dist/*
  only:
    - main