I’m trying to build and push tcx2gpx to PyPI automatically and wish to do so only when a tag is applied to a commit so that only versions I want to release get built and uploaded.
Currently I’m testing against Test PyPI using twine and tokens and so I’ve added variabes to the CI/CD for $TWINE_USERNAME (as __token__) and $TWINE_PASSWORD (as the token value itself). The options for both of these variables are set as…
IMPORTANT - I have protected both $TWINE_USERNAME and $TWINE_PASSWORD in their settings by enabling “Protect variable : Export variable to pipelines running on protected branches and tags only”.
I started with the following in my .gitlab-ci.yaml
pypi:
stage: pypi
rules:
- if: $CI_COMMIT_BRANCH == "master"
# - if: $CI_COMMIT_TAG
script:
- pip install .[pypi]
- pip install build
- python -m build
- twine upload --repository testpypi dist/*
…and this works. I’m using setuptools_scm to automatically version and I get packages with post### uploaded (e.g. 0.1.4.post140 and 0.1.5rc3.post3).
This is great it works, however, as mentioned I don’t want everything merged into master to trigger a build and upload so instead I switch the rules to use $CI_COMMIT_TAG and not $CI_COMMIT_BRANCH == "master"…
pypi:
stage: pypi
rules:
# - if: $CI_COMMIT_BRANCH == "master"
- if: $CI_COMMIT_TAG
script:
- pip install .[pypi]
- pip install build
- python -m build
- twine upload --repository testpypi dist/*
This results in the job only being triggered if I commit a tag, which is what I want, however uploading fails with…
Successfully built tcx2gpx-0.1.5rc5.tar.gz and tcx2gpx-0.1.5rc5-py3-none-any.whl
$ twine upload --repository testpypi dist/*
Uploading distributions to https://test.pypi.org/legacy/
Enter your username: Traceback (most recent call last):
File "/usr/local/bin/twine", line 8, in <module>
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.11/site-packages/twine/__main__.py", line 33, in main
error = cli.dispatch(sys.argv[1:])
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/twine/cli.py", line 123, in dispatch
return main(args.args)
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/twine/commands/upload.py", line 198, in main
return upload(upload_settings, parsed_args.dists)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/twine/commands/upload.py", line 127, in upload
repository = upload_settings.create_repository()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/twine/settings.py", line 329, in create_repository
self.username,
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/twine/settings.py", line 131, in username
return cast(Optional[str], self.auth.username)
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/twine/auth.py", line 34, in username
return utils.get_userpass_value(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/twine/utils.py", line 248, in get_userpass_value
value = prompt_strategy()
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/twine/auth.py", line 85, in username_from_keyring_or_prompt
return self.prompt("username", input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/twine/auth.py", line 96, in prompt
return how(f"Enter your {what}: ")
^^^^^^^^^^^^^^^^^^^^^^^^^^^
EOFError: EOF when reading a line
Cleaning up project directory and file based variables 00:00
ERROR: Job failed: exit code 1
To investigate this I added a short debug job and allowed these variables to be shown in the logs…
print-all-env-vars-job:
stage: debug
script:
- echo "GitLab CI/CD | Print all environment variables"
- env
…and re-ran the pipelines using both rules (i.e. one to only build on master and one to run only on tags). Saved the env output and diffed them and it transpires that the $TWINE_USERNAME and $TWINE_PASSWORD variables are not present when running the pipeline on $CI_COMMIT_TAG and so twine is not getting the value of $TWINE_USERNAME passed to it (nor would it get $TWINE_PASSWORD if it got that far) which is I think the cause of the failure.
I’m confused by this because as I noted above I have protected these variables and they should only be _Exported to pipelines running on protected branches and tags only_ (i.e. master or commits with tags) and would therefore expect them to be exported on pipelines that are triggered by tags.
Any suggestions on what I’ve done wrong would be gratefully received.
