Custom testing with Auto Devops

Infrastructure as Code & Cloud Native
,
1

I’m currently running an auto devops pipeline for a python based rest API (flask), and so far I’m pretty happy with it. It builds to the gitlab registry and deploys to my on-premise kubernetes cluster in all environment, very clean, very easy.

What I’d like to do is introduce a test plan into the cycle, ideally using something like postman/newman against a rest API. Is this possible?

I don’t want to copy all of the auto devops pipeline and customize it as it’s really clean the way things are now. Maybe something can be done in the herokuish buildpack? I can’t seem to find any decent documentation on Herokuish and how I might use it effectively so I’m really not sure if that’s the right approach or not.

2

I’m not sure if this is the best way, but I was able to “solve” my problem by using a .gitlab-ci.yml file that looks like this:

variables:
  ROLLOUT_RESOURCE_TYPE: deployment

include:
  - template: Jobs/Build.gitlab-ci.yml
  - template: Jobs/Deploy.gitlab-ci.yml
  
stages:
  - build
  - review
  - staging
  - test_postman
  - production
  - canary
  - incremental rollout 10%
  - incremental rollout 25%
  - incremental rollout 50%
  - incremental rollout 100%
  - cleanup

postman_tests:
  stage: test_postman
  image: 
    name: postman/newman:5.2-alpine
    entrypoint: [""]
  script:
    - newman --version
    - newman run demo_python.postman_collection.json --environment demo-python-staging.postman_environment.json --insecure
  rules:
    - if: '$CI_COMMIT_BRANCH != "master"'
      when: never
    - if: '$STAGING_ENABLED'

  
production_manual:
  environment:
    url: https://demo-python.$KUBE_INGRESS_BASE_DOMAIN
    kubernetes:
      namespace: demo-python-prod

staging:
  environment:
    url: https://demo-python.$KUBE_INGRESS_BASE_DOMAIN
    kubernetes:
      namespace: demo-python-staging

review:
  environment:
    url: https://demo-python.$KUBE_INGRESS_BASE_DOMAIN
    kubernetes:
      namespace: demo-python-review

This way, my test is run after staging, and against that environment with a manual step to deploy to production (assuming it all passed). Hope this is able to help someone else out.