Missing rights for k8s project serviceaccount on CRDs

Infrastructure as Code & Cloud Native
,
1

Hey guys,

I’m struggling a bit with k8s integration when dealing with cert-manager ressources. Right now I deployed my own helm, cert-manager and nginx-ingress due to limitation of running from baremetal server at home. When the gitlab CI tries to deploy the certificates and/or issuer, i get errors as bellow :

when i use cluster issuer:
from server for: "ingress-cert.yaml": certificates.certmanager.k8s.io "<project>-letsencrypt-crt" is forbidden: User "system:serviceaccount:<project>:<project>-service-account" cannot get resource "certificates" in API group "certmanager.k8s.io" in the namespace "<project>"

when i try to create a namespace issuer:
"issuer-cert-staging.yaml": issuers.certmanager.k8s.io "<project>-gitlab-k8s- letsencrypt" is forbidden: User "system:serviceaccount:<project>:<project>-service- account" cannot get resource "issuers" in API group "certmanager.k8s.io" in the namespace "<project>"

Service account for the project was created after the deployment of cert-manager and associated CRD

My CI is super simple:

deploy_live:
image:
name: lachlanevenson/k8s-kubectl:latest
entrypoint: [“/bin/sh”, “-c”]
stage: deploy
environment:
name: production
script:
- kubectl apply -f deployment.yaml
- kubectl apply -f service.yaml
- kubectl apply -f issuer-cert-staging.yaml

Everything else is working fine as long as I dont touch to cert-manager CRD. I had similar errors when using the integrated helm/nginx-ingress/cert-manager apps, but again, due to limitation of my setup I removed them.

k8s, helm and all charts (nginx-ingress, cert-manager) are running the latest version with RBAC enabled.

Edit:
After creating some custom roles and assign them to the above serviceaccount, deployment goes through.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
rbac.authorization.k8s.io/aggregate-to-view: “true”
rbac.authorization.k8s.io/aggregate-to-edit: “true”
rbac.authorization.k8s.io/aggregate-to-admin: “true”
name: cert-manager-view
rules:

  • apiGroups:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
rbac.authorization.k8s.io/aggregate-to-edit: “true”
rbac.authorization.k8s.io/aggregate-to-admin: “true”
name: cert-manager-edit
rules:

  • apiGroups:
    • “certmanager.k8s.io”
      resources:
    • issuers
    • certificates
      verbs:
    • create
    • delete
    • deletecollection
    • patch
    • update
kubectl create rolebinding cert-read --clusterrole=cert-manager-view --serviceaccount=<project>:<project>-service-account
kubectl create rolebinding cert-edit --clusterrole=cert-manager-edit --serviceaccount=<project>:<project>-service-account

I’m just confused how I’m suppose to run that from the gitlab CI…

Any idea?

1 Like
2

I think what’s needed is a way to add custom roles to the service accounts created by the k8s integration in gitlab.
Out of the box, they cannot work with anything that uses CRDs.

3

Definitely. I ended up using ArgoCD for such small applications, it has all the required rights at installation and is easier to deal with, but doesnt have all the fancy gitlab options.