Django System Checks, Migrations, and Tests Pass Locally, but Fail in CI/CD Environment

How to Use GitLab
, , ,
1

Problem

Honestly, I’m just firing this one into the dark because I have exhausted every possible avenue before asking this. I’m not sure what the problem is here.

I have a Django application that works completely fine to run locally. I can run migrations. I have developed at length with this locally, and not had a single issue with the models, the testing, or any feature.

The issue here is that the second I use GitLab’s CI/CD Runner and perform the exact same steps I’m performing locally I get this output.

ERRORS:
piano_gym_api.LearnerEnrolledLesson.enrolled_course: (fields.E300) Field defines a relation with model 'piano_gym_api.LearnerEnrolledCourse', which is either not installed, or is abstract.
piano_gym_api.LearnerEnrolledLesson.enrolled_course: (fields.E307) The field piano_gym_api.LearnerEnrolledLesson.enrolled_course was declared with a lazy reference to 'piano_gym_api.learnerenrolledcourse', but app 'piano_gym_api' doesn't provide model 'learnerenrolledcourse'.
piano_gym_api.LearnerEnrolledLesson.enrolled_school: (fields.E300) Field defines a relation with model 'piano_gym_api.LearnerEnrolledSchool', which is either not installed, or is abstract.
piano_gym_api.LearnerEnrolledLesson.enrolled_school: (fields.E307) The field piano_gym_api.LearnerEnrolledLesson.enrolled_school was declared with a lazy reference to 'piano_gym_api.learnerenrolledschool', but app 'piano_gym_api' doesn't provide model 'learnerenrolledschool'.

Environment

I’m using Python 3.7 with Django 2.2. My dependencies look like this:

certifi==2019.3.9
chardet==3.0.4
coreapi==2.3.3
coreschema==0.0.4
Django==2.2
django-cors-headers==3.0.2
django-extensions==2.1.7
djangorestframework==3.9.4
djangorestframework-jwt==1.11.0
gunicorn==19.9.0
idna==2.8
itypes==1.1.0
Jinja2==2.10.1
lxml==4.3.3
MarkupSafe==1.1.1
music21==5.5.0
PyJWT==1.7.1
pytz==2019.1
requests==2.22.0
six==1.12.0
sqlparse==0.3.0
uritemplate==3.0.0
urllib3==1.25.3
whitenoise==4.1.2

I’m using the free version of GitLab, with GitLab Runner.

It’s a simple django project. There is one project, and one app.

My settings.conf's INSTALLED_APPS looks like this

# Application definition
INSTALLED_APPS = [
    # Django Default
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    # Third-Party Apps
    "corsheaders",
    "django_extensions",
    "rest_framework",
    "rest_framework.authtoken",
    "whitenoise.runserver_nostatic",
    # Custom Apps
    "piano_gym_api",
]

Steps to Run Locally

This will pass

  • pip3 install virtualenv
  • virtualenv -p python3 venv
  • source venv/bin/activate
  • pip3 install -r requirements.txt
  • python3 manage.py makemigrations piano_gym_api
  • python3 manage.py migrate
  • python3 manage.py test

Steps to Run in GitLab CI/CD

This will fail

I’ve installed the GitLab Runner

I created a .gitlab-ci.yml file in my root directory. All it has is this:

stages:
  - test

api-test:
  stage: test
  image: python:3.7
  script:
    - cd piano_gym_back_end
    # Create environment for python
    - pip3 install virtualenv
    - virtualenv -p python3 venv
    - source venv/bin/activate
    - pip3 install -r requirements.txt
    # Set up and run tests
    - python3 manage.py makemigrations piano_gym_api
    - python3 manage.py migrate
    - python3 manage.py test

Then I commit everything on the branch and run
gitlab-runner exec docker api-test

Which then goes through everything and outputs this

$ python3 manage.py makemigrations piano_gym_api
SystemCheckError: System check identified some issues:

ERRORS:
piano_gym_api.LearnerEnrolledLesson.enrolled_course: (fields.E300) Field defines a relation with model 'piano_gym_api.LearnerEnrolledCourse', which is either not installed, or is abstract.
piano_gym_api.LearnerEnrolledLesson.enrolled_course: (fields.E307) The field piano_gym_api.LearnerEnrolledLesson.enrolled_course was declared with a lazy reference to 'piano_gym_api.learnerenrolledcourse', but app 'piano_gym_api' doesn't provide model 'learnerenrolledcourse'.
piano_gym_api.LearnerEnrolledLesson.enrolled_school: (fields.E300) Field defines a relation with model 'piano_gym_api.LearnerEnrolledSchool', which is either not installed, or is abstract.
piano_gym_api.LearnerEnrolledLesson.enrolled_school: (fields.E307) The field piano_gym_api.LearnerEnrolledLesson.enrolled_school was declared with a lazy reference to 'piano_gym_api.learnerenrolledschool', but app 'piano_gym_api' doesn't provide model 'learnerenrolledschool'.
ERROR: Job failed: exit code 1
FATAL: exit code 1

Models

Now I understand that this is stating that it can’t find the models in the app piano_gym_api. But that doesn’t make sense.

The model here is:

class LearnerEnrolledLesson(Model):
    is_enrolled = BooleanField(default=True)
    learner = ForeignKey("piano_gym_api.Learner", on_delete=CASCADE)
    # ---
    enrolled_school = ForeignKey("piano_gym_api.LearnerEnrolledSchool", on_delete=CASCADE)
    enrolled_course = ForeignKey("piano_gym_api.LearnerEnrolledCourse", on_delete=CASCADE)
    # ---
    school = ForeignKey(School, on_delete=CASCADE)
    course = ForeignKey(SchoolCourse, on_delete=CASCADE)
    lesson = ForeignKey(SchoolLesson, on_delete=CASCADE)
    order = IntegerField(default=1)

    REQUIRED_FIELDS = ["learner", "school", "course", "lesson", "enrolled_school", "enrolled_course"]

    objects = LearnerEnrolledLessonManager()

    class Meta:
        ordering = ("order",)
        unique_together = ("learner", "school", "course", "lesson", "enrolled_school", "enrolled_course", "order")

The only thing I’m doing here is using strings to reference the piano_gym_api.LearnerEnrolledSchool and piano_gym_api.LearnerEnrolledCourse.

This is only done because those models have a function that returns a LearnerEnrolledLesson and that’s a circular dependency, so I have to reference the models without using the import path.

Appeal For Help

I have no idea why this is failing in my CI/CD docker environment. I’m not doing anything different. My settings.py isn’t changing between the develop environment and the ci/cd environment. And, the steps are the EXACT same.

What could I possibly be doing wrong here?

2

Could you also post your database connection string? My guess is that when you developed locally you have ran postgres in the background. However, in the gitlab ci runner postgres is not configured yet. To address this you should add some additional lines to your config file. Here’s my guess

# ADD THIS
services: 
  -  postgres:latest

stages:
  - test

api-test:
  stage: test
  image: python:3.7
  # you may need to configure the database url here if you are using dj_database
  # if you are already using the postgres connection dict you may omit this, example:
  # DATABASES = {
  #     'default': {
  #        'ENGINE': 'django.db.backends.postgresql_psycopg2',
  #        'NAME': 'your-db-name-here',
  #        'USER': 'postgres',
  #        'PASSWORD': 'postgres',
  #        'HOST': 'postgres',
  #        'PORT': '5432',
  #    },
  # }
  variables:
    DATABASE_URL: "postgresql://postgres:postgres@postgres:5432/<your-db-name-here>"
  script:
    - cd piano_gym_back_end
    # Create environment for python
    - pip3 install virtualenv
    - virtualenv -p python3 venv
    - source venv/bin/activate
    - pip3 install -r requirements.txt
    # Set up and run tests
    - python3 manage.py makemigrations piano_gym_api
    - python3 manage.py migrate
    - python3 manage.py test

I hope this helps.

3

@jseam thank you for taking the time to assist on this!

I did manage to get this working and posted my solution on stackoverflow! The issue was an interesting problem that arose because of my model directory hierarchy in the file system!