Run builds in parallel with serial pipelines

Hi,

I have some troubles with GitLab CI configuration.
I would like my builds on the same stage to be run in parallel while running pipelines in serial.

Here is my .gitlab-ci.yml:

build:
  stage: build
  script:
    - bundle install
    - cp config/mongoid.yml.example config/mongoid.yml
    - cp config/settings.yml.example config/settings.yml
  artifacts:
    expire_in: 10 mins
    untracked: true

test:rspec:
  stage: test
  script:
    - RAILS_ENV=test bundle exec rspec --tty

test:brakeman:
  stage: test
  script:
    - bundle exec brakeman -z
  dependencies: []

test:rubocop:
  stage: test
  script:
    - bundle exec rubocop --color --fail-level R
  dependencies: []

test:codeclimate:
  stage: test
  script:
    - bundle exec rubycritic --mode-ci --no-browser app
  artifacts:
    paths:
      - tmp/rubycritic/
    expire_in: 10 mins
  dependencies: []

deploy:develop:
  stage: deploy
  script:
    - bundle exec cap new deploy
  only:
    - develop
  dependencies: []

deploy:localhost:
  stage: deploy
  script:
    - bundle exec cap develop deploy
  only:
    - develop
  dependencies: []

deploy:codeclimate:
  stage: deploy
  script:
    - "[[ $CI_BUILD_REF_NAME = develop ]] && CODECLIMATE_BRANCH=develop || CODECLIMATE_BRANCH=current"
    - "CODECLIMATE_PATH=/home/dev/codeclimate/$CODECLIMATE_BRANCH"
    - "rm -rf $CODECLIMATE_PATH"
    - "cp -r $(pwd)/tmp/rubycritic $CODECLIMATE_PATH"
  dependencies:
    - test:codeclimate

I have concurrent = 2 in config.toml.

test:rspec is the longest build on test stage.
There can be situation when pipeline A runs test:rspec and here comes pipeline B and also starts to run test:rspec. They use the same DB so most of specs fail.

Is there any way to enforce my GitLab CI to run pipelines sequently while running my test:* builds in parallel?

Thanks.

UPDATE

I came up with the solution as follows:

Created rake task in order to dynamically prepare mongoid config file

namespace :ci do
  namespace :config do
    desc 'Set up all configs'
    task setup: %w(mongoid settings)

    desc 'Set up mongoid config'
    task :mongoid do
      config = YAML.load_file(Rails.root.join('config', 'mongoid.yml.example'))
      config['test']['clients']['default']['database'] = "exactfarming_proto_test_#{ENV['CI_PIPELINE_ID']}"
      File.open(Rails.root.join('config', 'mongoid.yml'), 'w') { |file| file << config.to_yaml }
    end

    desc 'Set up settings'
    task :settings do
      src  = Rails.root.join('config', 'settings.yml.example')
      dest = Rails.root.join('config', 'settings.yml')
      FileUtils.cp(src, dest)
    end
  end
end

Updated .gitlab-ci.yml to invoke rake task

build:
  stage: build
  script:
    - bundle install
    - bundle exec rake ci:config:setup
  artifacts:
    expire_in: 10 mins
    untracked: true

test:rspec:
  stage: test
  script:
    - RAILS_ENV=test bundle exec rspec --tty
  after_script:
    - RAILS_ENV=test bundle exec rake db:mongoid:drop

...

I dont think this is possible. Guess your true objective would be to make your test:* jobs less dependent of one database.

Thank you @stefanvangastel. Yes, I thought about it.
So I should create temporary databases for each test:rspec build?
Is there any convenient way to do this? Right now I have DB name hardcoded in config/mongoid.yml.example config file which is copied on during build stage.

Assuming you use mongodb based on you reference to a config file I think you could/should create a new database before each test job, giving it a dynamic name based on a Gitlab CI pre-defined env var like db_$CI_BUILD_ID making it unique for that specific job (allowing test jobs to run in parallel with different databases).

Example (And I don’t know ruby or your context so; read it as pseudo code pls)

test:rspec:
  stage: test
  script:
    # Create database
    - mongo db_$CI_BUILD_ID --eval "db.getName()"
    # Run tests
    - RAILS_ENV=test bundle exec rspec --tty
  after_script:
    # Remove database (always, even when failing otherwise pollution)
    - mongo db_$CI_BUILD_ID --eval "db.dropDatabase()"

Thank you very much @stefanvangastel.
I am working in using $CI_BUILD_ID way.

Cool, keep us posted on issues or a solution!

I have update original post with the solution. Thanks once again @stefanvangastel!

Nice, np! (Can you add [SOLVED] to your post title?)

Done. Should I close topic somehow?

Dont know if that is possible. Dont think so.