How to launch a job at 11pm only when there was a new push

Hello,

Is it possible to launch a job at 11pm only if there was a new push ?
Can we do this for all branches without creating a scheduler for each branch ?

code I tried :
job:
only:
- pushes
- schedules
script:
- make

Thanks for your help!

Hi @alaska

I think this is tricky, because a pipeline for a push event will be triggered by the push itself, although you can delay the pipeline.

I think you probably want to use rules which are more flexible that only/except and start with something like this:

rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
      when: always
    - when: never

which means that your pipeline or job will only run in schedules, but it will run on every branch. Then if you want to start limiting the branches, you could start by ignoring protected branches:

rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"' && '$CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH'
      when: always
    - when: never

If you want to do something like checking that there are new commits on the branches, then I think you need something a bit more sophisticated. I would be inclined to have a preceding stage in the pipeilne that runs on the schedule and does some Git magic to figure out which branches need building, then pass that information as a text file in an artifact to the stage discussed above.

So, something like this:

stages:
    - what_to_build
    - build_on_schedule

what_to_build:
    stage: what_to_build
    script: 
        - git-magic-goes-here >branches.txt
    artifacts:
        paths:
            - branches.txt
            - expire_in: 1 day
    rules:
        - if: '$CI_PIPELINE_SOURCE == "schedule"' && '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
          when: on_success
        - when: never

build_on_schedule:
    stage: build_on_schedule
    dependencies: what_to_build
    script:
        - build_branches.sh branches.txt
    rules:
        - if: '$CI_PIPELINE_SOURCE == "schedule"' && '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
          when: always
        - when: never

With pipelines there are usually many ways to do the same thing, so there may well be much better solutions than this, but hopefully it gives you some ideas of where you might start.

Sarah

I knew how to delay the pipeline, but I would like to configure a date.

Thank you for your help @snim2 !
It gives me some ideas to continue.

No worries, @alaska

BTW you can configure the date from the UI and also set environment variables there.

I would go with dynamic child pipelines. And a reference here.

Something in this nature:

generate childpipeline:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
      when: always
    - when: never
  script:
    - some_very_smart_script > .child-gitlab-ci.yml
  artifacts:
    paths:
      - .child-gitlab-ci.yml

trigger child pipeline:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
      when: always
    - when: never
  needs:
    - 'generate childpipeline'
  trigger:
    include:
      - artifact: .child-gitlab-ci.yml
        job: 'generate childpipeline'
    strategy: depend

The key component will be the some_very_smart_script that would need to fetch git history and determine what branch had push in last xy hours. While getting date of a push is not easy thing there are some options, but I haven’t tried them with GitLab. After you get that info you can generate the child pipeline file which would include job you need. Last point would be if this can be done on another branch.

@balonik this is a really neat idea. To get the date of the latest push, if the branches have MRs associated with them you could use the API to get the updated_at values for each MR.

Last point would be to trigger pipeline on a different branch then the scheduled one. You can use API to trigger a pipeline on whatever branch you want. Or the Downstream pipelines, but I haven’t tried to trigger one on the same project. So together with a script that would determine which branch got changed this should be able to implement.