CI/CD pipeline - get list of changed files

Hi folks,

On my Gitlab CI/CD pipeline, is there a way I can get a list of the changed files?

Basically, I’ve got some linting set up - but the repository contains a number of separate files which don’t work together to form an application, they’re just individual files hosted in the same place.

I’d like to lint only files changed on commit/push. Is this possible?

Many thanks,
Duncan

1 Like

I have the same question! From what I can tell (below), we’ll have to use the git command: git diff-tree --no-commit-id --name-only -r <commit hash> in our CI scripts to obtain the list of changed files, one per line, and then iterate over that list.

How to get a list of changed files in a commit (GitLab Forum)
How to list all the files in a commit? (Stack Overflow).

2 Likes

Thanks for this @quilty. This worked nicely.

files=(git diff-tree --no-commit-id --name-only -r $CI_COMMIT_SHA)

Can then use a standard Bash for loop to iterate through the files.

1 Like

If there are multiple commits in a merge request, how to get all changed files?

1 Like
git diff

Can work with 2 commits SHAs

Reference:

In case the pipeline runs on merge requests, one option it is to compare it with the target branch.
git diff-tree --name-only --no-commit-id $CI_MERGE_REQUEST_TARGET_BRANCH_SHA

2 Likes

I am running the below command in my pipeline and it throws an error that
git command not found

command:
git diff-tree --no-commit-id --name-only -r

I am trying to capture list of changed files in my pipeline , if there is another way

This command does not work for me.
I get the error fatal: ambiguous argument '{branch_name}': unknown revision or path not in the working tree., even though the merge request originates from this branch.

A quick
git branch -a
returns
* (HEAD detached at b487990)

Which explains why it does not work but I don’t understand why the head is detached in the first place.

FInally found a solution starting from @mpp’s idea, and an answer in this thread:

  1. If this is a private repository, you will need to add a new ssh key in the CI.

  2. Fetch the branch you want to merge:
    git fetch origin $CI_MERGE_REQUEST_TARGET_BRANCH_NAME

  3. Compare the current commit with the branch you want to merge with:
    git diff --name-only $CI_COMMIT_SHA $CI_MERGE_REQUEST_TARGET_BRANCH_NAME

Voilà !

1 Like

You saved my life!

I’ve been experimenting with this, but unless I’m missing something - this only seems to work for public repos. For a private repo it seems that I would need to set up some credentials - for example having an account with a key checked into the repo so I can pass it to the git command, which isn’t crazy but seems a little round-about and potentially insecure for a use case that will become more and more popular - taking action based on a change set, not the full file set.

None of the above really worked for me on a CI/CD pipeline in gitlab but what I ultimately did was very similar to what @loicm did, here’s the complete solution, these are the commands added to my .gitlab-ci.yml:

git checkout $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
DIVERGE=$(git merge-base origin/$CI_MERGE_REQUEST_TARGET_BRANCH_Name $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME)
FILES_CHANGED=$(git diff --name-only $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME $DIVERGE)
echo -e "Files changed since diverge: $FILES_CHANGED"

For some reason a lot of these pre-defined environment variables are empty for me during CI build for a merge request:

somejob:
  script:
    - echo $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
    - echo $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
    - echo $CI_MERGE_REQUEST_SOURCE_BRANCH_SHA
    - echo $CI_MERGE_REQUEST_TARGET_BRANCH_SHA

I tried multiple suggestions above and none of them work for me because these variables have no values. The only one that has value is $CI_COMMIT_SHA

hi guys, I just wanted to drop my solution. It’s based on all the answers here so I wanted to share back, maybe it is useful for someone. I wanted to find all changed yaml files in my merge-request to run a job against it. Solution is pretty easy once you know it:

script:
    - git fetch
    - git diff --name-only origin/$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME | grep '\.yaml$' | xargs yamllint
1 Like