Windows docker executor deletes cache.zip before cache is restored

Hey,

I wanted to make a bug report but before I do that I wanted to ask here, to ensure I have not miss understood how it should work.

It seems that the gitlab-runner helper is removing cache.zip files when it shouldn’t.

My set up is as follows:
I have a gitlab runner (12.1.0) on a windows machine with “Docker desktop community 2.1.0.1”. The runner has the following config.toml:

    [[runners]]
      name = "builder-docker"
      url = "https://git.***********.net/"
      token = "**********************"
      executor = "docker"
      [runners.custom_build_dir]
      [runners.docker]
        tls_verify = false
        image = "alpine:3.9"
        privileged = false
        disable_entrypoint_overwrite = false
        oom_kill_disable = false
        disable_cache = false
        cache_dir = "/cache"
        volumes = ["/cache"]
        shm_size = 0
      [runners.cache]
        [runners.cache.s3]
        [runners.cache.gcs]

My project has the following .gitlab-ci.yml:

    cache:
      key: "sbt-cache"
      paths:
        - "sbt-cache/"    
    variables:
        SBT_OPTS: "-Dsbt.global.base=sbt-cache/.sbtboot -Dsbt.boot.directory=sbt-cache/.boot -Dsbt.ivy.home=sbt-cache/.ivy2 -Dsbt.log.noformat=true"
    build-publish:
      image: openjdk:8
      script:    
        - ls -al
        - ./activator $SBT_OPTS compile
      tags:
        - docker

All seems fine but when I run my job I see the following lines:

    Running with gitlab-runner 12.1.0 (de7731dd)
      on builder-docker **********
    Using Docker executor with image openjdk:8 ...
    Pulling docker image openjdk:8 ...
    Using docker image sha256:08ded5f856cc9c3d9ef07d81d06646c35849b9f0359d0a52a8fab6d43cc2cc5d for openjdk:8 ...
    Running on runner-********-project-60-concurrent-0 via IQIQ-7N9DN3S9UR...
    Fetching changes...
    Reinitialized existing Git repository in /builds/web/api-services/.git/
    Checking out f38f9381 as testing...
    Removing "..\\..\\..\\cache\\web\\api-services\\sbt-cache\\cache.zip"
    Removing .settings/
    Removing modules/common/target/
    Skipping Git submodules setupChecking cache for sbt-cache...
    No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.

The line Removing "..\\..\\..\\cache\\web\\api-services\\sbt-cache\\cache.zip" seems wrong. Also my test line - ls -al shows the project has no sbt-cache folder.

Anyone have any ideas about this. Am I missing something?

Best,
Jonathan

Hi, I am facing the exact same issue. Did you ever get it solved?

Hi,

I’ve run into this problem as well and implemented a fix. I haven’t issued a merge request. If you find my code change helpful, feel free to do that.

I have modified the gitlab-runner which inserts the commands into the helper in the following way:


The important bit is fixing the backslash-containing path with

file = filepath.ToSlash(file)

Over the last two months I have used the modified gitlab-runner quite a lot and successfully. I could not find any side effects when using the patched gitlab-runner.exe with Windows clients as well. The forward slashes are handled gracefully there as well.

The job output would look the following:

Restoring cache
00:01
BUGFIX file before …\cache\my-group\my-project\build-linux-on-windows-master-1\cache.zip…
BUGFIX file after …/…/…/cache/my-group/my-project/build-linux-on-windows-master-1/cache.zip…
Checking cache for build-linux-on-windows-master-1…
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache

Saving cache
00:01
BUGFIX file before …\cache\my-group\my-project\build-linux-on-windows-master-1\cache.zip…
BUGFIX file after …/…/…/cache/my-group/my-project/build-linux-on-windows-master-1/cache.zip…
Creating cache build-linux-on-windows-master-1…
_build/: found 81 matching files and directories
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache

For convenience, I have just rerun the binaries windows/386 windows/amd64 job and constructed the gitlab-runner artifact for download here:

1 Like

One side note on usage:

When using this unofficial gitlab-runner you will have to explicitly specify a helper image in your config.toml file, e.g.

[runners.docker]
helper_image = “gitlab/gitlab-runner-helper:x86_64-738bbe5a”

helper_image = “gitlab/gitlab-runner-helper:x86_64-738bbe5a-servercore1909”

Also see:

Thanks for finding and fixing this. I’m new to using GitLab pipelines and couldn’t figure out why my cache was never restored.

Sadly I dismissed your fix when I first found your post. After re-reading an hour later and having a better understanding of what was happening in my pipeline, I returned and sure enough, this was the problem. I originally failed to make the connection between my cache failure and path separators but another issue I read demonstrated how the cache path was being treated as a filename, so the zip was never actually saved to the cache. A little slow, but I finally made the connection.

I got a patched runner installed and declared a helper_image and for the first time cache was restored properly and my pipeline succeeded!

1 Like