Gitlab pipeline stuck in running state until timeout

Hi Guys!

I’m using pipelines to build the software but after successful build and return Job Success the pipeline keeps in running state until timeout and return fail status:

The below screenshot is what i’m getting on the pipeline:

My pipeline

stages:
  - build_windows
 
build_windows:
  stage: build_windows
  image: mrdasilva/mpswinbuild
  allow_failure: true
  tags:
    - runner-windows
  script:
    - powershell ./gradlew.bat build
    - echo "Finished build_windows"

`

Versions and Infos:

• Self-hosted GitLab-CE: 12.8.2
• Runner: 13.9.0~beta.84.ge25e39c6 Also tested with 13.7.8
• The runner is installed in a windows dektop running docker-desktop

Gitlab Runner

gitlab-runner.exe -version
Version:      13.9.0~beta.84.ge25e39c6
Git revision: e25e39c6
Git branch:   master
GO version:   go1.13.8
Built:        2021-01-25T16:42:03+0000
OS/Arch:      windows/amd64

Runner config.toml

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "GIT-Runner-01"
  url = "http://gk-git/"
  token = "p4cwuSVyTzDgGs-qqcRW"
  executor = "docker-windows"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "mcr.microsoft.com/windows/servercore:1909"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["c:\\cache"]
    shm_size = 0

• The docker image used in the pipeline is based on the same docker-image used on the runner

The Debug trace returns this:

Use '--warning-mode all' to show the individual deprecation warnings.

[5423](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5423) See https://docs.gradle.org/6.7.1/userguide/command_line_interface.html#sec:command_line_warnings

[5424](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5424) BUILD SUCCESSFUL in 42m 44s

[5425](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5425) 35 actionable tasks: 35 executed

[5426](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5426) DEBUG: 1+ if( >>>> !$?) { Exit &{if($LASTEXITCODE) {$LASTEXITCODE} else {1}}

[5427](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5427) }

[5428](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5428) DEBUG: ! CALL function '<ScriptBlock>'

[5429](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5429) DEBUG: 1+ >>>> echo "`$ echo `"Finished build_windows`""

[5430](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5430) DEBUG: ! CALL function '<ScriptBlock>'

[5431](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5431) $ echo "Finished build_windows"

[5432](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5432) DEBUG: 1+ >>>> echo "Finished build_windows"

[5433](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5433) DEBUG: ! CALL function '<ScriptBlock>'

[5434](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5434) Finished build_windows

[5435](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5435) DEBUG: 1+ if( >>>> !$?) { Exit &{if($LASTEXITCODE) {$LASTEXITCODE} else {1}}

[5436](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5436) }

[5437](http://gk-git/dev/MediaProcessorService/-/jobs/444#L5437) DEBUG: ! CALL function '<ScriptBlock>'

The pipeline uses a windows container to execute a gradle build and is returning a successful state
Any ideas what this could be related to?

I’m really uncertain about this. I have/had the same problem, but for me it was not always reproducible. Like 1 in 3 times this behavior occurred.
I suspect that since gradle starts a daemon (and that maybe paired with some faulty powershell behavior) somehow gitlab waits for the gradle daemon to stop and therefore doesn’t finish the job. Could you try calling “gradlew.bat --stop” after the build task and see if that works? I haven’t been able to reproduce it ever since (but I also only implemented this today, so let’s see )
Update: I think it really works. But it also seems to work with the option “–no-daemon”

Hi,

Thank you for sharing, I’ve changed our script and included the “gradlew.bat --stop” and looks like it really fixed it.
4 days and no stuck pipelines until now.

Thank you for sharing. This solved my issue as well.

Thanks for sharing, this also worked like a charm for me!