Git clean sometime failing with "invalid argument"

I’ve got a local Gitlab installation.
I set up a build/test/deploy pipeline for an ASP.NET Core project using a build agent.
Everything runs under Windows.

Very often, the deployment step fails during the automatic git clean pre-everything step that the Gitlab process does.
I see the process is deleting the files and folders and gets stuck on a particular file with an “invalid argument” error.
Retrying the step then makes it pass.

Because the error happens before the pre-script hooks and anything user-dependent there’s nothing I can do.

Is there a missing setting or something I could do to get around the issue?

Redacted log:

Running with gitlab-runner 12.10.2 (c5874a4b)
on Shared001 xxxx
Preparing the “shell” executor
Using Shell executor…
Preparing environment
Running on xxxx…
DEPRECATION: CMD shell is deprecated and will be removed in 13.0:
Getting source from Git repository
Fetching changes with git depth set to 50…
Reinitialized existing Git repository in xxx
Checking out xxx as xx…
Removing Configuration/bin/
Removing Configuration/obj/
(Some more deletes here…)
warning: failed to remove xxx/xxx.dll: Invalid argument
Uploading artifacts for failed job
ERROR: Job failed: exit status 1