This one is strange and I’ve never seen anything quite like it.
Environment
- gitlab.com cloud service
- Mac/Linux/Windows Runners hosted locally, using shell
- MacBook Air running macOS runner
- Dell Xenon running Windows runnner
- Linux runner running on Ubuntu inside docker on WSL2 on the above Windows computer
- All computers are in the same room, sitting within 2 meters of each other. All are connected to the same access point, and all are running the latest version of gitlab-runner, downloaded yesterday.
- MacBook has integrated wifi card
- Dell has a choice of a wifi dongle or an ethernet connection to a Netgear EX2700 wifi extender.
Observed behavior
MacBook
No issues to report with gitlab-runner on MacBook Air.
Dell
When triggering a build on either Windows or Linux, the build starts and runs on the Dell, but when it comes time to upload the artifact the throughput of the wifi network goes to 0. Devices remain associated to the access point, but none can reach the internet.
This affects all devices-- computers, phones, etc…-- on the access point used by the Dell, but not on other access points which use the same upstream router.
If the Windows computer is connected to another access point (in this case an Asus instead of a Netgear Orbi), running on a separate network, then a similar failure is observed on the new network.
Occasional successes
About one time out of twenty the upload goes through on the Windows gitlab-runner. For the Linux runner, I have only tested 5 times and none worked, so unlike the .gitlab-ci.yml’s Windows configuration we cannot yet conclude that the Linux one is not somehow at fault.
Network observations
Uploading large files from the Dell to TestMy.net Upload Speed Test does not trigger a network collapse. So far, I can only trigger this with the gitlab-runner artifact upload from the Dell.
I also see identical behavior if I’m using the wifi dongle or the ethernet<–>wifi-extender, so we can be sure that it is not a driver problem or some other kind of Windows wifi malfeasance.
Watching the Windows Task Manager’s Performance tab, I can see that there is little-to-no data throughput. So it’s not a case of the channel being slammed/spammed by the computer.
Other data
Gitlab artifacts upload failed - GitLab CI/CD - GitLab Forum seems to have similar error messages, but they are not reporting a wifi network collapse so I suspect it’s similar messages for very different reasons.
Questions
- What on Earth is going on?
- Is there any difference between the uploading routines/code/backend on the Mac vs the Linux and Windows?
- Is there a workaround by which I could upload an artifact manually?
- Are there alternate upload strategies or tuning parameters which could be tried?