I just upgraded our gitlab to 12.0.3. I was glad to see that there is now a garbage collector for docker registries. I was not so glad when it SIGSEGV’d on me. Has anyone seen this before? Any clues as to why it blows up?
root@gitlab:~# gitlab-ctl registry-garbage-collect
ok: down: registry: 0s, normally up
Running garbage-collect using configuration ["/opt/gitlab/embedded/bin/registry", “garbage-collect”, “/var/opt/gitlab/registry/config.yml”], this might take a while…
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xc5965d]
goroutine 1 [running]:
github.com/docker/distribution/registry/storage/driver/s3-aws.(*driver).doWalk.func1(0xc000333700, 0xc00042e601, 0xc0001e76e8)
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/registry/storage/driver/s3-aws/s3.go:973 +0x9d
github.com/docker/distribution/vendor/github.com/aws/aws-sdk-go/service/s3.(*S3).ListObjectsV2PagesWithContext(0xc0000aa7d0, 0x7f9ab40ac838, 0xc00043ee70, 0xc0002a0e10, 0xc00016f7e8, 0x0, 0x0, 0x0, 0x1, 0x2)
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/vendor/github.com/aws/aws-sdk-go/service/s3/api.go:4198 +0x111
github.com/docker/distribution/registry/storage/driver/s3-aws.(*driver).doWalk(0xc000333680, 0xfff4a0, 0xc00043ee00, 0xc0001e78a8, 0xc00042a8d1, 0x20, 0xeb34f5, 0x1, 0xc000440dc0, 0x0, …)
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/registry/storage/driver/s3-aws/s3.go:971 +0x3a1
github.com/docker/distribution/registry/storage/driver/s3-aws.(*driver).Walk(0xc000333680, 0xfff4a0, 0xc00043ee00, 0xc000319e40, 0x20, 0xc000440dc0, 0x2, 0x0)
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/registry/storage/driver/s3-aws/s3.go:919 +0x160
github.com/docker/distribution/registry/storage/driver/base.(*Base).Walk(0xc000444420, 0xfff4a0, 0xc00043ee00, 0xc000319e40, 0x20, 0xc000440dc0, 0x0, 0x0)
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/registry/storage/driver/base/base.go:239 +0x234
github.com/docker/distribution/registry/storage.(*registry).Enumerate(0xc00043ed90, 0xfff1e0, 0xc00044a630, 0xc00042e640, 0x0, 0x0)
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/registry/storage/catalog.go:67 +0xe3
github.com/docker/distribution/registry/storage.MarkAndSweep(0xfff1e0, 0xc00044a630, 0x100b340, 0xc000444420, 0x10013a0, 0xc00043ed90, 0x0, 0x10013a0, 0xc00043ed90)
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/registry/storage/garbagecollect.go:40 +0x1e7
github.com/docker/distribution/registry.glob..func3(0x1733fc0, 0xc000325a10, 0x1, 0x1)
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/registry/root.go:80 +0x48a
github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).execute(0x1733fc0, 0xc0003259c0, 0x1, 0x1, 0x1733fc0, 0xc0003259c0)
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:495 +0x191
github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).Execute(0x1734300, 0xc0001e7f88, 0xc000086058)
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:560 +0x2f4
main.main()
/var/cache/omnibus/src/registry/src/github.com/docker/distribution/cmd/registry/main.go:23 +0x2d
Failed to run garbage-collect command, starting registry service.
ok: run: registry: (pid 17392) 1s