GitLab graphql project pagination problem

For paging projects within a group, “first:N”, with ‘N’ less than the total projects, is often
returning the same projects in subsequent pages. A pattern has proved difficult to reveal.
For some ‘N’ values, the second response starts with the same handful of projects that
were found in the beginning of the first response. The example below is for “first:10”.
The group I’m querying has 58 projects, and if I specify “first:200”, all 58 are returned in a
single response without duplication. It seems relevant that the same exact query sequence several
days apart exhibits deterministic behavior and produces identical erroneous results.

I POST the queries below to https:///api/graphql.

In the diagnostic output of each response, I show the cursor and the ID from the edge.

Initial query
{“query”: “query{
group(fullPath:“some/thing”){
projects(first:10, after:”"){
edges{
node{id fullPath}
cursor
}
pageInfo{endCursor hasNextPage}
}
}
}"}
It returns 10 projects, ‘hasNextPage’ is True, and ‘endCursor’ is NTg3.
NjA2: gid://gitlab/Project/606
NjEw: gid://gitlab/Project/610
NjE3: gid://gitlab/Project/617
NjEz: gid://gitlab/Project/613
NjA1: gid://gitlab/Project/605
NjI0: gid://gitlab/Project/624
NjE1: gid://gitlab/Project/615
ODIw: gid://gitlab/Project/820
ODIy: gid://gitlab/Project/822
NTg3: gid://gitlab/Project/587

The second query (less than one second later):
{“query”: “query{
group(fullPath:“some/thing”){
projects(first:10, after:“NTg3”){
edges{
node{id fullPath}
cursor
}
pageInfo{endCursor hasNextPage}
}
}
}”}
It returns 10 projects, ‘hasNextPage’ is True, and ‘endCursor’ is NTIz.
NTE4: gid://gitlab/Project/518
NTIy: gid://gitlab/Project/522
NTIw: gid://gitlab/Project/520
NTE3: gid://gitlab/Project/517
NTgy: gid://gitlab/Project/582
NTE1: gid://gitlab/Project/515
NTI5: gid://gitlab/Project/529
NTI4: gid://gitlab/Project/528
NTMw: gid://gitlab/Project/530
NTIz: gid://gitlab/Project/523

The third query (less than one second later):
{“query”: “query{
group(fullPath:“some/thing”){
projects(first:10, after:“NTIz”){
edges{
node{id fullPath}
cursor
}
pageInfo{endCursor hasNextPage}
}
}
}”}

Returns only 8 projects with ‘hasNextPage’ False. It is not an exact subset of the 2nd query’s response, but contains duplicates. One can’t help wonder if the quantity ‘8’ is related to there being 58 projects in this group.
NTE4: gid://gitlab/Project/518
NTIy: gid://gitlab/Project/522
NTIw: gid://gitlab/Project/520
NTE3: gid://gitlab/Project/517
NTE1: gid://gitlab/Project/515
NTIx: gid://gitlab/Project/521
NTE5: gid://gitlab/Project/519
NTE2: gid://gitlab/Project/516

For completeness, below is the same diagnostic output specifying “first:200”
with comments about correspondence with returns using “first:10”.
NjA2: gid://gitlab/Project/606 (same project, same cursor)
NjEw: gid://gitlab/Project/610 "
NjE3: gid://gitlab/Project/617 "
NjEz: gid://gitlab/Project/613 "
NjA1: gid://gitlab/Project/605 "
NjI0: gid://gitlab/Project/624 "
NjE1: gid://gitlab/Project/615 "
ODIw: gid://gitlab/Project/820 "
ODIy: gid://gitlab/Project/822 "
NTg3: gid://gitlab/Project/587 (identical 10 returned by “first:10”)
NjIy: gid://gitlab/Project/622
NTE4: gid://gitlab/Project/518 (11th returned by “first:10”, same cursor)
NjA5: gid://gitlab/Project/609
NjYw: gid://gitlab/Project/660
ODIx: gid://gitlab/Project/821
NTIy: gid://gitlab/Project/522 (12th returned by “first:10”)
NTIw: gid://gitlab/Project/520 (13th)
NTE3: gid://gitlab/Project/517 (14th)
NTgy: gid://gitlab/Project/582 (15th)
NTE1: gid://gitlab/Project/515 (16th)
NjA4: gid://gitlab/Project/608
NTI5: gid://gitlab/Project/529 (17th)
NTg4: gid://gitlab/Project/588
NTI4: gid://gitlab/Project/528 (18th)
NTMw: gid://gitlab/Project/530 (19th)
NTI2: gid://gitlab/Project/526
NTIz: gid://gitlab/Project/523 (20th)
NTIx: gid://gitlab/Project/521 (included in 3rd query’s return)
NTE5: gid://gitlab/Project/519 (included in 3rd query’s return)
NTI1: gid://gitlab/Project/525
NTI3: gid://gitlab/Project/527
ODI0: gid://gitlab/Project/824
NTE2: gid://gitlab/Project/516 (last of 3rd query’s return)
NTI0: gid://gitlab/Project/524
NTg1: gid://gitlab/Project/585
NTgz: gid://gitlab/Project/583
NjI1: gid://gitlab/Project/625
NjIz: gid://gitlab/Project/623
NjI3: gid://gitlab/Project/627
NjIw: gid://gitlab/Project/620
NjE2: gid://gitlab/Project/616
NjIx: gid://gitlab/Project/621
NjE4: gid://gitlab/Project/618
NjE5: gid://gitlab/Project/619
NjI2: gid://gitlab/Project/626
NTg2: gid://gitlab/Project/586
ODI1: gid://gitlab/Project/825
NjU5: gid://gitlab/Project/659
ODI2: gid://gitlab/Project/826
NjEx: gid://gitlab/Project/611
Nzk5: gid://gitlab/Project/799
NjA3: gid://gitlab/Project/607
NjEy: gid://gitlab/Project/612
ODI3: gid://gitlab/Project/827
NjE0: gid://gitlab/Project/614
ODAw: gid://gitlab/Project/800
NTg0: gid://gitlab/Project/584
ODIz: gid://gitlab/Project/823

If anyone sees something I’ve done incorrectly, I am more than eager to hear your thoughts.
I’m hoping our configuration is just running an older version that lacks a more recent fix.

Hi,

please use Markdown code blocks to better format the output, e.g. with wrapping this into 3 backticks before and after.

```
NjA3: gid://gitlab/Project/607
NjEy: gid://gitlab/Project/612
```

Second to that, please add the version involved so one may reproduce this. With adding some standalone code or shell snippets to reproduce the problem, this will help too, I haven’t used graphql yet.

Cheers,
Michael

The URL I post to actually has a hostname part; what I had in a notepad didn’t paste into here the way I intended. I POST the queries shown to https://my.org/api/graphql

I’m not familiar with CURL, but it seems like what I’m doing should be easy to recreate through that tool.

@RanHug00 is this still an issue? We’ve been noticing this as well when testing out graphql. One ‘solution’ we’ve found was to make the query, make requests until we hit the end, then make requests backwards until we hit the beginning. Then you can deduplicate results, depending on how you were collecting them.

It seems like the pagination code works, if you have a start or end cursor. If you have neither, then transitioning from having no cursor to having a cursor is what seems inconsistent.