I have a GitLab Project/Repository that contains ~5k JSON configuration/object files and now I’m trying to create a frontend with React to manage these files using the GitLab API.
For single files this seems to be an easy task, but as I have thousands of (rather small) JSON files in the repository, I’m looking for a way to fetch them most efficiently (I guess loading them one by one would not be very efficient).
My idea was to use the Repositories API archive feature and download the complete repository compressed (which is only ~100KB), uncompress it via zipjs or something into memory and cache it in localStorage or via File System Access API.
But the problem is, GitLabs Repositories API is not allowing me to download the archive as the fetch-Request has Sec-Fetch-Mode set to
cors and this triggers the Hotlink Protection and I’m getting a 406 - Not Acceptable. I did not find an option to remove or manipulate that header in the browser (which is by design I guess ).
I also thought of just getting the files when I need them, but I would also like to have a search feature that searches files with a specific content and show an overview in some Data Grid, which needs only a subset but still data from all of the files. And building and storing this index also in the repository feels redundant and would increase the project size significantly - and it increases effort as the index would have to be updated with every change as well.
I would like to host the frontend with GitLab Pages, so that I only need GitLab for repository and frontend is needed and not any additional backend. So setting up some middleware between GitLab and Frontend is not an option.