Using nuget package from package registry in CI pipeline

I am trying to use two nuget packages from a package registry in my CI pipeline. For this, I create a nuget.config file at the root of my folder with the following content:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="my_registry value="https://<gitlab_instance>.com/api/v4/projects/<some_project_id>/packages/nuget/index.json" />
  </packageSources>
  <packageSourceCredentials>
     <gitlab>
         <add key="Username" value="deploy_token_username" />
         <add key="ClearTextPassword" value="deploy_token" />
     </gitlab>
 </packageSourceCredentials>
</configuration>

But this doesn’t seem to be enough because every time my pipeline gets to restoring the two packages, it hangs because it can’t access the feed. I didn’t really find anything in the docs regarding this. Any ideas?