From the documentation, it’s not clear if a personal access token with the api scope would allow the user to access the source code (git repository) or not.
The description in the docs hints at this:
Grants complete read/write access to the API, including all groups and projects, the container registry, and the package registry.
Looking at the api docs, I would say yes it does:
since the access token has full access to the API, it means you can download repository archive. From the second link, looks like you can also create, read, update and delete files within the repository itself.
You could grant read_api instead though for read-only access to the repository to stop people updating/deleting. Unfortunately it doesn’t stop anyone downloading the repository files though.