Can I restrict users so they can only clone and update a specific branch

I want to restrict a contractor’s access to a specific branch. It is possible to allow them to only clone and update a specific branch? Or even a subset of a specific branch?

The use case is, I have contractors that will be working on a specific part of our UI. So they don’t need the entire repository, just the subset of the UI. Also, we only want them to have update privileges on their specific branch. We will merge their code to master when we have verified it is ready.

Is there a way to set this up?