Filtering/Permissions at directory level

Is there a way to filter access to a private repo by directories;
i.e. dev A can access/pull directory #1, #2 and #3
whilst dev B can access/pull directory #1 and #4 only
etc.

Unfortunately, it seems that git and by extension Gitlab was not designed to have this level of security in mind.

I am hoping that I’m wrong!

/wai-sun