Sharing components between projects but restricting access.

In this scenario there are two project spaces. The first project space (project a) contains a component which has development that is being shared with the second project (project b). So project b has a stream constructed referenecing the component created in project a. A portion of project b's intended development staff however, does not have permission to view all of the source in project a. (i.e. there are streams and work items unrelated to project b which project b team members are not cleared access). Other project b team members will and are expected to have access to both project a and b because they are expected to share and deliver changes between streams existing in both project spaces.

Currenlty it seems the only way to provide project b developers access to the source on the steam constructed from project a's component in project b is to add the team members to both project a and project b. Is there a way to restrict developers access to project a while alowing them to view/modify source in the stream constructed in project b without losing the capability of delivering changes between streams in both projects?