"until then it'll still be seen as a "branch" for those who couldn't quite cut it in their own world... "
Try telling that to 007!
Unfortunately, creating a branch that is only filled on an ad hoc basis will never be a recipe for success. You will have some very good individuals who are specialists in certain areas, but no depth or structure to it.
If you want a dedicated N2 branch, you can't do it on an as and when basis, otherwise you will never be seen as the credible operators that you generally are.