The whole idea of the various leave campaigns, rightly or wrongly, is that should they be successful in getting the UK to exit the EU then they will not allow free movement of EU citizens.
The whole point of this referendum is that the anti-EU brigade want to stop the Poles, Slovaks, Latvians, Romanians et al from having free movement to travel, work or reside in the UK.