The technical difference is that, prior to 1997, the Brigade of Gurkhas was based in Hong Kong while, after that – with the return of the colony to the Chinese – the Brigade has been based in the UK so residency can apply.
The fact that the EU increasingly governs our immigration law should not come as any surprise. Since Maastricht – under the Justice and Home Affairs chapter – immigration policy has become a competence of the EU and, progressively, the EU is working towards a Common Immigration Policy, defining the rules under which third country nationals may enter.
The details of the elements for a common EU immigration policy were set out at the October 1999 European Council in Tampere (Finland) and confirmed in 2004. We – or, in this instance, the Gurkhas – are now paying the price.
Furthermore, as long as we are members of the European Union, this is how it must be. Third country nationals who become British citizens also acquire rights as "EU citizens", set out in Directive 2004/38/EC. These include the rights to bring in their spouses, parents, parents-in-law and children, all of whom then acquire citizenship rights.
So it is that the EU must set the rules, which define not only entry to the UK but to the whole of Europe, and the UK has no choice but to obey.