If I remember correctly, its because its part of the EEA - an agreement between the EU and the countries that form the EEA (Iceland, Norway, Switzerland, Lichtenstein, Andorra, San Marino, Vatican City, Monaco), regarding movement and trade restrictions.
That's why at T4 LHR there a separate and oft unused queue for them and Non-EU British Residents.
Jordan