They're not just taxing you, they are taxing the company for employing you in the UK. You are just getting caught in the cross fire. The UK tax people are coming after CX for national insurance (a UK stealth tax of about 20% gross income) which it gets from both the company and the employee. It also means it gets its hands on your tax first and then you have to get it back off them. To do this you have to prove you have the right to work in UK, you are genuinely non-resident, and of course you will always be liable for some tax for work duties carried out in UK. If it doesn't believe you or thinks you are fibbing then you may get 'investigated'.
Bottom line - be prepared to fork out extra cash for professional advice on dealing with (for you) 3 hungry tax agencies!