Riverboat,
I'm certain that unless you were receiving a 'direction' from an ATC provider, you could leave and argue about it afterwards. It's your aeroplane, and as it's already at the airport anyway, you already have the landowners permission to be there..Just make sure you 'book out' to satisfy the legal requirements..
Unless you owe the receiver money, then I reckon you're free to go.