If you've got planning then you are pretty safe provided you stay within the terms of the planning permission.
If I ever got my own strip somewhere, I would go for full planning from day 1. It might take a couple of years (planning officer approving it, committee chucking it out, inspector approving it) and would cost 4 or even 5 digits in noise surveys etc, but then one is safe and doesn't have to worry about whether somebody is counting the active days.
Nobody should withdraw a planning application simply because somebody doesn't like it. I know many people do exactly that but that's because they don't know that every village has at least a dozen people who object to everything.
The local planning system (the planning committee system in particular, with the dozen councillors who usually vote without ever even seeing the site) is set up to make the applicant (whose beloved application has just been viciously torn to pieces with barely a few minutes' discussion) really despondent and to make him withdraw it.
Applicants who are smarter will go to the inspector (the appeal) at least, as a matter of principle, every time, because this bypasses the local issues.
Makes me sad to see people withdraw planning applications due to local objections... it's the 'bully' winning without moving a finger.
There may be exceptions e.g. a pub with mostly local clientele where not p***ing off the locals is a sensible move. But an individual who just lives in a village should never withdraw an application at this stage and should always proceed to appeal.