"Their English is a bloody sight better than your Mandarin"
That's exactly the right approach. After decades of dealing with Asian people, I'd never accept their apology for their poor English, Rather, I'd say something like 'Don't worry, there's no need to apologise. You should hear my ......."! That would always result in a good laugh with a relaxed and good outcome. It's all about patience, tolerance and empathy.