Luftwaffle
I agree with all your points except this one:
"I could see the traffic on final and judged the gap too narrow for my safe departure"
That's ATC's job; if we think the gap is big enough, then its big enough. But if you have any of the other reasons for refusing the "immediate" just say "no". That'll be just fine.