Simple answer. Yes.
Obviously you do your best to design it out, and as you noted spoilers are generally outboard (though there can be overlap with the tail). But some configurations - military types - say - don't have the luxury of such high aspect ratios as airliners, and are physically smaller too, so interference is possible.