Having previously worked at such an airport it's not as hypothetical as you infer
VMC - Provided neither of use any words that infer that an instrument approach aid is being used then no problems at all - because you're not are you. Once you say something (remember - it's all taped) then you drop yourself (and possibly said ATCO if they 'go along' with it) in the pooh.
IMC - the same applies. After all, he's not in the aircraft - you are. Therefore said ATCO, although the weather conditions they report might intimate IMC, doesn't actually know you're IMC where you are.
It's a matter of conscience really.