I think it was just an early morning bit of banter and neither yourself or the gentleman in question thought that the topic would be discussed any further.
You have those type of conversations with people and it's not until afterwards that you think to yourself 'what the hell was that guy going on about'!