Well, before the outrage bus picks up too much speed, and having unfortunately not seen the programme, nor indeed the dress code concerned, I would quietly put forward the thought that Mr Cowell was wearing what he and, many other people, undoubtedly regard as his dress "uniform".
Incidentally, and noting TTN's mention of Jock Slater, I seem to recall him wearing black suede shoes with his initials on the toecaps, together with a cashmere woolly pully, whilst in his "uniform"!
Jack