I’ve been with carriers who do both. I much prefer no uniform. I have never once thought about what the other guy/girl wears.
Oh, he wears a uniform so I must be in the real aircraft? Nope, I know where I am.
If I have to pick up or deliver an empty aircraft, I wear normal clothes as well. Unless the place I fly to or from is some weird place where a uniform will help you get to or from the aircraft.