Yes, FbW. You have the same time to LISTEN but much less time to LOOK. And gliders will not be using radio in the circuit.
In addition, I don't think it's wise to rely on radio for your information about powered traffic either. I had a memorable experience joining a certain airfield a few weeks ago when the FISO informed me 'there's lots of stuff coming from all directions' and then got my number in the circuit wrong...
I don't blame the guy because I'm the Captain of my a/c, not him. Just to say the downwind join is rather safer in my view.