Saveloys, looks like you have a point old chap/chapess. Admittedly, gliding sites are easily avoided if a chart is studied carefully but the performance of modern gliders does tend to lead them well away from these marked sites.
I do find they are difficult to spot myself and have had several close calls but as someone mentioned above, chances are high that the "glider-man" saw you a long time before you saw him.
I've never been in a glider(550 hrs PPL A) but if anyone wants to kindly donate a trip I'm happy to fly to your base and see what it's all about!!!!!!!!!!!!!!!!!!!!!!!!!!!!