My three pen'orth. Started gliding at 15, long break (exams and University) then started again at 21, then motorglider PPL as it then was - really started off practicing for land outs and caught the sort-of-powered flying bug. Then a 'proper' PPL but a) it got too expensive and b) it was just too restrictive (all the procedural stuff and so on) compared to soaring so at age 26 back to gliding and it was just SO much more fun.
It really depends on what you want to do. For me the real joy is and always was pure flight. The ground support stuff was worth it for that, and in itself was often fun with good company and banter. I enjoyed winch driving and retrieving cables, pushing and pulling gliders etc!
Find a friendly club!