Chewy - it's your choice and yours only. If you go to another airline then chances are your colleagues will be older, married etc too. Have you tried starting conversations on subjects that you are interested in?
BA isn't perfect by a long way but don't give it up without very serious thought. Money isn't everything but terms and conditions are and you'll not find as good as BA easily.
Perhaps this type of flying isn't for you but I have to ask just what did you expect - a crew of 23 year olds with the same interests as you? Don't just sit there and stay out of conversations - join in and steer them towards something you are interested in.
I may sound very unsympathetic but you have to make an effort.
Good luck.