OEVU - I have had a fairly varied career but I'm still fairly young for the average pruner (under 35!). I would say that I don't fit the 'standard UAS' image of the play hard culture. To be honest, a pretty large number of my colleagues don't, we just lead normal (boring?!) lives. Certainly don't take your UAS experience as an indicator of what life inside is like. I've spent a lot of time on/around UAS's (3 now) and all of them, whilst being full of good people, have definitely revolved around a good night in the bar.
There's a time and a place for that, and I don't have issue with those that do, but don't feel you have to enjoy that to 'fit in'.
Secondly, regarding whether it's what you want to do with your life, you're right to ask difficult questions. However, don't think that by doing it you're committing to being a monk-pilot for the rest of your life. I'm looking at a second career right now, not in aviation at all, and my previous stands me in excellent stead in many ways across the board.
Thirdly - regarding pleasant places to live 50 miles from London. Consider Oxford, Reading and Winchester. About 5-6000 RAF personnel work within easy commute of the above, including 1/2 the multi fleet and all of the rotary fleet (although you do have to become the unloved step child of the RAF if you go rotary).
Fourthly - regarding the average working week Vs flying etc. Everything comes in fits and starts. Every pilot will love to whine about the times they only got 15 hours in 3 months, but they'll totally fail to mention the time they got 100! Those times are the ones where the memories are made, on a shooting op or, like the guys recently deployed to the Carribbean trying to make a difference.
All the above is pretty pro-RAF. Bear in mind I'm actively looking to leave, and that should tell you I don't think it's all rosey. I've spent a not inconsiderable amount of time working my ass off, and I've had several years spent at least half away from home, which works at some stages of life but not at others. For me, I love flying (and I've spent this week flying my ass off, so I'm happy!) but it's time to go back to studying and try my hand at a different challenge I think.
All I'd say to you is, at the age of 35-40 you can change your mind and go get a PhD or do whatever floats your boat that way. However, you can't get your hands on a Typhoon, Chinook or A400 at the second bite of the cherry, so if you choose something else, it's permanent.
Wish you all the best either way, continue to be thoughtful, but don't beat yourself up whichever way you choose, just own your decision.