I have been in that "is it really worth the effort" mood several times - the thing that keeps me coming back is remembering how much fun it really is.
IMHO the "low hours Sunday pilot" is only a danger if they do not realise their limitations. I cannot afford to fly as often as I would like, as a result my log book is full of dual checks and circuit checks with relatively few cross countries in between - but I enjoy my flying so I keep coming back for more.
I would recommend converting to other types too - I tried some tail dragging and loved it. I will not give up "spam can" flying though - sometimes having something solid and predictable to fly allows you time to enjoy other aspects of the experience.
I would say - before giving up for good make at least one more flight (perhaps two if that one does not go well
) and do the sort of flying you like - if that is boring to others, ignore them
.