Both these family members have said that LSA and microlights get blown around way too much, and are particularly prone to LOC on finals, and RLOC. But hey, why would I listen to them, I love flying, right ?
Well you certainly shouldn't listen to them on that topic since they clearly don't know what they are talking about! Utter tosh... Don't think that because they have many hours sitting in a commercial aircraft being flown by the autopilot that they know anything about flying little aeroplanes. Are you sure they haven't been unintentionally misleading you by talking up their sky-god status?
If you want to fly and have the finances to learn, then do it. If you don't want to do it, then don't - its not compulsory and people will not think less of you because of that decision.
Agonising about dubious safety statistics won't convince you one way or the other.
If I sound a bit irritated, its because I am.