Personally, I don't ask anyone to come flying but if they express an interest and are keen then I'll take them. Usually wait for them to ask a couple of times though!
I reckon most people, rightly or wrongly, would put flying in a light aircraft in the same danger category as skiing, motorcycling, parachuting or sailing. Some people are suited, others not.