How safe compared to what?
Very, when compared against wingflying, skydiving, technical diving, base jumping, unprotected sex in third-world countries, soldiering as an ATO...
..but not when compared to going shopping, lying in bed, or riding on a bus, boat or commercial aircraft.
In the end, it's subjective. A look at the AAIB reports over the past decade will show you how easy it is to sometimes get it horribly wrong. I haven't counted them but I guess there must be three or four hundred fatalities or so over the past twenty years on there for GA.
However else you describe it, going for a "normal" flight in a light aircraft is not the same as going for a "normal" ride in a car. For myself, I decided that I would happily take adults on a "joyride" with me but I have always managed to come up with a reason not to take kids under 16. Arbitrary I know but that's my decision and I'm the Cap'n.