I agree with yawning dog - a good summary.
My take is that most accidents are pilot error and by being a careful pilot I can avoid that. Accidents from mechanical failure or other circumstances beyond the pilot's control are rare. Even then with careful attention to the weather forecasts and with the route when flight planning, a careful pilot can minimise the risk of injury. I feel more at risk driving the car or crossing the road.
Someone mentioned microlights as an alternative. I gain comfort from the high maintenance standards that are mandated in GA - it gives me greater confidence with the aircraft than what I would have in a microlight.
As far as the financial risk goes, if you cannot carry it personally, talk to an insurance broker. But dont get paranoic about the risk. It is not high if you are careful. Flying is NOT inherently dangerous. It can be unforgiving to the careless.
Bentleg