There is a HSE document on the acceptability of risks of various kinds in various scenarios called "Reducing Risks and Protecting People" [generally refered to as "R2P2" in the engineering profession] which is widely respected as a practicable approach to this question. It's actually worth a read, because it talks a lot more sense than most people's prejudices would expect!
PDR