It's an interesting topic. Personally I think it is one that most of us are taking too lightly.
Years ago the German ALPA (VC) had an interesting article in their magazine about cosmic radiation based on scientific research. They had their own working goup at the time.
A few things I remember from that (if memory serves): The limits for flight crew exposure are somewhat randomly chosen and at the time were higher than those allowed for folks working in nuclear power plants for example. The exposure accumulates over the course of our lifes. So, if you start flying at high levels at a young age and continue throughout a long career, that is more harmful than someone who only started flying high late in their career. Basically the body cannot "get rid" of radiation. Closer to the poles exposure is a lot higher since it is not shielded by the earth's magnetic field. Radiation is also exponential rather than linear with altitude. So flying a bit lower might already makes a big difference. The amount of radiation we are exposed to also varys greatly with solar activity (on the surface of the sun).
As a rule, down at turboprop levels there isn't much to worry about. In the high 300s and above I do consider radiation as one of the factors in choosing my level. Especially on long flights closer to the poles.
I once spoke to a former flight attendant. Her husband was a physics professor specialising in gamma ray type radiation. After they had met he strongly encouraged her to give up her job based on the research he had done.
Sure, live is dangerous and we shouldn't panik, but in my view it's a real threat that deserves serious consideration.
A google search on "cosmic radiation" is well worth it for those who would like to learn more.