I would say that yes, the low atmospheric pressure has its own health risks, in addition to how much O2 is actually present. You get fluid in the lungs percolating out of the cells, swelling of the brain, burst blood vessels, etc. that would not occur (or as much) simply due to oxygen deprivation alone.
Pretty much equivalent to the fact that - no matter how much air there is - if you were suddenly exposed to the pressure of 1,000 feet down in the ocean, you would be crushed like a tin can in a vise.