It's a hell of a lot of money to risk otherwise, considering that basically all of the normal means of hour building will be unavailable to you.
Its definately a valid point but looking at EASA CAP 804 Hours building can be done on a Class 2 Medical and push comes to shove you can Instruct on a Class 2 Medical (as per EASA CAP804 Pages 462-465) and even to CPL Level (just not for the "integrated" course) - which would not really be applicable as I have no intention of going to ATO's that would be "Integrated" only.
Of course this would all be SEP time and not sure how useful this would be in comparison to probably valuable multi crew building time.
It will of course always be a lot of risk as I don't have 50k sitting in my back pocket! But even for those with an unrestricted medical it will always be a risk, i guess it depends on how "risk averse" one is.