The idea that lighter aircraft are somehow safer does not seem intuitive
At the same speed, a lighter aircraft holds less kinetic energy, so it will do less damage on crashing. The whole concept of ultralight regulation is to keep kinetic energy down so as to minimise damage at impact - whence also the stall speed limitation.
But this is about safety of third parties, not about the safety of occupants. Actually I suspect the basic reasoning to be that, if accidents are likely to harm the pilot most of all, said pilot will think twice about behaving carefully, both in piloting and in maintenance.