I had this same discussion with a PPL student of mine. I told him that IMO the only truly safe way to carry an infant in a light aircraft is to use a proper car seat secured with the aircraft seatbelt. This of course uses a seat position and so you can’t carry an infant as an “extra” passenger.
Personally I think just because the practice of not counting infants carried in a persons lap as “passengers” is legal, does not make it safe.
I can think of 2 light aircraft accidents where the only survivor was an infant in a secured car seat located in the back seat of a 4 place airplane