In the event of an accident or incident requiring evacuation not seating families with children together compromises the safety of both the family and other passengers. Human nature would mean that instead of heading towards the nearest exit a parent/ guardian will be heading towards the child. Obstructing the flow of passengers.
Secondly, in aircraft with drop down 02 and children too small to reach the deployed masks, who is responsible or liable for assisting the child?
That said, the phenomenal statistical safety of US and European commercial air transport makes it highly unlikely that it would ever actually be an issue...
Similar to infants in extension seatbelts, ok ish for turbulence possibly worse for the infant in a severe deceleration accident but unlikely to ever be a factor.