People generally choose aisle and window seats as a preference, it's a bit rich to not bother using the facilities in advance or even phone the airline in advance and expect them to automatically accommodate you when you arrive at the desk.
A lesson learnt for next time I feel.
However harsh that is, on a flight that short I'd have offered up my aisle or window seat to help a family sit together no problem if I'd been asked.
Did you ask the crew when you boarded if they could find a couple of seats for you and your youngest?