The above explains why have a height below which you don't do anything other than fly.
Your question contains part of the answer as to why the specific value of 400ft exists. I believe it does indeed relate to the definitions of the segments of the takeoff in the early regs (CAR4b, BCARs, whatever) where 400ft was an assumed height below which the configuration was essentially fixed. I think there was just a tendency to use the same height in abnormal procedures as in the performance definitions, for simplicity, and because there is no real objective reason for 400ft compared to, say, 300ft.
For example, the modern takeoff flight path rules include:
CS 25.111(c)(4) The aeroplane configuration may not
be changed, except for gear retraction and
automatic propeller feathering, and no change in
power or thrust that requires action by the pilot
may be made, until the aeroplane is 122 m (400 ft)
above the take-off surface;
It's just a question of digging back to find when that was first introduced. CAR4b.116 does NOT have a segment break at 400ft. So it's not that old then.