It's always courteous to ask crew first if you wish to move around the cabin, particularly when on the ground..... for various reasons.
The allocated seating in case of an accident theory is a myth. Remember, if an accident is particularly bad, then your row 21 may well become row 32 if you catch my drift.
Remember also that some airlines have/did have free seating and no recording of who sits where happens at those carriers.