But why, during a bank, does the aircraft not simply move laterally, staying parallel to its original course?
Basic Newtonian laws. Action equals reaction, force leads to acceleration and so forth.
Consider the opposite. Suppose what you're saying is true and the aircraft simply displaces itself laterally when you bank the wings. In order to do this, you first need a force sideways in the direction of the displacement, accelerating the aircraft towards its new track. But after that, you need an opposite force somehow which decelerates the sideways motion so that the aircraft eventually comes to "rest" on the laterally displaced track. Where is that last force coming from? It doesn't exist, so the sideways acceleration never stops.
So it's actually very simple. The sideways force starts an acceleration sideways. Then the fin (as described above) ensures that the fuselage now aligns itself with the new flight path. Because of that yaw, the sideways force also changes direction forcing yet another sideways acceleration. And so on and so forth.
Of course these are not two separate steps (unless you're really ham-fisted) but a continuous motion. But they ensure that the sideways force is always perpendicular to the aircrafts flight path. And if you have an object that continuously experiences a sideways force perpendicular to its trajectory, you eventually end up with a circular motion. A rock on a piece of string is no different.