So if one "accidentally" activates the approach, is there anyway to "un-activate" it?
Yes... Re-enter a cruise level.
The box does not memorise weight and freeze the speeds. Also the airbusses in the largest airbus operator of europe change speeds in the box as the weight reduces, for example due to extended downwind vector in.
Why is this even a discussion? You activate approach phase when you start final deceleration for the approach. In other words when you or the controller thinks that it is time for deceleration to minimum clean (in controller jargon 220kts).
The biggest problem with activating approach too early is that your fmgs descent prediction no longer works properly. In open des mode the blue arrow is no longer based on your actual selected speed.