Brian,
Not so I'm afraid, I think you are misreading the intent behind § 23.423. VA relates to the entire airframe.
I agree with everything you said, except that sentence. Why do you think that Va relates to the entire airframe?
You go on to explain Vo and how it's related to the entire airframe (by ensuring that the limit maneuvering load factor isn't exceeded), but Va does not do that.
Reading the FAA AC 23-19A will confirm that Va does not "permit the pilot unrestricted flight-control movement without exceeding airplane structural limits".