My understanding, Jack, is that the Queen, or more specifically the Crown, is the embodiment or personification of the State, the State being the Nation or, if you prefer, "The People".
Thus whilst even in a constitutional monarchy the Monarch theoretically weilds ultimate power (as opposed to absolute power) he or she does so on behalf of, and in the name of, the People.
Swearing alleigance to the Crown, then, means that one accepts both the authority of the People, and the ability of the Monarch to exercise that authority correctly and responsibly. The Crown and The People become, by this relationship, mutual providers and protectors; thus the integrity of the nation, or State, is maintained for the collective greater good.
The swearing of alliegance does not, however, extend to the Government of the Day, who are generally considered to be a bunch of untrustworthy Bad Eggs, against whom both Crown and People, through their unique partnership, must remain forever on their guard.
So even if you're not a Royalist, your loyalty is sworn to the State or nation anyway, and you undertake to remain faithful to that entity regardless of the embodiment of its official head.
The Government is still secondary in your loyalties.
It may seem like a fine distinction, but it is a very terribly important one.
At the end of the day it is all that stands between those regimes which are civilised and those which are not.