I have presciption sunnies and whilst I've never tried contacts, I don't find it any more time consuming changing over specs than you would if you retrieved your normal sunnies.
I usually hang the other pair off the storm window latch and quickly change them over when required.
I don't have a requirement to carry a "spare pair". My script is about -1.25 / -1.50-ish from memory