OpenOffice Calc is designed to open .xls just as excell would, so guess what happens?
You guessed it "Enter Password"
Things would be worse if Calc couldnt detect the password as all it would open is the Encrypted data in the spreadsheet.
The only way to recover the data is to bruteforce the password, or to decrypt the encryption.
I am sure there is some person on here who understand cryptology or "hacking" better that I, so see what they think.