To my way of thinking ~ and I must admit to not being a sophisticate when it comes to this specific technology ~ in the case of RFID chips in passports, it could be the hotel clerk himself who is capturing the data, or the airport security personnel, who could also have an alternate agenda. You don't need a bad actor in proximity, just your passport.
Regarding encryption, that's mathematics. My university degree is in mathematics; I'm in a heavily quantitative graduate program now. One of my friends in undergraduate school, along with a professor, published a peer-reviewed paper on the building of algorithms designed to unwind encryptions, which included an authorized penetration of the encryption set in place by a famous investment bank. While this is not my field, I'm confident unwinding encryption in RFID chips would not be impossible.
Essentially, writing the proof for a mathematical problem is the same as reverse-engineering it. That's what we're speaking of here.
Regarding people's fears of moving from paper-based technology to electronic, it is the matter of moving from transmission of individual physical records to mass transmission and storage of millions. This opens up the opportunity for theft of gigantic volumes of material without the problem of actually having to lift and transport it. I know this is obvious, but think it worthy of restatement.
It's highly inefficient to sift through individual cans of garbage. It's highly efficient to have to look in only one place.
About a year ago I read a news report that one of my credit card companies had been the victim of the electronic theft of over 500,000 account records. Calls to the company led to nothing but a run-around. They would neither confirm nor deny that my account records had been compromised. I cancelled the account, demanding that I receive a letter indemnifying me against account abuse due to record theft. It took an exchange of letters between my mother (a lawyer) and their corporate counsel to receive such a letter, five months after the fact.
It will be some time, if ever, before reasonable fears of theft and misuse of personal data go away.