It's not a smart move for visitors, but only mildly inconvenient as it requires a little forethought. I have had an Oyster card for a couple of years for my infrequent trips across London by bus between rail termini. Yesterday we had to travel from Waterloo to Kings Cross early morning, which I would normally have done by bus but as we had one Oystercard between three of us, I knew we'd have to take a taxi. Easier, faster, and more pleasant, and TfL lost the revenue, although I don't suppose it will greatly affect their financial results!