I've done it many times. The only downside (as someone I know found out) is if a "well meaning" local phones Plod. My friend's son made a precautionary landing since he was unhappy with the way his engine was behaving. He did an initial pass to confirm the field was suitable, then landed, and taxied over to the adjacent farm. He spoke to the farmer who was quite happy and even offered to help him move it out of the field. Next thing the local emergency services (all 3!) turn up, sirens blaring, and he is told in no uncertain terms that it is now a "Crime Scene". He was even threatened with arrest!!.
It took a phone call to another pilot who happens to be a serving police officer to de-fuse the situation. I told them to make an official complaint to the police, but I believe they didn't bother in the end. Hardly inspires confidence does it?