If you come back later and find it blocked again then maybe you need to go there on the ground and talk to the owner. Or you can phone the owner from your alternative and ask her/him to shoo away the visitors. Don't try phoning from the air unless you have someone else to take care of the flying.
And try going lower if you want but it will do no good. Even if you pass at 6ft over the grass they will only wave more and maybe film it with their mobiles for YouTube, the CAA and the police. They will not get the hint.
Best plan IMHO is to ask the owner to explain things to the locals and maybe put a small sign requesting they stand to one side and restrain their pets when they see an A/C approaching.