Drop the odd bottle with poison or emetics in it. Once word gets round who will trust anything dropped?
(In the Dhofar campaign the SAS would carefully dig up any rebel ammunition caches they found and “Salt’ the rounds with ones with charges designed to blow up in the users face. The end result being the rebels wouldn’t trust any buried caches and relied just on the ones they could physically carry, In doing so the SAS effectively took out the entire rebel logistics train)