Downwash and ground effect are two different things.
You can have downwash hitting the ground, but not affecting the performance (this is the more than about 1.5 rotor diameters part where ground effect is considered to be non-existant).
You can be less than 1.5 rotor above the terrain, but if it's something like trees, you may also have no ground effect.
Ground effect will depend on the disk loading, so for the same helicopter lightly loaded ground effect will disappear at a lower height above the ground than when you are heavily loaded (unless you have variable diameter rotors, something I haven't noticed recently)