If the aircraft has a different Vfe for different flap settings, then obey those individually. If it only has one, common sense dictates that flaps must be UP above this speed, and can be lowered below it. While small flap settings probably put less stress on it, just be smart and make sure that no flaps are down above Vfe