A few noteworthy mentions here.
You do NOT need a clearance to avoid another aircraft....
Speed in class E is restricted below 10,000 feet to a maximum 250 knots, and the reason is traffic avoidance. I do agree that a few VFR flights venture higher than that, but most stay below.
Flying IFR in class G you'll meet IFR flights when IMC and both IFR and VFR flights while VMC, whereas you'll only meet VFR flights while VMC in class E. So no, in reality class E is the safer airspace....
Yes as evasive action you do whatever it takes, but the idea is you sort it out before it gets to that point. That will require a clearance and a radio conference.
Class E works fine when pilots are aware of the airspace classification, and the amount of VFR traffic is relatively low.
And there's radar coverage.