An aircraft with sweep and anhedral has a high degree of lateral stability when flying inverted. Fact.
I did a BofI on a Gnat, abandoned during an aeros display, which recovered to inverted flight and droned around quite happily until it carried out a textbook forced landing upside down.
But we are talking sideslip stability here, not roll rate, so the Gnat did and the Harrier should (no practical experience here) roll just as rapidly from inverted to upright as it did when rolling inverted in response to aileron input.
"Pendulum effect" is just an analogy to help the visualisation of the effect of high wings, high tails, T tails etc. The mean sideways force is applied above the aircraft CG, so tends to roll the aircraft upright.
Dick W