You can think of it this way too:
Lots of airplanes have used symmetrical airfoils - My father instructed in the Vultee BT-13 during WWII and it flew rather well upside down. AOA is all that mattered.
A cambered airfoil optimizes lift in the "normal" direction - It can create some lift at zero AOA. But at some negative AOA value, the lift goes to zero, and a further negative AOA value means negative lift. That's what you need when inverted!