Sure Ryanair virtually deserted BHX the last time the airport upped it's fees to them....but they came back, I believe on the airport's terms not theirs.
I reckon that Ryanair needs BHX nearly as much as BHX needs Ryanair. if FR walked out I don't think it would take long for Easyjet to fill the void - though probably with rather less imaginative routes - though they would probably fly to airports closer to the cities they advertise than FR does!!