Not quite. To fly around Europe legally according to the FAA you need an FAA licence. To fly in EASA land (I assume the 'operator' is you and you are 'established in the EU') you need an EASA licence (issued any where, so your UK one is fine)
If you were going to only fly in France, then the FAA side would be happy with a French issued EASA licence, which would of course also cover the EASA side.
The EASA angle is one of the very limited occasions where a non-ICAO entity has compelled a group of ICAO signatory states to force flight crew licensing regulations onto foreign states' aircraft.