I assume for flying N-reg you hold a standalone FAA PPL and not a 61.75 validation based on the Part.FCL license (in which case your flying would have been illegal).
No, it actually is not illegal.
The certificate issued under 14 CFR 61.75 does not become invalidated merely when an SEP included in the foreign licence expires. See FAA Office of the Chief Counsel legal interpretations to
Krausz 2012 and
Großmann 2014.