It isn't bypass air; it's bleed air from an compressor stage which supplies an appropriate pressure.
I agree with the risk you mention but there's no such possibility with compressor bleed air. On the ground you can get some re-ingestion of your own and others exhaust gas but airport workers breath that all the time.
The theoretical source of in-flight contamination is a leaking oil seal on the forward fan/compressor bearing.