The reason that Hitler didn't use these agents against us is (IMO) that he fervently believed that we did have our own and would retaliate (since there had been a CW 'tit for tat' during WW1). This was because the Germans had become aware of the USA discovering the insecticidal properties of dichlorodiphenyltrichloroethane (DDT) and synthesising it in large quantities. The initial chemical pathways to DDT are the same as those to nerve agents.
It appears that both sides monitored each others scientific journals and the basic research on DDT was taken as proof that we probably had nerve gases