Hold the power button down for ten seconds until the slide to power off message appears. Power down the phone completely. Wait for a bit (30 seconds) then power on again to re boot the phone.
Mail is one of the functions that is part of the operating system so it cannot be deleted, only shifted and nested inside another folder at best.
Obviously, never under any circumstances download apps from anywhere other than the official Apple app store.