The external static ports are placed where they will be (relatively) immune to airspeed and turning motions, so that they give a true indication of the air pressure. This takes careful analysis and often testing by the airframe designer, but can determine altitude to within a few feet.
An alternate source is generally inside the fuselage structure somewhere, and is not corrected for any of those effects - it will show errors due to the speed of the aircraft, the pitch attitude, any yaw, whether the cabin vents are open or closed, and so on. The alternate air source may have errors of over 100 feet relative to the true air pressure. The alternate source is therefore an emergency facility, not for normal flight.