Unless I am very much mistaken, the original question was only about attitude and heading reference gyros, and nothing to do with position. So, notwithstanding the discussion around position that ensued, here's my 2c worth:
Many (most?) EFIS solid state gyros do NOT have GPS connections to aid them. They use conventional magnetic slaving and acceleration (gravity) based erecting.
They also have errors, some of which increase during a flight. These errors are often greater than those of an inertial navigation system, although the absolute errors are still quite small. The reason that even tiny errors in an inertial system are important is that they are integrated over a long period of time. For example, if your calculation of ground track is wrong by one quarter of a degree, then after 5 hours of oceanic flight, your position solution will be "out" by quite a bit. On the other hand, if your heading display on the EFIS is out by one quarter of a degree, you will never notice or care.
As a small digression, in the very low-end EFIS systems, such as those used in experimental aircraft, they use much simpler systems which primarily rely on accelerometers. These rely on integrating accelerations to determine attitude, and they also therefore accumulate errors quite quickly. They generally DO require a GPS input as a "hint" to continually constrain the error bound.