Using the case of a type certified aircraft in North America, operating under the flight authority of a standard C of A, the following requirements must be met: (my words)
A pilot may not attempt to fly an aircraft which does not conform to it's type design. It is either the original C of A, or the revalidation of that C of A by qualified inspection which confirms the conformance to it's type design at the required intervals.
When the plane leaves the factory, only approved and certified (as conforming to their type design, and thus being worthy), "pieces of kit" will be installed. There after, only similarly certified "pieces of kit" which are approved on that aircraft, may be installed, or you are invalidating the measure of "conform to type design", and thus the C of A.
The fact that something is certified as airworthy, TSO'd etc. does not automatically qualify it for installation in any aircraft. A Gramin 430 is "certified" and obviously commonly approved for installation in lots of aircraft types - but not helicopters after they leave the factory, other than by special approval. A tire will be TSO'd, and thus certified, and it might be the right size to fit your aircraft, but that does not make it's installation legal (conforming). It might be the wrong ply rating. The TSO'd tire, battery etc. still must be approved as a part of the type design.
Generally any change to the aircraft which has "other than a negligible effect" (those are government words, not mine) requires an approval.
If you're attaching your hand held GPS to the gawd awful transformer robot mount that grips the control column tube, and you do it right, and check everything moves as it should/shouldn't, you're probably okay, though do remind yourself that the instructions will say in there somewhere, that you must assure that you have not affected the safety of flight. If you don't think of something, and have an accident, you the installer pilot will suddenly be repsonsible for flying an unairworthy aircraft. In turbulance, could the GPS flip back, and jam full nose down control? Might the "temporary" GPS installation block the normal view of a gear unsafe light? I have seen many pilot/owner installed mods which definately would not be considered "minor", and would certainly require an approval or certification to be "airworthy".
The rule of thumb used by Transport Canada locally is that if a tool is required to perform the installation, or it wires into other than the cigarette lighter socket, it probably requires an approval.
There are lots of very worthwhile "things" which might be installed in an aircraft, which no one has yet approved. It does not make them not worthwhile, they are just not proven as conforming to the required type design yet.