According to the (2005) Flight Guide, it is possible to fly into RNAS Yeovilton, but only during their operational hours - Mon to Fri.
I've not heard of anyone trying it, but allegedly, the Services are supposed to be coming more GA-friendly by allowing access to military bases.