I'm afraid that virtually all of my FJ experience is on the P&HQ or operational side, so I shall have to bow to JF's far greater experience concerning the Fwd CG limit of aircraft like the Harrier.
However, on the little aeroplane side, I've been directly responsible for finding the CG limits on quite a few, and without fail the limiting factor for the aircraft that I've dealt with has been the ability to adequately control the aircraft in the roundout and flare. (Actually not strictly true, on one type it was defined by the point when we ran out of ballast).
The aft CG limit incidentally is *usually* defined by an acceptably low limit of LSS, but I have come across one type (with a rather small fin) where we defined it by directional stability running out.
Bear in mind however that many aircraft are designed right to the limit of permitted stalling speed, which is usually pitch authority limited in a light aircraft. So, a fwd CG limit defined by Vso is certainly a possibility, and I suspect that this is probably the case with some of the higher performance FAR-23 aircraft.
G