Wiki has this to say about the F104:
The stabilator (horizontal tail surface) was mounted atop the fin to reduce inertia coupling. Because the vertical fin was only slightly shorter than the length of each wing and nearly as aerodynamically effective, it could act as a wing on rudder application, rolling the aircraft in the opposite direction of rudder input. To offset this effect, the wings were canted downward, giving 10° anhedral.
A ventral fin was also added fairly early on in the programme.