Even when flying VFR you are meant to set an MSA. Which, when I was learning, was to take the height of the highest obstacle in the grid on the CAA chart and add 1000'.
When flying IFR, it's a legal requirement, unless when on departure or on approach.
When flying VFR, it's a bloody good idea but it's not always practical or possible. For example, in the Netherlands, there is a TV mast near Lopik that extends to 1234', with class A airspace above at 2500', and class A airspace at 1500' just a few nm away. I regularly pass near there at 1200' or so (my destination is under the 1500' shelf), and well within the 5 or 10 nm that people use for MSA calculations. But the thing is: I'm aware that I'm below the grid or leg MSA, and I know exactly what I'm looking for. And as long as you are not closer than 500' to the mast then you're completely legal. (But may I suggest that 500' laterally is too close anyway, particularly since this particular tower is supported by near-invisible guy wires.)
The MSA calculation when flying VFR is important for another reason though. If you happen to enter cloud and cannot regain VMC immediately with, for instance, a 180, then your next priority should be to climb above the MSA, regardless of whatever is above. Squawk 7700 and contact ATC if necessary, but don't fly around blind below the MSA.