How is that relevant if you don't have the GUI running or don't even install it?
It's not, but the point is that GUIs are sometimes installed and running on servers. That's certainly true for Windows servers, and the inexperienced administrator might also have Linux or UNIX servers installed with GUIs, particularly if they were set up with some sort of default install that always puts in a GUI. As mad_jock has indicated, if you start one day at a new job and you see a server room filled with screens and screen savers (either is a bad sign), it tells you something about the people already running the place. It's a bit like configuring your server farm to run Seti @ Home.
I don't think UNIX or Linux systems should ever have default installations that put in any type of GUI. If you are running these operating systems and you don't know how to set up a GUI yourself, you don't know enough to be using these operating systems. I know that this is often done to encourage the use of these operating systems on the desktop, but they are not suitable for the desktop. The obvious exception is OSX, which has UNIX-like underpinnings but has nevertheless been heavily modified to serve more or less exclusively as a desktop (if you remove the GUI from a Mac, well, why bother paying for a Mac?).