Yes, but separate or not, GUIs consume a great deal of resources and introduce many complications to the system. While this may be justifiable on a desktop, it's a tremendous waste on a server.
How is that relevant if you don't have the GUI running or don't even install it?
As you commented SSH in from a desktop or if you really want a GUI on the server run something light on resources & only run it as needed.