Nice to see the old Hermes thread resurrected. The RAeS interviews with D.P. Davies have an interesting section on the Hermes (about 5 mins in to the Comet/707/Britannia episode).
He said it "flew like the Forth Bridge, it was terribly heavy". He recounts the story of doing engine out testing at Boscombe Down and after 3 circuits he had to have a rest as flying the test was so exhausting. He puts it down to the flying controls not being sufficiently balanced - huge aileron/elevator loads and rudder loads that were right up to the limits of the requirements.