You said you had a job offer AND a contract. Why would they give a contract prior to them contacting referees if the reference check was going to be used to make the decision?
Professional Referees/References are valuable to you.
Referees are to confirm the decision made by the recuitment process, not to make the decision. No one would name a referee that is going to say something bad about them.
Organisations/Recruitment Agents that demand 3 referees upon application are to be steered clear of as you will get some first year HR graduate that harrasses the
out of your referees and sooner or later it is going to annoy your referee that is there to say good things about you.
Simply state that you will name referees at 'preferred candidate stage' and not before.
It sounds like an organisation that you wouldn't want to work for anyway.