No you cannot fly outside the 180 minute range ring (unless an emergency, although it'd be hard to imagine one). Your ops department have chosen a weight and corresponding speed to work out how far your aircraft would travel on one engine in 180 minutes in nil wind. Call this distance X. This distance X is your EDTO limit. You cannot fly further than Xnm from your EDTO Alternate.
However, on a given day, you are Xnm from your EDTO aerodrome, and to divert there there is a massive headwind and it might take you 3.5 hours to cover Xnm. This is fine.
Your colleagues are correct in that it's a planning exercise in nil wind. You could be more than 180 minutes flight time away from safety on a given day, but not more than the distance defined in your ops manual that has been determined to equate to 180 mins.
Things get a bit more complicated at higher ETDO limits but don't worry about that yet....
Just understand that a 180 minute is actually a predetermined distance range ring, not a time ring.