I think it's one of those things where you need to (quickly) weigh the circumstances of the event rather than have a rigid rule.
I think you are wrong ... I think you, as a captain, need a strategy in advance to execute (and brief) should it happen, so as to avoid a surprised reaction from your colleague.
None of these grey area 'how close to RTOW are we', 'what is the viz like today', 'what is the fpv giving us, less 10%' type calulations at a critical time serves anyone well.
At the end of the day, who cares if you divert/don't reach destination as long as all survive the event. The job is not about making decisions that 'may' work out ok, it is about making decisions that 'will' work out ok and being clear of blame in any investigation in this litigious era. If in any doubt, apply the OEI procedures, don't gamble!
Having said all that, the Captain's decision is final ... so long as he can justify it .... and that responsibility/accountability comes with the fourth stripe