is there a clause somewhere in ICAO's Annexs saying about the rights of pilot to refuse to fly if it seems unsafe?
There probably is. (Operation of aircraft) - ICAO Annex 6 maybe?
But realistically if you were to refuse to fly based on an ICAO article, you'd likely get the sack