I got the same problem as yours my eye tend to drift outward that happened during my class 1 and i failed the medical, The problem name is exophoria. I appealed to the CAA and i got my unrestricted class 1 because im able to compensate with good fusional reserves and i dont have diplopia. Dont lose hope to quickly, if you are sure that you dont have double vison than elaborate your case well send it to the CAA appeal to the CAA.
Sorry for my bad english.