I am really sorry to hear that, and for my part, since you were not flying an airplane or even driving a car, I see no call for it. You can be as jolly as you want when you are off duty.
I guess that Transport generally does their job in a reasonable fashion, however they have indeed abused their authority every once in a while. Somehow, if something is aviation related, ''the minister'', as they refer to TC, has sweeping powers that transcend the law as we know it, written in policy, and even reasonable judgements that reasonable people would normally make simply do not apply.
I guess you can take the decision to the federal court for appeal. No guarantees of course, since the federal court also seems to be reluctant at times to interfere with ''the minister's'' decision. Read some cases where the judge actually said that it is not the decision he would make, but hey, since procedure was followed, too bad....Makes your head shake. The constitution really does not seem to mean much in Canada.