Seems like you have had bad luck.
Even though I was nervous about my FI test, I found it to be one of the most relaxed flight/ground tests I have ever done, once you are into the swing of it, it turns out to be enjoyable.
You must remember you are not expected to know everything, however, you must know where to find the information should you not know the answer to a question.
On the flying side, take your time and make sure you know the correct patter, and that you are proficient with the mandatory items such as the spin demo and PFL.
Good luck with it. Once you start working it really is a great job.
All the best.