If you want stress, report the matter to CASA. In this regard our "safety" watchdog is culpable by punishing rather than congratulating wise decisions and such an attitude has the potential to cause a significant safety issue.
You are the best judge of you fitness to fly. You appear to have made the decision already. Remember "if in doubt bug out".
Stress can lead to depression which can lead to not very nice places. Be careful you don't let it go too far and consider talking to Black Dog
Depression - Home - Black Dog Institute or Beyond Blue
http://www.beyondblue.org.au/index.aspx?
There is no shame in doing this and you will probably find that there are fine lines that you may already have crossed.