What taxes to claim back?
If you are non resident for tax purposes there shouldn't be any taxes on income arising outside SA.
I think that you will find that it is The Revenue who will determine your residency. You need to make application to them for such a determination and each year will be assessed individually. As a non resident, you will not, in any event, escape South African tax on monies arising within the republic, such as rents.
ps: skyvan is right. Do not mess with the little weasels.