I agree that the number of visitors is possibly a little high. The data on days and spending are in-line with research conducted on other events in South Africa (cricket and rugby world cups) and other soccer world cups. These numbers are probably an understatement since a significant number of people stay for all 6 weeks (mostly media over 10000, officials 500, etc).
FIFA do own the World Cup and the TV rights and essentially the country pays FIFA an amount in order to host the World Cup. FIFA does provide a fair amount of the TV rights to the local organizing committee. Even bigger than the TV rights are the corporate sponsorships which are also shared since some are local sponsors (i.e. FNB) while others are global (i.e. Coca Cola).
In terms of infrastructure here are some projects which will leave a legacy - these are just some of the government funded projects off the top of my head (never mind the private sector investments):
- Johannesburg airport: new central terminal, additions to international departures, new international pier, new parking deck
- Cape Town airport: new central terminal, new parking deck
- Durban: new airport altogether
- Major public transport: Gautrain, Johannesburg BRT, Cape Town IRT, Port Elizabeth BRT, redevelopment of Cape Town Station
- Major roads: redevelopment of Koeberg interchange on N1 in Cape Town, redevelopment of Hospital bend interchange on N2 in Cape Town, multiple highway capacity increases in Johannesburg
By the way, your comment on the UK in 2014 is rubbish since the 2014 World Cup has not even been awarded and Australia and the US are favorites to win the bid. Furthermore, such a tax exemption would be illegal under European law.