Thanks to all for the answers.
Personally I don't see anything wrong with this autotrim function...
Whenever the Airbus pilot inputs a command via the sidestick when flying the aircraft manually, it helps relieve the pressure on the elevator in the same way on a conventional aircraft one would have to use the electric trim (or manual trim) to relieve the pressure and this is not unlike the autotrim feature on aircrafts I flew when the AP is engaged it's just the Airbus takes it a step further and does it when being flown manually.