hi,
I operated the 319/320/321 only three years, but got confronted with that terrible feeling of having full deflection on the sidestick with no apparent airplane reaction twice... Upon questioning the Airbus guys, no real answer was gained, except they said that granting a faster and bigger roll-rate on final was not acceptable because the risk of pilot-induced-oscillation was too great in a light-gust situation. In other words: "live with it!"
Although you might have had a run of bad luck, thus accruing more go-arounds than average, don't change anything to your thinking: sure, on the one hand, keeping that stick right there, and trusting that the roll will kick in before the concrete jumps up, will prove correct 99% of the time. However, 10 go-around are vastly preferrable to 1 wing-tip or engine strike.
I'm still a great fan of airbus, but it's got it's peculiarities, which have to be known by everyone operating it. The most important thing for you to do seems to be filling out a flight crew report with exact details every time something like this happens, eventually raising the awareness of chief pilots and instructors to the potential handling problems. At my company, we dramatically reduced the incident rate on the airbus fleet once those little "quirks" were spread around the corps during refreshers..