I know what you're talking about
: when I had to transition from B777 to A320 I found that an horrible shortcoming. As a matter of fact, one of the worst things to get used on the Bus.
It won't kill you, but it's scaring and will sometime prompt 'dual input' from your fellow - remember that LH go around on windy cond in Hamburg couple of years ago.
Fortunately, I'm back and happy on the triple.