Flying manually with it engaged is OK - but not recommended due to pitch/power couple..
It's best to use it in "arm" mode when flying manually - ie: autothrottle on with no mode selected (eg: lvl chng, v/s etc.) On a manual approach, it's best to have it armed for alpha floor protection, and also TOGA available, just in case.
But... just disarming it (ie de-selecting speed mode) also ditches the flight directors - so if you are flying it, the other pilot can't monitor as easily by reference to the flight director bars - so you need to turn the autothrottle off, with the modes selected as normal so that you can get FD indications.
Make sense? Probably not!
