I defer to those with extensive flying experience on whether it is, indeed, just a physical skill. I draw a strong distinction between the 'simple' mechanics of making an aircraft go "up diddley up" etc and the actual 'operation' of the aircraft in whatever role it may be. I have always thought the latter aspect was what really constituted 'flying' ... e.g. leading a Harrier 4-ship at low level in sh1t weather to strike a target, where the stick/rudder bit is a subliminal adjunct to all the planning, navigation, threat analysis, target acquisition, weapon aiming/firing, defensive manoeuvring and getting the formation home again in one piece.
As to ATC, I have oft described it as something of an art form rather than a mechanical skill. You can be taught the basic procedures [see previous para] but unless you are wired in a certain way you can never hope to be fully effective in that 4-dimensional game. I do believe that controllers perceive things slightly differently, in a time/space context.