Yes, it is even more difficult in high Summer when the sun is shining through the canopy and you are sweating like crazy. You will find that it suddenly comes together as your brain burns the synapse pathways - do you remember learning to ride a bike or to swim?
The day it clicked for me was when I was trying hard to get out of pendular instability when the instructor asked me what was the aircraft type that had just landed on the runway. I studied it and we had a short discussion about it, then he asked me what I was doing. Strewth, I was actu ally hovering in one place! Because my brain had been diverted to think about something else, the automatic reflex actions were starting to kick in. You will find that in a few hours time, you don't actually manually initiate a move to the right or left, you just sort of think it and lean that way and the helo follows.
Later it gets easier. It has to as your brain needs to be concerned about other things like RT, collision avoidance, other traffic etc etc.
Best fun you will ever have and this is the demanding bit.