At the moment it's possible your previous experience is hindering rather than helping - you are distracted by it, right now you need all your attention to simply fly the plane. I'm sure in time it will become exceedingly useful.
And yes, I agree it sounds like they are rushing you. Until you have the control basics fairly sound you won't have the spare attention to deal adequately with where you should be in the circuit and so on.
As others have pointed out, learning in one's 50s is quite different to in one's 20s or 30s.
I'm sure you will be just fine, but it takes more than 4 lessons!