It happens, with flying and any other skill - use it or lose it. You notice most if you've been flying every day and then have a break. After I did the CPL(H) flying couse over a few weeks I felt like an R22 was a part of me; I strapped it on rather than flew it. I've not been able to get back to that point since. On the other hand, I'm better - or perhaps less bad - after a break than I used to be. And the more you fly, the longer you seem to be able to leave it without your skills totally deteriorating. Oh, and the quicker they come back too. And all of the above seems to vary between individuals.
So now I''ve probably totally confused you! I hope you'll get some answers from some of the really experienced people who post on this forum, as I'm still finding my way with this one too.