In my experience, basic handling skill isn't usually what's lapsed most when I've not flown for a bit.
RT and following of procedures are usually the first to go - I spot myself making small mistakes, or getting into a bit of a muddle over some complex RT if I've not flown for a bit. The other biggie is capacity - if I try to do something fairly complex: say a DME approach, or flying a PFL into a smallish field, I can fairly easily get overloaded and less able to cope with anything additional coming my way.
John Farley wrote an excellent chapter on this in his book "A view from the Hover", which also described the importance in his mind (and I'm certainly not going to say that JF was wrong!) of also making sure if you fly relatively limited hours or usually in a similar pattern of taking the limited time you have to rehearse the most important skills. He advocated constructing a personal checklist to ensure that you've done various things that matter to your flying within a reasonable period of time.
I don't formalise it quite as much as he proposed, but certainly I think that his arguments are right. For me RT in controlled airspace, PFLs, stall recognition and recovery, and flying accurate (IFRish) height and speed occasionally are the things I most carefully watch. I'm sure that other people on self analysis would identify other things that matter more to their flying.
G