Ham Phisted,
There may be as you say, "a natural progression" one way and not the other. Whilst that may be true, my two colleagues ahead of me that I mentioned, are certainly not finding the transition back to VFR flying difficult. Theyre actually rather enjoying it and finding it pretty easy compared to the IR. They also commented that the CPL 1st would not have prepaired them for IR.
I guess theres never going to be a right or wrong with this old debate.