Am I correct in thinking that on start-up the PC will dump the info it stored to go into hibernation and it wont be clogging memory.
Yes, that's the whole way hibernation works.
Sleep saves your computer's state in RAM.
This makes it quick to restore, but is obviously volatile (i.e. lose power, lose stored state).
Hibernation stores the contents of your RAM on disk (similarly to what happens during paging when you don't have enough RAM and the computer uses your disk as virtual RAM).
This makes it slower to restore, but is not volatile (i.e. you can remove power source and not worry).