The thing about navigation is that it is almost impossible to do too much preparation. If you have done that properly, then the basic rule applies: If you fly the headings accurately for the right times at the right speeds etc, you have to arrive in pretty much the right places. That takes confidence which only comes with (continuous) practice.
I have noticed that when people say that sometimes it all works and sometimes it doesn't, it's because sometimes two errors cancel, ie inaccurate heading counteracted by forecast wind being out. Unfortunately, when the errors are in the same direction, you can go off track rather quickly.
When you look at an airfield chart, turn the picture round and visualise it from your approach direction, same principle as track orientating your map in the air.
If it's any consolation, when I was doing it every day I never thought about it. Now I don't do it every day, I have to work at it a bit harder, ie more preparation. In the end however, practice is everything.