Krallu
As you know IFR and VFR are purely rules so you can fly a complete IFR flight in perfect weather.
You can also fly an IFR trip where you may climb through layers of cloud and then sit in VMC conditions on top of cloud.
You can fly IFR where you enter cloud at 200 feet and dont pop out again until 200 feet at your destination.
It maybe that all your training has been with an instructor so that you are not confident on your own. In that case going with an instructor is not going to help. Try flying in IMC with a PPL who is less experienced than you so you feel in command and on a day with a high cloud base so that you can drop in and out of cloud at will.
You sould be as relaxed in cloud as out of it familiarity is the key.
Some people suffer with claustraphobia. A pilot friend of mine is very comfortable flying VFR in VMC but panics when out of sight of the ground either in cloud or on top.
Pace