Flying under the instrument flight rules is like any other flying activity the more you do it the easier it gets.
What we need to do is look at it as two separate issues though.
Flying the airplane with reference to the flight instruments is easier than flying looking outside because there is less to have to look at and with only the flight instruments you fly far more accurately.
Personally I feel more comfortable flying the panel than flying by outside visual reference.
However flying on instruments and under the IFR rules in high density air space is another matter and the work load goes up, way up. that is why an auto pilot is a must have aid if you are flying single pilot under the instrument flight rules.
By the way I personally quit flying single pilot IFR many decades ago and am a firm believer in the two crew concept and at least two engines.
I do not fly single engine IFR period.