Bloody hell, this is hard work.
I don't know how to explain it any simpler.
Fine, fly your aircraft with blind adherance to the FD if it makes you happy and you really think that's what you should do.
I started my commercial flying on an aircraft with no auto-pilot, no auto-thrust, and no flight directors. I still fly by reference to the pitch and bank indications, the speed tape and the N1/EPR gauges. Everything else on top of that is nice to have, reduces my workload and I use it, but the FD is not my primary reference. I can, and do, ignore one or both axes of it in the short term if I need to in order to fly the plane safely.
.