Well, if I want to keep my altitude in an A320 during manual flight with FD off, I would indeed be monitoring the digital readout, but you could also look at the area I highlighted with an orange ellipse in the picture below.
You can see that the selected altitude in blue is slightly above the yellow reference line, so you know that your slightly low just by looking at that.
At FL370 in RVSM airspace, you would normally be flying on the autopilot of course, but there's nothing stopping the pilot from flying manually, FD off, during approach (unless you fly for some stupid company which doesn't allow their crew to do so)
