I understand your point but it is only a question of definition.
Perhaps one could look at it like this. I fly a commercial, two crew aircraft. on a visual approach I actually control the path of the aircraft by looking out but the actual control of the aircraft is by reference to the instruments; very often you just establish yourself visually onto the ILS.
When one does this one can still maintain lookout for other aircraft and there is also another crew member who should also be looking out. Often a clearance will be "with visual reference to the mentioned traffic you are cleared visual approach as number 2".
Another interesting point is that trying to fly accurately with sole reference... in VMC is harder than in IMC. Very often people fly better on a partial panel than they do otherwise; a matter of distraction again and concentration.
As you can see it is very difficult to define what one is actually doing in minute to minute terms for logbook purposes and it is not practical to deduct one minute from one's instrument time to cover the take-off run and the last flare and landing.