I find this fair enough as the captain(and not the FO) is responsible for the safe operation of the flight and even more for safe taxi procedures. If something goes wrong, even initiated by the FO, they always go for the guy in the left seat.
I agree that captain is overall responsible for safety, but I consider takeoff (including V1-cut scenario) and landing much more dangerous than taxiing an aircraft at 10-15 konts and making some turns. If FO lands so hard that the landing gear collapses, they will come after you as well - but I guess you still let them land?
Best example - in my current company we do as well lots of flights into Africa, had already a few "wing touches" on narrow parking areas while under control of a marshaller. He waived them right into the obstacle and of course they made the captain the black sheep - even changed procedures so captains would be even responsible for damage caused to the aircraft while under marshallers guidance. In this case I prefer to drive the thing into the wall by myself rather then let the FO do it .
Well, it is captain's decision when to delegate PF duties to FO. I can understand if you wouldn't let an FO taxi the aircraft on a narrow apron in Africa at night (just as you would probably take controls in case of any major flight controls problem for landing), but why not let them learn on a not-so-busy airport with large empty apron and wide taxiways at broad daylight?
One way of looking at this is - if they know how to do your job (e.g. taxi the aircraft, start the engines, etc.), they can be much better at noticing something is going wrong during this part.
The same goes for a Rejected TO. I see(like most companies) no problem why a FO could not call out a malfunction, but making a STOP or GO decision should be(and is nearly everywhere) the captains decision. To many "opinions" during such a critical phase can cause confusion and lead to errors where you do not want to have them.
In most companies where FO can taxi the aircraft, callout "STOP" during takeoff roll automatically transfers the control of the aircraft to the guy in the left seat anyway.