The reality of delegated authority is that the FAA picks and chooses what is delegated and what it retained, whatever you might think of it in your reality.
Delegation allows the authority to provide a service with more efficient use of authority staff time. If there were no delegation, the taxpayer would be covering the cost of hundreds more staff, both for the volume of certification work, and the in depth knowledge of certain disciplines.
Oh, by the way, 'ever had a pilot flight test? Was it administered by a staff pilot of the authority? Or a delegated examiner? C of A's issued to aircraft - most commonly by a delegate, rather than an authority staff member.
I always invite the authority to witness certification tests I carry out, even if the test is entirely delegated to me. Doing so affords the authority the opportunity to learn, while witnessing testing they otherwise probably could not see first hand. In co witnessing testing, the standards for the test and acceptance, and agreement as to how to disposition unexpected outcomes becomes a joint task, rather than just the delegate - 'builds confidence in the certification system.
Delegation is not perfect, but it's necessary, or our industry would grind to a slow pace.