Now "Unfrocked" (as of a clergyman) has a clear derivation. He/Her has the right to wear the vestments of the Office withdrawn.
That's probably the origin of the term. A frocked clergyman is one who is entitled to wear the vestments of his ecclesiastical office. A defrocked clergyman can no longer wear those vestments.