It would be interesting to know what specific “criticisms” you have of your friend as this would provide and insight into how to address the problem and also the extent of your own experience.
Whilst not an instructor I have had a lot of enjoyment doing check outs with pilots. Experience has taught me that your own “criticism” of other pilots develops with the time you spend flying with others and with your own level of experience. For example with just a few hundred hours to my credit I recall flying with a pilot and thinking the fellow was a complete mad man. With hindsight he did nothing dangerous and in fact was far more comfortable with the aircraft than I. I am not suggesting you have insufficient experience to make the assessment you have reached, just enquiring into the basis on which you have made that assessment.
Of course any pilot should welcome criticism - it need not come from an instructor. It would be as well to raise some of your “concerns” with your friend and see what reaction and explanation you receive. The simple truth is that if our friend is not prepared to listen to your views (whether he has more or less experience than you) he should not be flying. If you really do need to contrive the situation you might each do a short leg - go first and suggest to your friend that if he sees you do anything with which he is uncomfortable then please let you know and you will return the favour on the return leg.
Whatever you do your friend deserves more than simply refusing to fly with him if you really feel he is a danger to himself and to others.
Do you know when your friend last had his biannual check ride?