If you can drive or fly in a place where there's no property or people that you might damage in a state of intoxication, then doing these things while drinking is indeed your own business. If you risk damaging the property of others or injuring people while intoxicated, however, then your state becomes a concern for everyone. Laws against operating vehicles while intoxicated are predicated upon this reality.
I'm afraid I find it difficult to trust pilots who engage in substance abuse, including the consumption of ethanol, no matter how much time elapses between their abuse and their flying activity. And that is all the more true when they attempt to rationalize their abuse and criticize anyone who suggests that such abuse might not be a good idea for pilots. This includes those who think it's okay to drink as long as you remain below some arbitrary legal limit.
I think JohnMcGhie's post said it best. More than a few people here seem to be in denial.