Go flying with other pilots first and get some confidence and then take family
Excellent advice. Simply bum a ride with more experienced pilots. You know how you fly, and can compare it to their flying ability. That will give you an excellent idea on how good or bad you really are. And you get to see some good and bad examples.
Next, take a few pilots with you while you fly as PIC, and let them honestly (but nicely) critique your flying. You will probably find you're not as bad as you thought.
Only then take non-pilot passengers.