My biggest complaint with the medical is that you're allowed to drive a car or a motorcycle without one. I believe it contributes in a very real sense to safety for private pilots (and their passengers); while probably politically impossible, I believe a similar requirement would provide a safety benefit to car and motorcycle drivers as well.
That being said I think there is room for slightly relaxed requirements for private pilots.