AA, I think you are generalising a bit about the public, or non-aviation folks.
It is a global phenomenon but to simplify it down to jealousy would imply that everyone is in awe of aviation and rotorcraft.
There are plenty of people that just couldn't care less and for whom getting into a helicopter has all the excitement and allure of caravanning on a typical British summer's day.
This isn't helped by either Hollywood or the seemingly endless list of online videos and press articles showing them falling from the sky at every opportunity.
Boats, even noisy ones, don't have the same probability of falling through your roof while on fire.
The right or ability to use a helicopter is very much about the benefits to the few and not the many.
I have found that it is actually those involved with aviation that can be the most grumpy about the private use of aircraft but that is a different topic altogether.
When faced with the critical and sceptical eye of the public (and that of their camera phone), discretion is always a better course.
Freedom's are great until someone doesn't use them responsibly or enough people take offence, at which point it disappears in a puff of regulation.