It CAN be done.
I tried a search of property in South Lantau entered 5-7 mill, and up came a few village houses well in excess of the sizes being discussed.
It's not everyone's cup of tea, but ask the people who live there and they think the rest of us are mad. I would take a trip to the Mui Wo Real Estate offices and setup a day of visits, see if it is for you. There are the villages all along the south coast with a different flavour to all and an array of prices. Maybe someone can help with more specifics or other similar areas. There are sometimes cards on the CX City noticeboard from S Lantau specialists. There are Mortgage issues, valuation complications and you need to research the area and transport, but as I said, some people love it.