I think the simple fact here is that unlike 46/s53s/60s/etc, which can be heard MILES away from an LZ during approach, especially in the valleys, the V22 is almost audibly undetectable until it starts conversion. Thats what is universally being referred to when you read these anecdotes regarding its quietness.
Sure, in hover its as loud as anything, but you didnt announce your presence 15 minutes prior.