I can't remember ever calling either a captain or FO "Sir" or "Madam", but I have told off a new dispatcher once for addressing all flight crew as "mate"...
As far as I am concerned, there is a minimum of respect required, which isn't evident in the term "mate".
Then again, I have noticed a few raised eyebrows on the part of PAX when I address the CPT or FO (or both) by their first name(s). I am on a first-name basis with just about every based crew (after all, they see my name on the loadsheet too). Those whose name I don't know (usually "visiting" crew), I just address without adding any title.
Simple really!
Cheers
P.S.: I have never witnessed a company or contract engineer address a CPT as "mate" either...