This means my loyalty factor to most airlines is low
I think you have answered your own question. Perhaps if you had stayed loyal to one airline or one alliance your chances of upgrade would be higher. Added to that the fact that you
scour the net for the best rate
would also mean less chance of an upgrade as an airline would upgrade full Y fares ahead of lower ones.
Of course staying loyal to one airline or alliance would have cost you more in the long run, in which case any upgrade you may have gotten would have been "paid" in higher fares throughout your travels.