I totally agree, the authorities should look deeper and not longer!
I personally know that none of my personal references were contacted. I still got my pass. I have flown before, this could be the reason. But still not acceptable. And like yourself I took lots ot time and effort to go back five years. And had to provide references for a 2 day break between jobs!!