So what you wrote in the skills test report and what actually happened (and was recorded in your personal logbook) were not the same thing ie operating from a non-licensed aerodrome but claiming you were for the purposes of the test.
Once your integrity is shown to be fragile you can hardly blame people for jumping to other conclusions.