Their view, and I understand their view and respect their view, is that unless you're an “outsider”--i.e., somebody who has never worked in the public sector--there is a risk that you will not be objective, that you will tend to favour former colleagues or friends.
That's it in a nutshell. I think they've said relatively positive things about me, except insofar as I am a member of a class of people who they believe should not be the Public Sector Integrity Commissioner.