I would think that there probably is a percentage of folks who might see that as a barrier.
Anecdotally, when I was a seagoing chaplain there was a tradition in the Canadian navy and in the Royal Navy where chaplains didn't wear rank at sea because it was seen that the rank itself might become a barrier for the sailors to seek the chaplain out as somebody who could be of support to them. I never felt any concern about taking the rank down because in our professional military there is a great rapport and working relationship among all ranks. There is a team approach.
There are obviously differences in responsibility, and job, and so forth, but I never saw the rank as the barrier. I saw the cross that I wore and the collar that I wore.... Because, you're right, there are a good number of folks in our culture and society for whom religious leaders, people who have a leadership role within organized religion, are seen somehow as distant or out of touch or even worse. There might be pain or injury that was caused in an individual's life by organized religion, judgment felt, and so forth.
Trying to overcome those barriers is an important part of what every chaplain needs to do when they're assigned to a unit, and it really does start by meeting them on their turf, journeying with them, getting to know them, and developing a relationship. I can honestly say that while not every soldier, sailor, airman, or airwoman in the forces.... In fact, the vast majority of them do not go to church or synagogue or mosque on any kind of regular basis, but they know who their padre is. They know how their padre can be a source of help to them, an encouragement to them, and the issues that they might identify they would never identify as being religious issues. They might not even use the term “spiritual issues”, but there is a spiritual dimension to those concerns.