I feel you may have been reading some of my work. This is actually a finding from a study I did with a couple of colleagues a few years back, but it's also common experience. Almost everybody knows that conflict of interest is a problem, but very few people can offer more than a very thumbnail definition. They often can offer an example. They might be able to come up with an example like my judge example.
I think we can point to two things as reasons. One is that conflict of interest is a modern concept. The term “conflict of interest” didn't appear in the English language until the 1950s and didn't enter law dictionaries, for example, until the 1970s, so it's a relatively novel concept.
The other thing is that because it's rooted in professional and business contexts, it's not part of the everyday texture of most people's lives. Most of us grew up with parents who taught us right from wrong, as in “don't hit your siblings”, “don't take other people's stuff”, “clean up your own messes”. Very few people's parents taught them about conflict of interest. It's not something that we're brought up knowing about. We may have some sense that some fairness issues are related to it, but that's why organizations need to do a lot of work, in some cases highly specific work. If you look at the conflict of interest policies of banks, for example, they offer extremely detailed, extremely specific guidance on how to deal with what for many people are quite novel situations.