That's a very technical question. As far as the technology inside the black box goes, governments or companies sometimes argue that the information is proprietary and therefore confidential, that it would reveal business practices. According to them, competitors could benefit from disclosure of the information.
It essentially comes down to replacing or supporting decisions made by humans with recommendations that are practically decisions in and of themselves. In a number of cases, in fact, it would be very tough for an individual to challenge what the computer has deemed a good decision. In order to challenge what comes out of the black box, a person has to be very confident in themselves and their expertise. That's key.
When people make decisions, they normally have reasons for making them. The data they used to arrive at their decision are known. It's important to keep that option open It's not a good idea to create a situation where it isn't possible to reconsider what the decision-maker did. If the decision came out of the black box, it's important to know that.
That doesn't necessarily mean knowing exactly how the circuits are connected, technology-wise. Rather, it means knowing which data were part of the data mix. It's actually a matter of knowing, overall, how the data were organized to arrive at decision x or y.