That is indeed the risk.
It is precisely because of this risk that we seek more transparency, so that we could know exactly what algorithm Chinook uses, how the system works and what directives are given to the officers. We also need an independent party who would be charged with overseeing these operations and making recommendations.