They do have an obligation to justify every refusal. It's just that they don't have to tell you right away what that justification is. If you were to take it to court, yes, the judge is going to look at whether that decision was properly motivated. I'm not sure whether it would make a difference if the officer had to give that justification off the top to every single person. They are really under a lot of time constraints.
One good idea that one of my colleagues suggested is that they should allow the posting of bonds for people. If there's a doubt as to whether they're going to go home after their visit, why not allow for the posting of a substantial financial bond to guarantee they're going to go home?