Thank you for your question.
That's something the research community is also trying to understand. There are certainly ways you could start with a prompt. A bare minimum thing that you could do as a test would be to have a prompt and vary it slightly to see how different the outcome is. This is a known approach that quite a few people use, and that's just before you set anything up. There are some potential mechanisms that you can have downstream, in which you're able to measure the differences: “A prompt like this produces that, and a prompt like that produces this.” There are ways you can do that, and I recommend doing this in order to understand the sensitivity of the summaries or the filtration you're seeing.
What we also see a lot of in research are these “benchmark datasets”. This is as you said: We have a series of scenarios that we decide upon ahead of time, and we test the models against those scenarios to see how they perform. We have a predefined way of judging their accuracy. A lot of benchmarks are built by people who are AI experts or AI engineers and might be auditing their technical capabilities, for example, and how well they perform with a science, math or programming problem.
There are more benchmarks that look at societal questions, such as safety, or the harm of AI systems. People define what is safe, what is harm and what are scenarios when you must act in this or that way. Canada needs a version of this, because there's nobody out there who's saying, “Here's the Canada-specific benchmark that's unique to our demographics and our problems.”
You can do it at both ends. Prompt variation in your office is simple. I would recommend that you give it a try and see what happens. A bigger Canadian values benchmark or audit is more resource-intensive, but it could be valuable to the discussion.