Public Accounts Committee on Sept. 26th, 2006

[D/Commr Peter Martin]

As I mentioned earlier, we have made a lot of significant progress on correcting the accuracy of the data.

Mr. Chair, let me speak to the issue of registering things such as soldering guns. The volunteer verifier program has been eliminated. We have had our professional agents, who are employees of the centre, exposed to extensive training, and that has improved our ability to verify the accuracy in the qualification of various firearms.

[Brian Fitzpatrick, Conservative, Prince Albert, SK]

I want to follow up on that point. You're getting me right into the area you're in. We seem to have a system that has a lot of existing errors in it, and it would seem to me if you're the pilot coming into the Toronto airport, you have to fix the errors in the system so that the system works.

We have a problem right now: there are a lot of errors in the system, there are two different computer systems here, and there are lots of problems. I've thought my way through this thing, sir. Some things...you might check the addresses and get those sorted out, but it seems to me the only way you can really verify the model and the serial number and the action of these firearms is to actually make a physical inspection of those items to get that information accurate. How many registered firearms owners do you have in your system right now?

[D/Commr Peter Martin]

I can't answer that.

Mr. Bidal, do you have that?

[François Bidal, Director General, Canada Firearms Centre]

It's approximately seven million.

[Brian Fitzpatrick, Conservative, Prince Albert, SK]

Seven million? Do you have any idea how many government inspectors it would take to go into the homes of all those gun-registered people in the country to verify and double-check this information to get it right? Do you have any idea how many people that would take?

[François Bidal]

We have not undertaken that study, to come up with an estimate of having government people conduct verifications on the seven million people.

[Brian Fitzpatrick, Conservative, Prince Albert, SK]

Well, how else would you do it without doing that? You can't do it on hearsay. We have legal people here; they'd understand the weaknesses of hearsay.

[D/Commr Peter Martin]

We have implemented analytical tools that will now allow us to go into the database and do certain checks. Certain makes of firearms have a sequence of numbers that will allow us to predict whether the validity of the serial number is right or wrong.

This program came over to the RCMP on May 17, and the address issue is one example of how some of the data in the database has been corrected. There are other examples, and we would be more than happy to provide that information to the committee.

[Brian Fitzpatrick, Conservative, Prince Albert, SK]

Would you agree with me? The Auditor General, if I'm reading her report correctly, says you have a mandate, and your mandate is to improve public safety. When she looks at what's going on in the department, there is actually nothing in place statistically to show that the activities taking place are meeting that goal. There's nothing in place that is devoted to that fundamental goal, which is to reduce the risk to the public of firearms. There's lots of self-analysis about what you're doing and so on, but there is no analysis statistically to link the registration system and all the rest of the stuff you're doing with actually decreasing or increasing public safety.

Is the Auditor General correct in saying you really don't have any systems that can measure that or are measuring it?

[D/Commr Peter Martin]

I can't speak to prior to May 16, but right now I can tell you that we do have analytical tools in place. We have a team that's working every day on the data quality issue, and there are operational examples of how this system has improved public safety.

You mentioned we're working on two systems. No, we're working on CFIS I. The issue around CFIS II is under negotiation right now; we're not using CFIS II. Within the RCMP we have a mature technology operation under the chief information officer, and we are able to maintain the CFIS I system with those technical resources.

[The Chair, Liberal, Shawn Murphy]

Thank you very much, Mr. Martin. Thank you very much, Mr. Fitzpatrick.

Mr. Christopherson, you have eight minutes.

[David Christopherson, NDP, Hamilton Centre, ON]

Thank you very much, Mr. Chair.

Thank you all for your attendance today.

Let me say at the outset that as these things go, this isn't too bad. There are some problems, and we're going to talk about those. But I do want to start out on a positive by saying there were huge problems prior to the 2002 audit, a lot of condemnation, and it looks like a lot of those things have been cleaned up and are on track, if not ideal. Folks are to be commended for that. I think that is a good way to start.

Having said that, however, I want to move to an item that the Auditor General has characterized as being important in her opening remarks and that has been addressed by others. I want to revisit the whole issue of performance targets and performance accountability.

Paragraph 4.36 of the Auditor General's report says, and I quote:

In particular, the Centre has not set any performance targets and has provided few examples of its outcomes. Instead of reporting the key results achieved, the Centre describes its activities and services.

The recommendation in paragraph 4.41, on page 109, says:

The Canada Firearms Centre should improve its reporting on the performance of the Canadian Firearms Program by providing targets and evidence-based results, and by showing through the use of a results chain how these results could contribute to public safety goals.

It would seem that in 2002, in the last audit, according to a brief given to us by the library staff in their report—and it's tabled, members have it in front of them.... This is the 2002 audit that says in part, and I quote:

The Centre’s performance reports provide a fair to good overview of its organizational context and planned strategic outcomes. However, the Centre has not set any performance targets, has provided few examples of outcomes, and does not have a results chain showing how its activities might contribute to public safety goals. The Auditor General recommended improving reporting of performance by providing targets and evidence-based results and by developing a results chain. The Centre agreed.

While I ask Ms. Fraser a few questions, perhaps, Mr. Bidal, you can turn your mind to answering—because you're next—why nothing was done. Why are we getting a second audit report telling us that exactly the same problem, which existed in 2002, still exists in 2006? You can think about that for a moment.

Perhaps, Ms. Fraser, you can expand on exactly what we are talking about in terms of these performance results. In layperson's terms, what exactly is it that's lacking?

[Sheila Fraser]

As we mentioned, the strategic outcome of the centre was to increase public safety or reduce risks to public safety. To make the link between the registry and public safety can be difficult. There is no automatic link, so you have to build various steps to do that. This is what we call the results chain. For example, it could be providing better information to police officers, and they would use this in a certain way that could increase public safety.

We're saying the centre had to develop that kind of results chain and then try to find indicators that would indicate what the outcomes of the registry are—not simply the licensing and registering activities and the number of hits in a day, because those are statistics on activities, as we say. They don't really give you any appreciation of how the centre is contributing to better public safety.

We can take an extreme case and say, yes, there are 5,000 hits a day, but if the information is no good, are people using it? And is that helping public safety? You could make the link and say, yes, there are 5,000 hits a day, the information is good, police officers are saying they are using it, and it has helped in these cases. It's to develop that chain and logic from the actual activity through to the results.

[David Christopherson, NDP, Hamilton Centre, ON]

I would assume that maybe, once they have set that, it's a bit of a template; then they just go back and fit in the results—

[Sheila Fraser]

Over time?

[David Christopherson, NDP, Hamilton Centre, ON]

Yes, over time, to see whether or not you're actually achieving—

[Sheila Fraser]

That's right, end results. Performance measures will have to be modified.

I think we have to recognize, too, that these are not easy things to do, and it takes time to develop the chain to find indicators that really do tell the story. Sometimes they can modify behaviours in an improper way. It does take time to develop these results chains. I think, though, in the current context, where there is a lot of discussion about the program, it would have been very helpful to have that kind of results information.

[David Christopherson, NDP, Hamilton Centre, ON]

Mr. Bidal, why do we have a second auditor's report telling us this is still not fixed?

[François Bidal]

Mr. Chair, I can only speak to the events as of June of this year, which is when I took the position of director general as a member of the RCMP. I note that already there had been a framework put into place by the previous administration to ensure that there was an improvement in how performance is managed and how it's reported.

[David Christopherson, NDP, Hamilton Centre, ON]

I don't want to be rude; I'm sorry. You've only been in this job since June?

[François Bidal]

Yes. I'm a member of the Royal Canadian Mounted Police. I've been director general since June.

[David Christopherson, NDP, Hamilton Centre, ON]

I'm going by this sheet. That's why I picked on you, if you will. I saw Canada Firearms Centre, and you're the director general.

Well, then, who is going to take responsibility for why this didn't happen, sir? Is it the commissioner, the former commissioner?

[William Baker]

Former.

When we arrived, the new team, in 2003, we had a number of problems, frankly, including that we were still very heavy on the production cycle; that's number one. We had very little confidence in the basic data. Unless you know how many licences there are, how many registrations, how many hits on the system, you're hard pressed to take the analysis any further and start suggesting impacts and effects on public safety. Our priority was to do what we could to get the data as accurate and up to date as possible.

Of course, it wasn't the only thing we were focusing on, as you can appreciate, including production, improving service to Canadians, and so on.

We had gotten to the point, probably toward the end of 2005, where we were starting to build some of the analytical tools to do this. That was not evident at the time of the audit; it was just in its infancy. And as Mr. Bidal has mentioned, further work has proceeded since then.

[David Christopherson, NDP, Hamilton Centre, ON]

I hear all that. I'm still having a little difficulty understanding, though. Concerning what you just said you needed to do first, I just heard Ms. Fraser for the most part say that those are different things, and you're mixing apples and oranges when you do it; that the number of hits, in and of itself, doesn't tell you anything.

It's the whole idea—I'm trying to get this right—of this chain to link all of these things. So it would seem to me that even in the absence of having those accurate numbers at the time you started, after this report came out you should have at least started on the template, begun the process to ensure that once you have those numbers and they're reliable, you have a context to fit them in, if I'm getting this right.

My time is up, so there you go.

[William Baker]

I think, with respect, the point is debatable.

My view as commissioner at the time was that if we tried to engage in that kind of analysis and reporting without having confidence in the core data, we would run the risk of being ridiculed, given the incredible scrutiny that was being placed on the data and the reporting from the centre. I erred on the side of underselling and trying to concentrate on ensuring the integrity of what we did report.

I appreciate the point. This is certainly valid for future consideration.