Public Accounts Committee on Oct. 5th, 2017

[Ian Shugart]

I agree that is the standard, and that is the practice that should be followed. We have now remedied that with information tools that will track that data so it is retrievable.

We use the Canada School of Public Service for the delivery of our basic core values and ethics training program, and they do have the information base, the data program that allows for that comprehensive tracking. We can download that for each employee as needed through that system. We're confident that in the future that information will be available, as it is now.

[Alex Nuttall, Conservative, Barrie—Springwater—Oro-Medonte, ON]

Thank you.

It's good to hear about that progress and also the progress at Indigenous and Northern Affairs that took place during the carrying out of this audit.

My second question, Mr. Chair, is through you to the Auditor General.

Just below, it says, “ensure that employees are taking mandatory training in a timely manner.” Is there a definition for what a timely manner looks like? Is it based on the start date of new a employee in a position? Is it the date on which we hit 80% or 90% of the employees?

[Michael Ferguson, Auditor General of Canada, Office of the Auditor General]

We didn't specifically define the term “in a timely manner”. Certainly, I think the results that we found indicate that the issue was more than just new hires who hadn't yet gotten around to taking the course.

I think that is always going to be the case. I think it would be very rare that an organization will get to 100%, because there will be new hires.

Again, I think some of the comments we've heard on this issue mean that there does need to be some reflection on the mandatory aspect of this training. We've heard some things like transient employees, student employees, and things like that. The departments have now said that the training is mandatory for everybody. If it doesn't need to apply to students or somebody else for whatever reason, then don't make it mandatory for them. On the other hand, maybe there should be some other type of program for the transient employees and the student employees.

I think the problem we ran into was that the departments had made this mandatory for every employee in the organization, no matter what. What we're hearing is that most of them are saying that they're actually applying it more on a risk basis, which could make someone ask why then it is mandatory for everybody. If it's only the ones who are the riskiest that this really applies to, then make it mandatory for them, and perhaps adopt some other approach for the rest.

We didn't actually determine the timeliness, but I think there are some fundamental changes the departments need to make, to make sure that the people who are getting the training get the training, and that they get it as quickly as they can. I think it will be up to them to define timeliness. In the end, I think it will come down to a judgment on the reasonableness of their definition of timeliness.

[Alex Nuttall, Conservative, Barrie—Springwater—Oro-Medonte, ON]

Thank you, Auditor General.

Through you, Mr. Chair, this is a comment from the Auditor General's findings:

...there were limited proactive prevention and detection activities. For the three selected contracting practices, we found that none of the federal organizations sufficiently and routinely analyzed contracting data to identify trends and signs of fraudulent behaviour or non-compliance and followed up on exceptions.

Auditor General, was that all of the organizations?

[Michael Ferguson]

Mr. Chair, certainly we did find issues in the different practices in all of the organizations. None of them were doing them all 100%.

[Alex Nuttall, Conservative, Barrie—Springwater—Oro-Medonte, ON]

Very quickly, and I will leave this open for all to answer, with regard to analyzing contract data to determine whether the practices are meeting the requirements of what the Treasury has put out, or the internal controls, say, if we're not checking those over on a routine basis, if we're not ensuring an internal audit, then how are we going to know whether frauds are actually taking place?

[The Chair, Conservative, Kevin Sorenson]

Why don't we start with Mr. Matthews, and we can work our way...?

We have a minute left here. My fear here is that there are so many witnesses, we get to six minutes, we ask a question that we want all the witnesses to answer—

[David Christopherson, NDP, Hamilton Centre, ON]

Who would do that?

[Voices]

Oh, oh!

[The Chair, Conservative, Kevin Sorenson]

—and then the next person goes on a different line of attack.

Mr. Matthews, I'm taking up time. Go ahead.

[Bill Matthews, Comptroller General of Canada, Treasury Board Secretariat]

Thank you, Mr. Chair. I'll be very quick.

I have two points on this. I think there's an issue to be pursued here around the quality and sophistication of our data analytics. That's a cross-government issue that was touched upon on Tuesday. As the tools get more sophisticated, our data is not always as relevant or useful as we'd hope, due to inconsistencies. So it may not be inaccurate, but it may not be consistent. There's some sophistication related to our analytics that has to be improved.

The second piece is with regard to internal audits across the board, done by both my organization and the chief audit executives in each department. The most consistent finding in internal audits is poor documentation. There may not be a justification on file, or something may be missing. It doesn't mean the rule was broken; there's just a lack of evidence.

If I could sort of pull one finding that crosses all audits, we keep bumping into that one.

[The Chair, Conservative, Kevin Sorenson]

On that, this is exactly what we hear on every Auditor General's report, on whatever the issue may be. It is exactly as you say, Mr. Matthews. It is data and documentation: how do we analyze all that documentation? I think these are really challenges to every department. They're challenges we need to find some better way of responding to rather than just some interim way and then we forget about it again.

We will move on now to Mr. Christopherson.

You may want to carry on with the same line of attack.

[David Christopherson, NDP, Hamilton Centre, ON]

Thank you, Chair. I do, actually. I will pick up where you left off, and then I want to return to the previous issue.

Mr. Matthews, you touched right on an issue that the Auditor General has raised as an issue. We've adopted that, and for the next year...we've been trying to focus on this. We should do a review. I think it's with somewhat limited success. We kind of hit the surface of it. But it's the first time, I think, anybody has really come forward and, from a macro point of view in government, acknowledged what the Auditor General is saying. The question now is what do we do about it?

We need to develop some kind of measurement—I guess I'm looking to both you and the Auditor General—but at some point we need some ability to measure whether or not we've been effective in effecting change within these departments. If all we're doing is holding hearings and throwing around some words and phrases, with the odd little headline and then it all goes away, what's really changed? What we're supposed to be about is meaningful, substantive change.

At a macro level, the Auditor General, and now confirmed in the very words of the comptroller, has said, look, we have a data problem. And I think we're echoing that; I'm just not yet convinced that we're getting any traction and any change.

In terms of going forward, I would look to you, Mr. Matthews and Auditor General, to give us some advice. What can we do to be more effective in putting the pressure on government to ensure that there's a cultural change as it relates to data, keeping it current, and analyzing it in a proper, useful way? How can we, as a committee, do that, working with the Auditor General and the management team, to bring about that cultural change? How do we do that? Help us.

[Bill Matthews]

Thank you, Mr. Chair. Maybe I'll start, before I turn to Mr. Ferguson to let him finish off.

I don't want to put words in his mouth, but I would tell you, and this is pure opinion, that the data issue is not new. It's coming to the forefront more and more, because the data analytics tools are becoming more sophisticated—we want to make greater use of data. It's thus putting pressure on our data. The people who are collecting the data are often not the people using the data. We have a cultural issue, in the sense that the accuracy and the relevance of our data are becoming more and more important. We need to drive that cultural issue down to make sure we get accurate data.

If I may offer advice to the committee on what to do, it would be to challenge the data that departments are collecting: is it relevant; is it accurate; how are you using it? Those are the three themes that have to come out.

Mr. Ferguson will likely tell you in a moment that we collect all sorts of data. The question is around how we use it and whether it's the right data. We probably collect more data than we need. For me the real issues are around how we use it.

[David Christopherson, NDP, Hamilton Centre, ON]

Mr. Ferguson.

[Michael Ferguson]

I certainly agree with the committee's challenging the department whenever we are bringing forward issues around data and data quality. The way you just posed the question very clearly sends a message. The fact that the committee is very clear that anybody who is going to be appearing here can expect to get questions about data and that you will continue to challenge it is, I think, a strong message. You need to continue to do so and to continue to reiterate this.

As to what needs to be done to actually fix the problem, the way I keep looking at it—and I think I've said this before to this committee—is to determine whether, if I look at any organization preparing a set of financial statements, the data is good enough in quality.

The quality is there such that we can produce an audit opinion on the set of financial statements. That's because the culture and the environment have been built up to put controls in systems to make sure that what's going into the data is properly captured and recorded. It's my opinion that departments don't have that culture necessarily in all of the other program support systems.

Systems that don't feed into preparing a set of financial statements don't produce a lot of financial information, but they are important perhaps for managing who has taken training or for managing how long it took to resolve a declaration.

When you're dealing with those types of program support systems, which are equally important to making sure that people are getting their values and ethics training and so are a very fundamental issue, the same level of controls isn't applied to the data being entered and to making sure that the data can be used.

The system, then, knows how to make sure that quality data gets entered into a system. The system certainly does it to an acceptable degree to produce a set of financial statements, but it doesn't always do it for the other systems. That's what needs to change. The culture needs to broaden that control-of-data mindset from financial systems only to all their other systems.

[Ian Shugart]

Let me just add from the perspective of a department that the Auditor General said has a number of specific data practices on contract management are in the positive ledger.

I would say that program data—and I agree entirely with what Mr. Ferguson just said—has been developed in departments over many years, typically for each program. The kind of culture we've had in the past has neglected to build in the retrospective data analysis that would then allow program managers and auditors to seek what they seek, which is assurance.

It's the assurance, then—how did you know that the right things were happening?—that has been missing in our program data. Program data systems have been designed to run programs effectively but not to do that last piece, of providing auditors and program managers with assurance.

That's one of the things we've done with our new contract data tool, which asks the kinds of questions needed for monitoring on a comprehensive basis. We now have it implemented as of September—last month—to provide assurance. It is, though, an enormous task to go back over all of the program data systems, retool them, and add that last piece.

[The Chair, Conservative, Kevin Sorenson]

I'm just going to go to Ms. Lemay. She signified that she wanted to answer.

[Marie Lemay]

I have just one bit to add.

One thing we've done concretely—we were using analytical tools and seeing that the data was not good quality—is actually provide training to people who enter the data, so that they really understand what it's being used for. We saw a dramatic increase in the quality, and the results, and that the queries were appropriate ones. That's one thing that's been done that we're seeing success with and that we're going to continue.

[The Chair, Conservative, Kevin Sorenson]

Mr. Chen, please.

[Shaun Chen, Liberal, Scarborough North, ON]

I want to thank the Auditor General and I want to thank the departments for their work. We have a number of deputy ministers and staff here. They do incredible work across the board through our federal government.

I want to stress the importance of accountability and transparency. If there's one thing that I hear a lot from constituents and from Canadians who email or call my office or speak to me in person, it's that they want to make sure that our public dollars are being treated with the utmost accountability and transparency. There are some concerns I have, but I want to comment first on some of the issues around data.

When I read things in an Auditor General's report, such as the example of having company names stored differently in a system, that is unacceptable. That is the most basic of checks that your system should be able to perform on a routine basis. That is a quick name search that identifies similar names within a company, that looks at the address, or that is able to use other factors to flag an issue when you have multiple entries for the same company or firm.

Ultimately, this leads to severe inaccuracies. I don't have the facts, but that could potentially mean that there is contract splitting, by using different company names. That is a simple check that should be taken care of. I won't even bother asking the question because it's too simple of a problem, in my mind.

I want to go on to the next piece, which is around allegations of fraud. Allegations of fraud need to be taken with the utmost seriousness. When somebody from within the public service, or even from outside, decides to step up and make that allegation, it needs to be treated properly. Based on what the Auditor General has written, there's a lot of policy in place, but it's not clear if the issue is being dealt with properly through documentation and logs.

What has happened since the Auditor General raised these concerns? I know that a number of departments are working through making changes. Can I get an update? Does the Auditor General have a sense of where one of the departments might be making good progress and about whether that could be an example for others?

[Michael Ferguson]

Thank you, Mr. Chair.

No. We haven't done any follow-up work on this issue since then, so I think you're better off hearing from the deputies about what they've done since the audit came out.

[The Chair, Conservative, Kevin Sorenson]

How about we begin with Ms. Laurendeau?

[Hélène Laurendeau]

I have a couple of thoughts on that. I think that the committee is raising a very important point. One of the things that we're seized with in our department is the fact that if we want to have continuity in being able to check whether or not things are being complied with, we need to work at the front end. We need to work on how data is tracked, and we also need to do after-the-fact checking.

To be specific about the tracking of allegations, since the report of the Auditor General, we have modified our allegation logs to make sure that we standardize how we capture the information. This is so that we can do the proper follow-up, to see and track some of the measures that we have put in place. If we have started an audit, a forensic audit, or an investigation of some sort, we can actually track until completion what happened to that allegation. They are recorded in the database, and we are capable of asking, on any given day, where we are on all the allegations that we have received.

We have now also included the status of any recommendation that may be coming out of those various ways of investigating those allegations. I'm also very seized with the fact that whatever comes out of these, even if they're unfounded, sometimes they can reveal the fact that some of our employees may not be properly trained.

We don't stop at saying that the allegation was founded or unfounded. We push it a little further and ask ourselves what we have learned from this. We want to know if there is an awareness that we need to extract in order to feed it back into the front end so that people understand, or so that we can actually correct whatever we see. Even if there is no mismanagement or reason for disciplinary measures, we try to extract the learning from every allegation. We do this to address the issue of transparency but also to correct the training that we're providing so that we mitigate errors as much as possible.

[Shaun Chen, Liberal, Scarborough North, ON]

Thank you.

To Global Affairs, I know that to detect potential fraud you have looked at automated tools to look through your procurement data. What progress have you made in that effort?