Public Accounts Committee on Oct. 29th, 2018

[Pat Kelly]

But that goes to the whole purpose and point of the program to help people become self-sufficient members of the labour force and to have the skills, the training and the experience that one would need to be able to continue to hold a job.

You didn't get a reason—

[Glenn Wheeler]

Regarding the CRA data? No.

[Pat Kelly]

Okay.

I want to return to that piece and make sure I understand it correctly. The department did not know whether more than 20% of all clients who received services found a job or went back to school. That means that for four out of five people who participated in the program, the department didn't know what happened to them after the participation in the program finished.

[Glenn Wheeler]

Mr. Chair, no, it was a situation that for 80% they did know the end result; it was for 20% that they didn't know the end result.

[Pat Kelly]

All right. Thank you for clarifying that.

Overall, Mr. Ferguson, some reports—and it actually might not have been in this one—have spoken of how the metric for success, if that's even the word for it, has been in too many cases simply how much money was spent, as opposed to the outcome. Are there some broad concerns about simply judging the action of the government based on simply just money expended rather than outcomes achieved?

[Michael Ferguson]

This obviously is a very important program that the department runs. What we focused on was that the objective of this program was to help indigenous people find sustainable and meaningful employment. We feel that if a program has an objective like that—“sustainable and meaningful”—then there needs to be a way to measure whether the people who participated in the program ended up getting sustainable and meaningful employment. That's why we raise concerns about saying that.... If somebody ended up with a job but it was maybe a part-time job and maybe it didn't last very long, is that really sustainable and meaningful?

The starting point to understand the success of the program is to go back and look at those terms. If that's what the program's intended to achieve, sustainable and meaningful employment, what is sustainable employment, what is meaningful employment, and then how do we track it all the way through?

That's our fundamental concern here. There's a critical objective to this program, and it's really important that the department be able to measure it. Again, if you look at the overall numbers and you see the unemployment levels are higher for this group than the rest of the population, and those numbers don't change, then you wonder whether the program is actually having an impact or not. Without having some measurement for sustainable and meaningful employment, it's not really possible to say what type of an impact the program is having.

[The Chair, Conservative, Kevin Sorenson]

Thank you very much, Mr. Ferguson.

Mr. Christopherson is next.

[David Christopherson, NDP, Hamilton Centre, ON]

Thank you, Chair.

I have to say that when I found out that the next report, at our next hearing, was going to be on indigenous people, I thought, “Please be a good report. Please”, because I'm out of options. I've already gone ballistic. What do you do after that? Now...it's just so sad.

Here's my concern. It would have been a lot easier, as a member of the premier oversight committee of Parliament, to understand if it were one department that just continuously couldn't get its act together. I took some comfort in thinking it was a problem child, it was a challenging ministry, and that we were just not getting it right there.

However, now we have a completely different ministry with the same kinds of results. At some point you start thinking the unthinkable. If it's not the department, is it the subject matter? When you are immersed in enough of this, you begin to understand some of the frustration that exists in the indigenous community, why Romeo did what he did, and why an honourable man like him would say what he said in that place.

The Auditor General, in his spring report and his message to us, said it was “the incomprehensible failure of the federal government”—and that's not just this one, but all of them—“to influence better conditions for Indigenous people in Canada. Our recent audits are two more in a long line that bring to light the poor outcomes of Indigenous programs.” Here's another one in the same year.

We can go only so long believing that these kinds of things can build up and build up and that there's not going to be a reaction at some point. I said before that if I were a young indigenous person faced with the history of what has happened to my people over all these decades, let alone the treaties and everything else, given the way I am, I'm thinking I can give a good guess where I'd be on this subject. How long are you going to keep me contained and quiet when this kind of stuff is still going on?

I realize I'm going on, but I don't know what else to do. We ask detailed questions. We get angry. We plead. We think it's one department, and still we come back to the Auditor General's...and I think he phrased it so well, “the incomprehensible failure of the federal government to influence better conditions for Indigenous people in Canada.”

We go to this report. I like to look at the focus of the report, as stated in paragraph 6.12 on page 3:

This audit focused on whether Employment and Social Development Canada managed the Aboriginal Skills and Employment Training Strategy and the Skills and Partnership Fund to increase the number of Indigenous people getting jobs and staying employed.

Right above that, the Auditor General says, “The Department is responsible for monitoring the agreement holders' use of program funds,” and yet, in his opening remarks, the Auditor General also had to say “the Department did not consistently monitor Indigenous organizations to ensure that they fulfilled their obligations under funding agreements”, blah, blah, blah.

The key thing that was supposed to happen in this program was monitoring, and you failed. Once again, why? Why, why, why do we have consistent failure when it comes to our indigenous sisters and brothers? Why?

[Graham Flack]

I'll focus on the monitoring question, if that's where you wanted me to start, but I'm happy to go wherever you want.

In the Auditor General's report in paragraph 6.83, they lay out the statistics on how the department did on the timeliness of its financial and activity monitoring.

You can see that monitoring on the core program we're talking about has improved over time and is now at 95%. On the skills and partnership fund, it is only at 50%. That's a much smaller number of agreements and larger agreements.

What I can describe to you is, I think, part of the challenge in how we've done monitoring and how we're going to change it going forward that I think will allow us to get better results.

If you take organizations like the Saskatchewan Indian Training Assessment Group that we've been working with for almost 30 years, you find they have superb data and results, and yet we are monitoring each of these organizations, including that one, at the same level of assumption in terms of their capabilities. We're not using a risk-based approach to monitoring.

In the tax system, the normal way we will do a risk-based approach to monitoring is that if you are a sophisticated taxpayer with a track record of performance, we will not do the intense monitoring for every single year for every single thing. The plan is to take a risk-based approach to monitoring. For those with strong track records and demonstrated track records, we will do less frequent monitoring on an adjusted basis, and that will allow us to focus more resources on those organizations that have lower capacity, because, in many cases, the monitoring ability also reflects the ability of those organizations to give us the data because they have had staff turnover and other issues. Although we've made improvements on this, to make this sustainable over time we're going to take this risk-based approach to monitoring that will give us more resources to focus on the areas where we've had less monitoring.

Does that...?

[David Christopherson, NDP, Hamilton Centre, ON]

It's an answer.

I hear the answer and I hear the sincerity. I've also sat here enough times when it just doesn't happen.

Let me ask a very specific, straight-up question.

On page 5, in paragraph 6.24, it says:

The Department developed a performance measurement strategy for each of the programs to measure and report on results. However, we found that the Department did not fully implement....

This is just as an example, and I'm quoting:

For example, the Department said it would survey agreement holders annually to assess whether it was adequately supporting them to deliver services under the Strategy. It also said it would prepare internal reports annually to inform senior management about the performance of key aspects of the Strategy, such as efforts to help increase the capacity of agreement holders. However, the Department did not fulfill either of these commitments.

You can understand my skepticism when you make promises and they aren't kept. Why were two obvious promises not kept?

[Graham Flack]

I'm going to need some help on this one, but before I go to that, I understand your frustration and I want to assure you that as public servants, we join to make a difference. We are committed to doing that. As a public servant who happens to have Perry Bellegarde as his neighbour, I have an added incentive, because he explains to me on a regular basis, along with Val, where we aren't holding up our bargain.

Part of the change that we've made going forward is rather than our determining all of the things we need to collect and what we need to do, Rachel has led a co-development process with the indigenous partners to identify for them what data is useful. Part of the challenge is that there have been complaints about the administrative burden of some of the collection we're doing, and you've seen this in the other part of the Auditor General's report. Some of the communities don't feel this is necessary for the things they're dealing with.

In the new performance management framework that we'll have in place in April, one of the differences is that it has been co-developed so that the partners on both sides see the validity in the pieces of data we're collecting and commit to be able to collect that data. That's one way I think it's going to make a difference in terms of the outcomes, because this isn't something that has been just opposed.

Rachel or Leslie, do you have specific information on that paragraph?

I apologize; in my studies I did not....

[The Chair, Conservative, Kevin Sorenson]

Perhaps you can give a very quick answer. We're at eight minutes already; we're a minute and a half over.

[Rachel Wernick, Senior Assistant Deputy Minister, Skills and Employment Branch, Department of Employment and Social Development]

The performance measurement strategy was focused on quick returns to work and school. What we heard from our indigenous partners is that it wasn't really tailored to the multi-barrier clients with whom they deal.

Fundamentally, most of their clients need several interventions and a lot of emphasis on skills development, starting, as we mentioned earlier, with basic skills and moving through employability skills to get to the point where they're ready for employment or to go to higher education.

There was a fundamental flaw in the way that we were focusing on returns to work and school, because it wasn't allowing our partners to measure progress and real results in improved skills development for their clients.

What we're doing in the new approach is distinction-based and better tailored to clients, because an Inuit youth in the north does not have the same challenges as a Métis urban youth. It's distinction-based, but we're putting in new indicators that will better help us measure the real results that are happening. There are real results happening on the ground. Our indigenous partners develop and design these programs, and they are looking for things that will better capture gained work experience, gained skills, and that people are moving up to where they need to be to access that sustainable employment.

[The Chair, Conservative, Kevin Sorenson]

Thank you very much. We're at 10 minutes. We now go to Ms. Yip for seven minutes.

[Jean Yip, Liberal, Scarborough—Agincourt, ON]

Thank you for coming. Just to follow-up on that, will indigenous people be consulted again before the final strategy is implemented, as a last check?

[Rachel Wernick]

Yes. It's a co-developed strategy that the indigenous partners will be agreeing to. We are working at all levels. We also meet with indigenous leadership, and we work through the government's permanent bilateral mechanisms to deal with indigenous leadership. It will be a co-developed and agreed-to approach.

[Graham Flack]

Just to give a concrete example of that, what that means on a distinction-basis approach is that our Inuit partners may say, for example, “We wish to focus our energies further upstream”, which means we need a different set of measures than just employment, because there are going to be multiple steps, given the multiple barriers those upstream clients are going to face before they can get to employment. We'll need measures for those steps. We may have another community that wants to focus for the folks who are closer to employment. For them, the measures we've been using around employment—with the added elements that the Auditor General has rightly stressed, namely durability and sustainability of that employment—would be more appropriate to have initially.

The measures will not be uniform across indigenous organizations. There will be some tailoring to cover the specific goals they are trying to achieve. That will differ based on the clientele they're focusing on.

[Jean Yip, Liberal, Scarborough—Agincourt, ON]

I have a question regarding the recommendation in paragraph 6.73 on labour market information. The department indicated it will further improve available labour market information through an extended survey pilot beginning in April 2018. The department states that they “recognize the importance of providing timely, detailed labour market information.” However, this information, as the OAG has stated, will not be available until 2022. How can a new strategy be developed if it's not going to be available until 2022?

[Graham Flack]

I've spent a lot of time digging into this, and as an economist I understand the limitations of statistical collection at a micro level. Let me just walk you through this.

What we want is comprehensive, accurate, timely and cost-effective data. At a micro community level, whether it's a small town or a reserve, this data does not currently exist in Canada, because the labour force survey is not a sufficiently granular tool for statisticians to be able to accurately predict, based on the surveys they do, what the micro labour market conditions are.

This data is not just absent for indigenous communities. If you ask the Government of Canada for the micro labour market data on the community of Canso, we are not able to give you assurance around that because the statistical collection tools do not allow us to have comprehensive, accurate and timely data that will do that.

We have embarked—this was based on funding that came through budget 2015—on a groundbreaking strategy, working with indigenous partners to try to collect that data at the micro level in an accurate way. Not only is that going to take time, but I want to be clear with the committee members that we are on ground where Statistics Canada would say it is going to be very difficult to achieve success.

Why would I say that? A survey will be the dominant vehicle that we use to collect data. If, in a large national survey, you have a low response rate, you can statistically adjust those results to equalize for regional data. If you're at a small community level and 30% of people don't fill out the survey, those results will not be comprehensive or accurate, because you're not able to project based on that data rate.

Another challenge we have is the frequent movement of individuals on and off reserves, which quickly changes the accuracy of the data. In this pilot, we're going to do annual collection—not monthly, which would be much more expensive—so the accuracy of that data will be a challenge. Statistics Canada, with the most expertise in the country in this area, has cautioned us that this is a very challenging endeavour we're undertaking.

The other feature of the endeavour, as you indicated earlier, is that we're doing it with indigenous partners and having them train the individuals to collect the data, because they believe they will get higher response rates if they do that.

I guess we're highlighting that we would love to have this data. No such data exists in Canada for any community. We are trying a pilot that will work through this, but there are real questions around whether at the end of this pilot we will be able to demonstrate that in a timely, cost-effective way, we can get comprehensive and accurate labour market data at the micro community level, because we've never done that before.

We are putting all of our efforts into trying to do this, starting with four communities and expanding to 44, but we have not, with the statistical experts in government and outside, been able to find shortcuts to get this type of data, which does not exist.

The Auditor General is absolutely right to point out that this absence of data in all small communities is a real gap, because if those communities are trying to predict workforce needs, having that data would be important. Again, I would point out that the data does not exist in any small community anywhere in the country. The statistical tools of the labour force survey are not able to give you that reliable data at the micro level.

This is how we're trying to take the approach, but it is going to take time. I want to be clear with the committee that there is no guarantee we're going to succeed.

[Jean Yip, Liberal, Scarborough—Agincourt, ON]

How much time do I have?

[The Chair, Conservative, Kevin Sorenson]

You have a minute left, Ms. Yip.

[Jean Yip, Liberal, Scarborough—Agincourt, ON]

Okay.

What is the status of the labour market programming overlap among the federal and territorial supports? Are these overlaps being immediately dealt with?

[Graham Flack]

The first point I would make, as somebody who's done a lot of federal-provincial programming over his career, is that overlap and duplication between federal and provincial programs are not a bug of federalism. It's actually a design feature. When you have two jurisdictions spanning those areas and they are continually adapting their programs, the risk of overlap is always going to be there.

We have mechanisms at the multilateral level with all of the provinces and territories, and at the bilateral level, to try to continually deconflict so that we can always make those programs align in a way that doesn't overlap, but this isn't going to be a situation of doing a study once and then everyone locks their programs in place and it doesn't move. They continually adapt, and we have to do that.

As an example, programs are increasingly introducing innovation elements to their programming and piloting different things at a community level. For that, you can't just deconflict at the program level; you have to look at the community A program officer in Nova Scotia might be trying something different, but it is similar to something we're doing. We have to get those officers on the ground talking to each other.

I just want to highlight that we do have a process to do this, but it's going to have to be an ongoing process. When the Auditor General pointed out that we said in our 2016 risk profile that one of the risks was overlap and duplication, that will always be a risk in federal-provincial programming.

[The Chair, Conservative, Kevin Sorenson]

Thank you, Mr. Flack.

I want to go back to Ms. Yip's question for a moment—not the last question but the one before, when you talked about the pilot projects you had.

From my perspective, it doesn't sound as if you have a lot of confidence in the process you're going through. You state that you're trying the pilot project, but there is no guarantee the data.... When I hear deputy ministers, it's almost as if they're waving the white flag of surrender. Now maybe I'm misinterpreting that, but it seems to me that you don't have a lot of confidence. You've been in the position for three weeks. Has this been designed by somebody else, and you aren't willing to sign on to it? You sound very hesitant.

[Graham Flack]

No. I have met with the folks who've done the design. We had a half-day session with Statistics Canada last week as well—not on this specifically, but on other elements. I'm confident that our best statistical people have looked at this and come up with the best approach to develop this groundbreaking data. I'm confident that the approach that has been developed has the highest prospect of success in something we have never successfully done in the country, which is collect micro community, accurate, timely, labour force data.

My cautions are that we are also going in aware that issues such as movement in and out of communities can create statistical challenges for us. We are trying to design it with communities in ways that will overcome this problem. For example, we think having indigenous delivery of the survey will increase the survey take-up, as opposed to having Statistics Canada or the department doing it.