Government Operations Committee on June 7th, 2018

[The Vice-Chair, Liberal, Yasmin Ratansi]

Ladies and gentlemen, we have quorum and, therefore, I will start the meeting.

We are studying the Phoenix report. We have with us Michael Ferguson, the Auditor General, with Jean Goulet. From the Treasury Board Secretariat, we have Peter Wallace and Sandra Hassan, and from Public Services and Procurement Canada, we have Marie Lemay and Les Linklater.

As per the request, the Auditor General is going to speak for 10 minutes first, followed by Ms. Lemay and Peter Wallace.

The floor is yours, Mr. Auditor General.

[Michael Ferguson, Auditor General of Canada, Office of the Auditor General]

Thank you.

Madam Chair, thank you for this opportunity to present the results of our audits of building and implementing the Phoenix pay system and of Phoenix pay problems. Joining me at the table is Jean Goulet, the principal who was responsible for both audits.

In 2009, Public Services and Procurement Canada started its transformation of pay administration initiative to transform the way it processed pay for its 290,000 employees. There were two projects: one to centralize pay services for 46 departments and agencies that employed 70% of all federal employees, and the other to replace the 40-year-old pay system used by 101 departments and agencies.

In our audits, we reviewed Public Services and Procurement Canada's development and implementation of the Phoenix pay system and how the department managed the pay problems that the implementation of Phoenix caused.

In our audit of building and implementing the Phoenix pay system, we examined whether the decision to launch the new pay system included considerations of whether the system was fully tested, functional and secure and whether it would protect employees' personal information.

We concluded that the Phoenix project was an incomprehensible failure of project management and project oversight which led to the decision to implement a system that was not ready.

In order to meet budgets and timelines, Public Services and Procurement Canada decided to remove critical pay functions, curtail system tests and forego a pilot implementation of the system.

Phoenix executives ignored obvious signs that the Miramichi pay centre was not ready to handle the volume of pay transactions, that departments and agencies were not ready to migrate to the new system, and that Phoenix itself was not ready to correctly pay federal government employees. When the Phoenix executives briefed the deputy minister of Public Services and Procurement Canada that Phoenix was ready to launch, they did not mention significant problems that they knew about. Finally, the decision to launch Phoenix was not documented.

In our audit of Phoenix pay problems, we examined whether Public Services and Procurement Canada worked with selected departments and agencies to fix Phoenix pay problems so that government employees would receive their correct pay on time. After Phoenix went live in February 2016, the government frequently could not pay public servants accurately or on time. We found that pay problems continued to grow throughout the period of our audit.

A year and a half after the government launched the Phoenix pay system, the number of public servants waiting for a pay request to be processed had reached more than 150,000 in the 46 departments and agencies whose pay services were centralized. Those 150,000 employees were waiting for about 500,000 pay requests to be processed. Those numbers did not include the outstanding pay requests in the 55 departments and agencies whose pay services were not centralized, and also did not include the outstanding requests required by the collective agreements with federal public service unions that were signed in the fall of 2017.

Problems grew to the point that the value of outstanding pay errors totalled more than half a billion dollars at the end of June 2017. This amount consisted of money owed to employees who had been underpaid as well as money owed back to the government by other employees who had been overpaid.

Departments and agencies struggled with Phoenix pay problems from the time the system went live. However, it took Public Services and Procurement Canada four months to recognize that the problems went beyond normal processing levels.

We concluded that Public Services and Procurement Canada did not identify and resolve pay problems in a sustainable way to ensure that public service employees would receive their correct pay on time. In fact, according to the department, a backlog of unresolved pay requests continued to grow after the date of our audit conclusion.

In my opinion, there were three issues that were common to both the Phoenix implementation project and the department's immediate response to the post-implementation problems.

Those were a lack of oversight or governance to guide management activities, a lack of engagement with affected departments, and an underappreciation of the seriousness of the problems.

Before I close, I want to mention that in addition to the two audits I referred to, included in our spring 2018 report of the Auditor General was a message from the Auditor General in which I indicated that I believe that the culture of the federal government enabled Phoenix to become an incomprehensible failure, and I believe the committee should consider how that culture contributed to the failure.

Madam Chair, this concludes my opening remarks. We'd be pleased to answer any questions the committee may have.

Thank you.

[The Vice-Chair, Liberal, Yasmin Ratansi]

Thank you, Mr. Ferguson.

Now we go to Madam Lemay for 10 minutes.

[Marie Lemay, Deputy Minister, Department of Public Works and Government Services]

Thank you, Madam Chair.

I'm joined today by our associate deputy minister, Les Linklater. We appreciate the opportunity to appear before you to discuss the Auditor General's studies of Phoenix. Public Services and Procurement Canada accepts the Auditor General's recommendations and has taken measures to respond to them.

I want to begin by acknowledging that this issue is fundamentally about people. This situation has put a strain on many public servants, for which I am deeply sorry. I know we've said it, but we're sincerely sorry.

This issue is also about those working hard to fix the pay system. I can assure you that we have extraordinarily dedicated teams that are working very hard to address their colleagues' pay issues. Without question, this situation has taken a heavy toll on too many employees.

The Auditor General's spring report examines the planning and development of Phoenix from April 2008 to April 2016. This period predates the senior leaders now working on pay at Public Services and Procurement Canada, but while the decision-makers have changed, the lessons of past actions cannot be forgotten.

There were serious flaws in the planning and implementation of Phoenix and in the broader transformation of the Pay Administration Initiative. We know that cost and timelines were prioritized. As result, poor decisions were made. Critical elements of the project, including some system functionality, testing, change management and training were removed or deferred. These decisions, coupled with the elimination of 700 compensation positions—and the resulting loss of critical capacity and expertise—led to a government-wide transformation that couldn't perform as planned and users who weren't prepared for this significant change.

Within the senior levels of PSPC, and I would say across government, the project's scope and implications were vastly underestimated. The Goss Gilroy Inc. report also cited this fundamental gap as a central failing of the project. When the deadline for implementation came, it was thought that the system was ready to go. Clearly, it was not.

I don't believe anyone acted with ill intent, but the two senior executives responsible for the project who reported to the deputy minister did not properly assess the complexity of the undertaking, the cumulative impact of their decision, or the seriousness of the early warnings.

The performance of these senior executives has been evaluated and appropriate measures have been taken. They are no longer involved in pay administration, and they did not receive performance pay for their work on Phoenix during 2015-16 when the system was finalized and launched.

As I mentioned, this situation has caused significant stress for many employees. Understandably, people want someone to blame. What has happened is unacceptable. The Auditor General has identified that senior executives at PSPC made significant mistakes, but he has also acknowledged that there were oversight and governance issues. I believe that there were multiple points of failure. We have to learn from all of them.

Accountability needs to be coupled with authority for projects like pay transformation to succeed. Public Services and Procurement Canada officials had full authority on Phoenix, but not over the government-wide human resources processes. For example, we should have had an integrated HR and pay approach all along.

We need to do better. We need to give our people the tools and the environment needed to succeed.

Madam Chair, we can't rewrite history, but we can benefit from it. The Auditor General's report provides invaluable advice that is helping us to move forward.

I'd like to turn now to the specific recommendations offered by the Auditor General in his spring report, which focus on future planning of IT projects. PSPC has a long history of managing many large-scale, complex, multi-faceted projects on behalf of clients, such as pension modernization and the Long-Term Vision and Plan for the Parliamentary Precinct. We currently manage over 430 projects with a budget of over $13 billion.

PSPC has rigorous and effective processes in place to manage large-scale projects. Our experience with Phoenix has highlighted the need for an enhanced approach for the management of complex enterprise-wide IT projects.

To fill this gap, PSPC will work with partners to develop a gating tool. This tool will determine if a specific project under our responsibility should be managed through departmental project management practices and processes, or through an enhanced government-wide IT project management approach.

We will also develop an enhanced project management playbook detailing how our government-wide IT projects are to be managed, clearly laying out the roadmap of what has to be done and by whom. It will address governance, and oversight project management principles, roles and responsibilities, and the audit function, among other elements.

These actions will allow us to respond to the Auditor General's recommendation, by ensuring that we have robust governance and oversight for all government-wide IT projects under our responsibility. This approach will also ensure that enterprise-wide IT projects include a stakeholder engagement plan to ensure active collaboration with departments and agencies. This engagement will allow us to better define system requirements and, importantly, validate system performance. These IT projects will also include a formal software upgrade plan and a comprehensive contingency plan.

Finally, PSPC will ensure that internal audits provide the deputy minister with the appropriate assurances regarding project governance, oversight, and management.

Madam Chair, work is under way to develop and formalize both the gating process and the playbook, but I want to assure members that the lessons drawn from Phoenix are already being applied within PSPC. For the here and now, our priority remains helping employees facing pay issues.

In November, we announced a suite of measures to stabilize the pay system and reduce wait times and late transactions for missing pay. These activities were based on the findings of the Goss Gilroy report and aligned with the Auditor General's fall recommendations. Oversight has been strengthened. Departments and agencies can now monitor and act on the quality of their data. Submission of timely and accurate HR information into Phoenix has a major impact on the timeliness and accuracy of pay.

In May, Minister Qualtrough visited the public service pay centre in Miramichi and announced the expanded rollout of a new pay pod approach, an idea that originated with the staff. Pay pods are teams of pay specialists who are assigned to exclusively serve a specific department or agency. A pilot of this approach reduced the case backlog of three pilot departments by 30% overall and their backlog older than 30 days has gone down by 41%.

We are very optimistic about the positive impact of the pay pod approach and we are looking forward to the impact, as it is rolled out across more departments over the next year.

Finally, we have increased capacity and service to employees.

As I mentioned, a critical gap was created when more than 700 compensation positions were cut in 2014. This meant that we implemented Phoenix with only 550 compensation staff. We have since almost tripled this amount.

Recognizing the need to provide more useful support to employees, we have also enhanced our Client Contact Centre and they have direct access to Phoenix as well as other systems. This allows employees to get detailed information about their pay files directly from public servants working at the centre.

We continue to refine and implement improvements and are beginning to see slow but steady progress.

Our most recent dashboard shows that our backlog is down 25,000 transactions from last month and, with minor fluctuations, has steadily declined since January 2018. As pay pods are expanded, we expect this decline in backlogged transactions to pick up speed.

Madam Chair, the ongoing public service pay problems are completely unacceptable and addressing them remains our department's top priority. We are working relentlessly to ensure that public servants are paid correctly and on time.

There were serious flaws in the planning and implementation of the pay transformation. It is clear that there were multiple points of failure. Our responsibility moving forward is to ensure that we learn from all of the mistakes made, so that something like this never happens again.

We look forward to your questions.

[The Vice-Chair, Liberal, Yasmin Ratansi]

Thank you.

We now go to Mr. Wallace for 10 minutes, please.

[Peter Wallace, Secretary of the Treasury Board of Canada, Treasury Board Secretariat]

Thank you so much for the invitation to appear before your committee and help with your important work in reviewing the Auditor General's report on building and implementing Phoenix.

Joining me today is Sandra Hassan from Treasury Board Secretariat, assistant deputy minister of the compensation and labour relations sector.

As Secretary of the Treasury Board, I support the board in its role as the employer of the core public service.

In that capacity, I want to start by saying that the government takes very seriously its responsibility to pay its employees accurately and promptly.

Without question, the pay issues affecting employees and their families are unacceptable, and we are deeply sorry for that. I echo the sentiments expressed by Madam Lemay. We are taking action not only to resolve these issues, but of course to also support those employees who have been affected.

The government has put in place several measures for those affected employees. These measures of course include ensuring that advances are made and paid on a priority basis, providing flexibility in the recovery of overpayments, and establishing effective claims processes to compensate employees for expenses and financial losses occurred because of the unacceptable challenges associated with Phoenix.

In tandem with these support efforts, a huge push is under way, led by Public Services and Procurement Canada, to stabilize the pay system. At the same time—and this of course touches on the key themes raised by the Auditor General last week—we are equally determined to mitigate the risks associated with Phoenix and ensure that we are never confronted by a project failure of this magnitude.

Last year, Treasury Board Secretariat commissioned Goss Gilroy to look at some of the very issues raised by the Auditor General. We see the recommendations of this report, along with those of the Auditor General, as foundational guidance. In its October 2017 response to Goss Gilroy the government said that it is committed to building capacity in such key areas as project management, information and technology management, revamping its approach and policies for project management and procurement, and of course excelling in service delivery for the benefits of Canadians.

On pay transformation specifically, the government has already put in place new interdepartmental governance, both within the senior ranks of the public service and of course through the ministerial working group.

I'll now turn to the Auditor General recommendation for the Treasury Board in this most recent report.

It states that we should, for all information technology projects: carry out independent reviews of the projects' key decisions to proceed or not, and inform the projects' responsible deputy minister and senior executives of the reviews' conclusions.

Without reservation we concur with this vitally important recommendation. We will conduct independent reviews of government-wide projects to identify corrective actions and we will communicate the conclusions and remedial actions required. In the meantime, we are of course strengthening our policies as well as improving governance, changing the role of the chief information officer to further ensure that government-wide IT initiatives are well planned and executed. I echo the sentiments expressed by my colleague moments ago.

Now I will turn to budget 2018 and Phoenix.

In budget 2018, the government made a number of commitments to address the ongoing challenges in the Phoenix pay system. One of these of direct relevance to Treasury Board Secretariat is the provision of $16 million, to be led by Treasury Board Secretariat experts along with our colleagues in public sector unions and technology providers, on a way forward for a new pay system.

In looking at all options, we will have a relentless focus on the user and on a system that links HR actions with pay transactions. This will not happen overnight, nor should it happen overnight, being mindful of the lessons we have learned from Phoenix. We will diligently apply those lessons and ensure that all projects move forward on a constructive, timely, and thoughtful basis.

In closing, let me reiterate that my department, along with senior management throughout the public service, is fully engaged in this file.

Thank you. We look forward to your questions.

[The Vice-Chair, Liberal, Yasmin Ratansi]

Thank you very much.

Committee members, the witnesses will stay until one o'clock, but we would like a little extension after one, so that we can go in camera for two minutes to discuss timing for Monday. Is that okay?

Thank you.

We'll start off the first seven-minute round with Mr. Jowhari, please.

[Majid Jowhari, Liberal, Richmond Hill, ON]

Thank you, Madam Chair.

Thank you to the witnesses.

Before I start my questioning, I'd like to table the following document, in both official languages. It was acquired through the access to information project. I'd like to hand it over to the clerk to distribute to all the members. This has to do with the briefing that was provided to the minister on February 18 vis-à-vis the status of the Phoenix implementation.

To the witnesses, thank you very much for your testimonies. A large business transformation enabled by technology, such as the Phoenix implementation, usually has a very structured implementation approach when it's being led by one of the five large implementation organizations. There have been a number of discussions around the challenges, such as oversight and culture, but I really want to focus on the activities that happened prior to that, what in the terminology of large business transformations is called, “go live”.

By way of background and transparency, I was a management consultant for 25 years. I've implemented a lot of large business transformations enabled by technology, so I'm very familiar with the process that Phoenix went through.

As part of the go/no-go or go-live decisions, traditionally there is a call, or there is a sequence of go/no-go decisions being made. During that process, the readiness assessments of the project from various aspects are reviewed, such as project management, the ability to see the “go live,” as well as the transition and stabilization.

Policies such as security, access to the information, processes—whether the processes are being redesigned—technology such as the functionality of the system working properly, and people, training, and whether the organization restructuring was completed..... There are two other elements that I haven't heard much of. One of them is the implementation of a rollout strategy, and the other one is the contingency plan.

As far as implementation of a rollout strategy is concerned, traditionally, either we have a phased approach, or we have what's called a “big bang” approach.

During a phased approach, a pilot project is launched for a specific department to make sure all the functionality works. From your testimony, it looks like that pilot was skipped, or that it was decided not to proceed with it. Therefore, as an implementation strategy, it was decided that all functionality, all sides, or what's called a “big bang” approach would be taken.

When an organization chooses to go with a big bang strategy or an implementation strategy, the huge risk as far as system implementation is concerned is if something goes wrong. Hence, the concept of a contingency plan, which is basically, “How do we roll back?” and in the case something goes wrong, “What is our recovery strategy?” plays a huge role in that.

During the readiness assessment review that was done, was there ever a recovery strategy or a contingency plan discussed? If it was discussed, what was it?

[Marie Lemay]

I'd like to start by saying that neither I nor Mr. Linklater was there at the time, so I will be answering based on my knowledge, which I gathered either through discussions, through reading, or through colleagues. Before I do, I just want to point out that we often forget that this pay transformation was really two projects. There was the modernization—the IT project—but there was also the consolidation, which was the pay advisers. Those two happened at the same time.

In terms of contingency, to go directly to your question, you're correct from what I gathered also that the pilot project was not done. The decision not to do the pilot project was made considering that there was going to be more testing done closer to wave one. The contingency plan was for the transfer of the data. My understanding is that the biggest risk felt at the time was that when you actually turned the switch on, the data would not transfer from one system to the other, and the paycheques would not go out every two weeks.

There was a contingency plan until after the first wave. If there had been any fatal errors in that transfer, they would not have moved to the second wave. There were no fatal errors, so, by default, the second wave was basically going.

In terms of a contingency plan for what we're facing now, in terms of the compensation advisers and being able to process all of the transactions, it's clear that there was no contingency plan at that level. Frankly, I think part of what we're learning is that the complexity of this was never estimated. It was underestimated. There was no contingency plan for that aspect, so we had to react in real time to increase the capacity of the people to be able to do the transactions.

[Majid Jowhari, Liberal, Richmond Hill, ON]

When I look at this and I look at the presentation, there was a review of the technology, and of the process, and a review of the people. When the switch was turned on, and we went from one system to another system, and then we started experiencing all these challenges, was there really an option considered that was “it's not working now so let's switch back to what we were doing before”, or was it that this was the path that we set ourselves...not ourselves, but that project management had set us on this path of no return and we did not have any other choice but to go live.

[Marie Lemay]

There was a very short contingency plan for the transfer of the data. Immediately after wave one, if there had been problems in the data transfer, then they could have stopped. There were no problems with the data transfer. That's how the project was planned. There was no fatal issue with the data transfer.

I'm thinking of the employees who are listening. That has nothing to do with their pay because it's actually the ongoing pay that we're doing.

The problems started when all of the change requests, the actions, the overtime, and all of those things that we had to do either manually or there was more that we had to do. The capacity was not there to be able—

[The Vice-Chair, Liberal, Yasmin Ratansi]

Thank you, Ms. Lemay.

Mr. McCauley, you have seven minutes.

[Kelly McCauley, Conservative, Edmonton West, AB]

Great. Everyone, welcome back.

Mr. Ferguson, I was watching the testimony with the Senate. We have some witnesses from Queensland, Australia, talking about their issues with their rollout and their payroll. Their Auditor General said they had very similar issues that we had here, as well, but they tackled theirs the third payroll. They said that, after the third pay run, they started to address issues after one month. They reached, what we've been calling here, a steady state after just four months.

Have you looked at how they tackled their problem, as compared with how we tackled ours in Canada, for lessons that we should learn here or things that we overlooked here?

[Michael Ferguson]

We did look at what happened in Queensland and how they resolved it. We really looked at that as reference material to assess what was going on here, both in Public Services and Procurement Canada and in Treasury Board, to determine how the government, here in Canada, was dealing with the issue.

That's what we were doing. We found the same types of things that you said. They recognized the problem very quickly and recognized that they needed to put in place a way of dealing with it. Fundamentally, we said that here, it took much longer to recognize the seriousness of the problem and put in place an all-of-government governance structure to deal with those problems.

I think the Queensland example is a very good reference point for how to deal with those types of issues.

[Kelly McCauley, Conservative, Edmonton West, AB]

In committee, from the very beginning with Phoenix, the Queensland issue came up, I think it was, two a half years ago, brought up by ourselves and our NDP colleague. Do you think that, if we had a chance to learn from them immediately, we could have avoided a lot of the problems we're facing now?

[Michael Ferguson]

I think the largest issue that probably caused the slow-down.... There were a couple of things. Number one was the need to have recognized that this was going to be a problem and that this was going to be a long-term problem, so not something that would be resolved quickly.

The other thing about it was—again, it goes back, very much, to the audit we just released and the decision to launch Phoenix, even though the information was there, indicating that the compensation advisers couldn't handle the volume. I think the biggest problem at that point, once it was understood that the compensation advisers couldn't handle the volume, was how you get more compensation advisers.

That was what needed to be identified early on and how many of them. Then it would take time to get those people, get them trained, and get them understanding the system to be able to bring them back on. I think those were the two things: the slow recognition but also the fact that there just were not enough compensation advisers to deal with the problems.

[Kelly McCauley, Conservative, Edmonton West, AB]

The Australians did mention they jumped on that immediately and flooded workers into the issue, again almost immediately, by their third or fourth pay run.

They also testified that they would not be considering switching that pay system at all. They've had a long stretch fixing it, like we are having with Phoenix.

Do you believe that they're going down the right path, by just stabilizing and fixing what they have, rather than switching to a new system? That leads into what you think of our path that we're looking at here of stabilizing Phoenix, pouring resources into it, but at the same time looking at starting a brand new program or system.

[Michael Ferguson]

It's the government's decision as to what it wants to do in the long term with respect to pay, so regardless of whether it's going to stay with Phoenix or it's going to move to something else, it needs to get Phoenix stabilized. It needs to get Phoenix to the point that it is processing transactions correctly, because, obviously, if you're going to stay with Phoenix, it needs to be able to do the job. If the government decides it wants to change systems at some point, it can't start a new system up with many thousands, or hundreds of thousands, of errors. The system needs to be able to have good information, so one way or another, they have to get that information stabilized. Then there is the longer-term decision, and that's a policy decision of government.

[Kelly McCauley, Conservative, Edmonton West, AB]

You don't have an opinion on it.

[Michael Ferguson]

I'm not allowed to have an opinion on it.

[Kelly McCauley, Conservative, Edmonton West, AB]

Thanks.

Over the course of your research, I'm just wondering if you found any indication that the minister's office was receiving regular and proper briefings. One of the things that came out of the Senate testimony is that IBM commented that—I think it was—in December, they got shut out entirely, weren't invited to do any briefings, weren't invited to offer any information, and back. I think we've heard other testimony that Minister Foote, who preceded Hon. Carla Qualtrough, was not receiving regular updates.

[Michael Ferguson]

Again, our reach does not extend into conversations that happen in ministers' offices. Once we get into that world of cabinet confidence, we can't get there, but I think what I can say very clearly is that our audit indicated that the senior executives of the Phoenix project were controlling all of the information and the flow of information that went up. We saw that they did not fully brief the deputy minister, so I think, by extension, you can surmise that information could not then have flown further from the deputy minister. Again, we have no way of knowing what actually happened in ministers' offices.

[Kelly McCauley, Conservative, Edmonton West, AB]

You've called the program, and actually other programs in your audit, incomprehensible failures and I think most of us in this room will agree, and it's not just over Phoenix. We hear again and again that people were consulted, but a lot of that information did not make it up to the top. I'm just wondering how we can force openness.

It's not just the Phoenix program. We have a lot of other programs, whether it's providing safe drinking water, the shipbuilding project—which is going to be $60 billion to $100 billion—or the fighter jets. How can we ensure, going forward, that we actually have the openness required for the minister to make the proper decisions? We don't want to end up in a situation like this again, where everyone seems to know there's a problem, but it's not getting up to the minister or to the proper decision-makers?

[Michael Ferguson]

I think there are two aspects to it. I think one is procedural, to make sure that people recognize when there is an issue and bring the issue forward. I think the other part of it is cultural, to make sure the culture within government is open to that type of conversation.

[The Vice-Chair, Liberal, Yasmin Ratansi]

Thank you.

We'll go to Mr. Blaikie for seven minutes.