Public Accounts committee  The management accountability framework was established in 2003 in order to assess the department's management results overall, over a broad range of areas of management. Evaluation is one of currently 19 areas of management that we assess. For example, we encourage departments to assess their capacity in terms of audit, in terms of financial management, and in terms of people management.

Public Accounts committee  Yes, Mr. Chairman. The strategic review for the Treasury Board Secretariat is taking place this year.

Public Accounts committee  We acted and we took note of the report's conclusions, but we also took note of the factors we identified. In November 2009, we provided departments with guides on performance measures. We will be finalizing that this year. Therefore, yes, we took note of the recommendations and we took measures in order to ensure that data collection complied with the standards and met the requirements.

Public Accounts committee  We have given them four years for this transition. Give us at least enough time to get halfway through that transition period. At that time we'll be able to assess whether or not adjustments need to be made. For now let's give them a chance to make some progress. Perhaps Mr. Shugart wants to respond to your second question.

Public Accounts committee  Thank you Mr. Chairman. With respect to your first question on whether or not departments work together, the answer is yes. In June 2009, we established a practices community. We have regular meetings where we share information on best practices. We have also had several discussions on key skills.

Public Accounts committee  It is by the end of that period.

Public Accounts committee  Thank you, Mr. Chair. We did recognize that gap, and we are working on many fronts to fill it. One is the work we are doing to recruit. We have also set guidelines on what the core competencies are. There has been some increased funding in departments. We've seen a growth in the number of evaluators available in the government.

Public Accounts committee  The new policy was put in place in April 2009. The five years will end in 2013-14, for the transition period. The formal period, when there will be the requirement to cover all of the evaluation in the five years, is from 2013-14 to 2017-18. That is when the obligation to cover 100% of direct program expenditures kicks in.

Public Accounts committee  Essentially, an effectiveness evaluation looks at the cost, the effectiveness, the results, and can occasionally look at the process around the delivery. It looks at the effectiveness and efficiencies around the program and the initiative. If you're setting up an initiative or program to achieve a certain set of results, it really is critical to set out what your performance measures will be—which is now clearly the responsibility of the program managers—for us to be able to assess whether the program and the initiative are achieving the intended results.

Public Accounts committee  One of the issues was coverage. Another was the nature. Another was the ability to tailor the type of evaluation to the type of program. Another was clarity around the collection of data, or performance measurement. We also addressed the issue of governance, the independence, if I could put it that way, of the evaluation function as well as the responsibilities of the deputy head in relation to the evaluation function.

Public Accounts committee  I would say that it did. It led us to undertake a fairly comprehensive review of the evaluation policy that we renewed and redid. It caused us to look at the performance measurement information framework. It led us to have a fairly extensive series of conversations with deputy ministers about how they used evaluation.

Public Accounts committee  We were doing this in the same timeframe, and we were encouraging departments to make improvements.

Public Accounts committee  Yes.

Public Accounts committee  Mr. Chairman, it is one of the fundamental factors in our ability to undertake evaluations. My colleagues have examined these issues, especially in the area of the environment, and they might like to expand on their approach.

Public Accounts committee  Mr. Chairman, perhaps I'll begin by answering and then I'll ask my colleagues to expand on their own programs and activities. First, with respect to data, we recommended that departments work with their managers. This was for the establishment or renewal of programs, in order to identify performance measurements unique to those programs.