Industry Committee on March 8th, 2011

[The Chair, Conservative, David Sweet]

Order.

Bonjour à tous. Welcome to the 61st meeting of the Standing Committee on Industry, Science and Technology.

We have three esteemed guests with us today: Wayne Smith, chief statistician with Statistics Canada, Ian McKinnon, from the National Statistics Council, and Ivan B. Fellegi, former chief statistician.

We also have with us the Honourable Carolyn Bennett, who is going to begin with opening remarks on her bill, Bill C-568.

I take it you're filling in for another member as well?

[Carolyn Bennett, Liberal, St. Paul's, ON]

Yes, for Mr. Garneau.

[The Chair, Conservative, David Sweet]

We'll begin with your opening remarks, Madam Bennett, and then we'll go to our witnesses. Everybody has been advised that they have five minutes for their remarks.

[Carolyn Bennett, Liberal, St. Paul's, ON]

Thanks very much.

I will pare my remarks in that we have such excellent witnesses that it doesn't seem appropriate for me to be quoting them as they're here in the room.

I think we know that it's been very clear that the replacement of the mandatory long-form census with the national household survey will produce usable and useful data that will meet the needs of many users, as we have seen in the Statistics Canada documents. It will not, however, provide the level of quality that would have been achieved through the mandatory long-form census.

I'm pleased to present this bill that would enshrine the taking of the mandatory long-form census every five years as well as remove the possibility of prison penalties for any of the violations.

We believe there has been some misinformation about the long-form census being added on, when indeed there was only a long-form census before 1971, and at that time they decided to make a short-form census. But before that, all of the information was collected from all of the citizens. In fact, it was the testimony of chief statistician Munir Sheikh that the Conservative Party had misrepresented his advice. I'll quote from Munir Sheikh's statement. He was at the industry committee this past summer. He said:

I want to take this opportunity to comment on a technical statistical issue which has become the subject of media discussion...the question of whether a voluntary survey can become a substitute for a mandatory census. It can not.

As a physician and former minister of state for public health, I also take great advice from people like Dr. David Mowat, the previous deputy chief public health officer for Canada, who stated that the problem with a voluntary census, with trying to elicit this detailed information from a voluntary rather than mandatory census, is that we know, from all of our experiences with a voluntary census and from the experiences of other countries, that certain categories of people will not respond proportionately to a voluntary census survey. In particular, we know that those least willing to provide information voluntarily would be those who tend to belong to socially and economically disadvantaged groups.

We can debate why this is so, but the reality is this: if we go to a voluntary census, the groups whose health and living conditions are most in jeopardy will be the most under-represented in the data.

You will hear from the witnesses, but someone who is not here is Mel Cappe, who stated that for the last 35 years people have been filling out this long-form census in one form or the other, as we have been doing for over 130 years. Now, from 2011 forward, we will not have a data point. This means that all those who filled out the form in the last 35 years did so for naught, because we will not have the next point on the series.

There has never been a case in the history of Statistics Canada where someone's personal census data has been released. All that is released is the aggregation by the census track, so they add them up. Statistics Canada has an unblemished record of keeping to themselves, privately, all of the returns of the census.

In her testimony this summer, Elisapee Sheutiapik, in her response to the questions about how intrusive or coercive the census is, explained that in the north there is a partnership with Statistics Canada, with the well-trained people who can speak Inuktitut, and that the people then willingly fill out the form because it is only by filling out the form that they can find the disparities present, where the average of 12 or 14 people live in one house.

Mr. Chairman, as you know, all the groups have come to the defence of the long-form census, including the very body set up to advise Statistics Canada. I would like also, with your support, to table some of the very interesting aggregations of the federally legislated census requirements of the 2006 long-form census, compared to the proposed national household survey, as prepared by datalibre.ca and Tracey Lauriault. I'm happy to provide that.

In closing, I just want to say that this bill actually speaks to the fact that we want taxpayers' dollars spent wisely. Group after group, all users of the data, all cities and provinces, feel that without the navigation system of a census, we will not know whether taxpayers' dollars are paying for programs that are actually making things better or worse. Turning off the navigation system allows ideologically based governments to just do what they want, and they will not be accountable for the complete waste of money for programs that are not based on the facts.

As we heard at the Assembly of First Nations meeting this summer, the census is the “count” in accountability.

[The Chair, Conservative, David Sweet]

Thank you, Madam Bennett.

Now we'll move to Mr. Smith, who is the chief statistician at Statistics Canada.

Please proceed, sir, for five minutes.

[Wayne Smith, Chief Statistician, Statistics Canada]

Thank you, Mr. Chairman, for inviting Statistics Canada to appear before the committee today.

There are really three points I'd like to make before the committee.

First, whatever is the ultimate will of Parliament with respect to private member's bill C-568, we have reached a point of no return for the 2011 census and national household survey. It is logistically impossible for us to change course now. As you know, the Constitution Act and the Statistics Act require Statistics Canada to conduct a census of population in 2011.

Significant work has been undertaken to integrate collection and other operations for the census and national household survey in order to minimize the survey's cost. The questionnaires required for collection are printed, and introductory letters will return from print in mid-March. The complex computer systems needed to capture and process data have been completed, tested, and locked down.

Recruitment of the large workforce needed to carry out the extensive field operations of the census and national household survey is well under way, having begun in January. Early enumeration for the census and national household survey began in northern and remote communities on February 1 . Early enumeration is necessary in these areas, since a portion of the population migrates to hunting and fishing grounds as soon as winter ends, making it impossible for us to complete their enumeration after that time. We are now well into the implementation phase of early enumeration.

The second point I want to emphasize to the committee is how critical it is that we gain the support of Canadians for the completion of the census and national household survey so that communities, businesses, organizations, and governments will have the data they need for decision-making.

Our efforts are focused on encouraging households to complete the census and national household survey, and we are counting on all levels of government, businesses, members of Parliament, and various organizations to support the census and national household survey by encouraging the people they employ, serve, and represent to respond.

I want to be clear that the success of both the census and the national household survey depends essentially on the participation of Canadians. In early enumeration, we have enjoyed excellent collaboration from Canadians in remote areas. We hope this will continue in southern Canada. The most recent data that I have indicates that we have an 85% response rate on the national household survey and a 99% response rate on the census in the north--so far.

Third and last, I want to inform the committee that Statistics Canada has in place mitigation strategies to address potential risks to data quality for the national household survey. We have not conducted a voluntary survey of this magnitude before, and we will not definitively know about data quality until the survey is over. However, if we can achieve strong and uniform participation rates across the country and in all segments of society, the national household survey can provide data that will meet the needs of many data users--data that will be “useful and usable”, as I first said some time ago.

To mitigate risks, we have, for example, increased the sample rate to one in three households. This will help reduce sampling error for smaller regions and populations. Data on response patterns from the 2006 census and information generated during data collection in 2011 will be used to guide our field follow-up efforts to minimize non-response bias.

Where possible, 2011 census data and other sources of data available to Statistics Canada will be used as auxiliary information in the national household survey estimation procedures to partially offset some of the remaining biases.

I want to assure the committee that Statistics Canada is applying all of its expertise to make the national household survey a success, as is done for all of our survey programs, but I cannot say often enough how important it is that we achieve broad support for completing the census and the national household survey. Success is very much now in the hands of Canadians and Canadian institutions.

Thank you.

[The Chair, Conservative, David Sweet]

Thank you very much, Mr. Smith.

We'll now go to Mr. Fellegi, please, for five minutes.

[Dr. Ivan Fellegi, Former Chief Statistician of Canada, Statistics Canada, As an Individual]

Thank you, Mr. Chair.

My position is well known, but I would like to reiterate it briefly. First, the census is enormously important, and the national household survey, whatever its name, is part of the census.

Why is it so important? Because many of the most significant national issues can only be assessed by it: from the progress of aboriginal people in catching up in education or living standards, to the impact on different groups of people of the 2008-09 financial crisis, the living standards of the elderly, the position of minority languages, the economic integration of recent immigrants, and so on, this is part of our regular national stock-taking. Indeed, some would say it is a critical part of the democratic accountability of governments at all levels.

Second, it is my professional assessment and that of the Statistical Society of Canada, as well as of the American Statistical Association and the French professional statisticians, that a voluntary national stock-taking could be deeply flawed. This cannot be proven scientifically. Indeed, it is conceivable, though hardly credible, that all those who choose not to respond to the survey are exactly like those who choose to respond.

However, in a practice extending over 50 years, I have never seen a study investigating the characteristics of non-respondents that would have concluded they are like the respondents on all the wide range of variables that are collected by the national household survey. I'm certain that the reason my professional colleagues in Canada, the United States, and France have felt compelled to write about our voluntary census is that they have not seen examples of such incredibly fortuitous behaviour either.

My third reason for being here is that most questions of interest are intrinsically relative, that is, they relate to the evolution of different groups over the medium and the long term. Indeed, my earlier examples are all of this character. Whether the aboriginal groups are catching up, whether recent immigrants are doing better or worse than earlier arrivals, whether the situation of minority languages has improved or deteriorated relative to 2006 or 2001, and so on.

If people behave differently as a result of the long-form census being voluntary rather than compulsory, then all these comparisons will be rendered potentially invalid. Since most variables change relatively slowly over time, the impact doesn't even have to be huge for us to be unable to differentiate between real change and illusions created by creeping biases.

Fourth, I'm here because I want to underline once more the fundamental difference between bias and sampling error. If we had a proper random sample and enumerated successfully most of those selected, our error due to having enumerated fewer than 100% of the population can be estimated. By contrast, it is exceedingly rare that bias can be estimated. The significantly lower response rate we can be pretty sure will be present, but we would normally have no idea even of its direction, let alone its magnitude.

That is why I referred to biases as “pernicious” when I testified last time. And that is why increasing the sample size from one in five to one in three solves the minor problem, but leaves the elephant in the room unaddressed.

My last point is to underline that nothing I've said is a criticism of Statistics Canada. I'm convinced they will do everything in their power to have as successful a survey as possible under the circumstances.

They will probably do better than almost anyone could have done under the circumstances, but we cannot expect miracles. They have an incredibly difficult task in front of them, not only to collect the best possible data, but subsequently to provide usable guidance to their hundreds of thousands of users about which data are likely to be relatively safe and which should carry the equivalent of what chemical companies label with crossbones.

With all my heart, I wish for them and for our users that my forebodings should all miraculously turn out to have been utterly wrong.

Thank you.

[The Chair, Conservative, David Sweet]

Thank you, Mr. Fellegi.

Now we'll go on to our last witness, Mr. McKinnon, for five minutes, please.

[Ian McKinnon, Chair, National Statistics Council]

Thank you, Mr. Chair.

My name is Ian McKinnon. I chair the National Statistics Council.

The council is the body of volunteer external advisers from across Canada appointed to advise on matters affecting Canada's statistical system. Since the announcement that the 2011 census would contain only the basic questions from the so-called short form census, later supplemented with the addition of questions on language, and that the voluntary national household survey would replace the long-form census, the council has continued to express its concern over the effect of these changes on the Canadian statistical system.

What's at risk?

First, the voluntary national household survey will suffer from self-selection bias, a flaw that cannot be corrected for without having solid benchmark data that the census in its entirety has traditionally provided for the Canadian statistical system. The changes will also likely result in Statistics Canada's not being able to publish as robust or as detailed small-area data for neighbourhoods, towns, or rural areas. Much of the analytic work done by municipalities, private firms, health agencies, and highway and transportation planners, for example, will be affected.

Our second concern is the potential loss of vital benchmark information. The mandatory long form meant that Statistics Canada has had an accurate benchmark for the demographics of populations who are difficult to reach or who are less likely to complete a voluntary survey. This, in turn, means that sampling and weighting strategies for subsequent voluntary surveys can compensate for the bias from differential response rates and produce more reliable information.

The importance of having census benchmarks available is readily apparent when one considers some of the populations who we know are more difficult to reach: young people making the school-to-work transition, urban aboriginal populations, the very affluent, and new immigrants are just some of the examples.

What, then, is to be done?

It is, as my colleagues have said already, far too late to change the manner in which the 2011 census will be conducted. Statistics Canada and everyone involved with the census will work extraordinarily hard to ensure that the information will be collected in a manner that is as useful as possible under the constraints imposed by having to use a voluntary survey to collect the long-form data. I would encourage all Canadians to support StatsCan by completing the census and the national household survey.

On the other hand, the underlying issues remain. Canada's statistical system will not be able to provide the detail and quality of data that users have had access to in the past. This issue will become larger over time as it becomes less useful to use the 2006 census results as a benchmark for subsequent surveys and extrapolations.

Looking to the future, having much of what has traditionally been census data collected in 2011 through a voluntary survey will create a significant discontinuity or break in the century-long census data series. Paradoxically, this actually makes it more appealing to re-examine the ways in which Canada should collect its fundamental information about its population.

If we draw back and look at what other advanced industrial democracies have done or are doing, there are two families of approaches. On one side, we have mandatory censuses, and on the other, we have population registers linked to extensive administrative databases. In addition, other countries trying various approaches have conducted large experiments, but no approach has emerged that provides high-quality data without either a mandatory survey or very extensive data linkage.

Given this situation, the Canadian statistical system would benefit from two things: first, a thorough evaluation covering costs, as well as issues like privacy and intrusiveness, and looking at data quality of all the ways to gather census-like information; second, a mechanism that will allow for the continued collection of the robust and detailed data that Canadians need while this broader evaluation and debate can take place.

Given the importance of this task, I thank you for your attention to this topic and for the work you do.

[The Chair, Conservative, David Sweet]

Thank you very much, Mr. McKinnon.

Thank you to all the witnesses.

Now we move on to our questions in rotation.

We'll go with the Liberal Party and Mr. McTeague for seven minutes.

[Dan McTeague, Liberal, Pickering—Scarborough East, ON]

Thank you, Chair.

Witnesses, thank you.

Thanks to my colleague, Carolyn Bennett, for driving this through Parliament. The will of Parliament is very different from the wishes of the government. For obvious reasons, the questions reflect very seriously the concern we have that the definition of who we are as a country is very much going to be skewed.

Mr. Smith, you find yourself in the unenviable task and position of having to.... I'm looking at a national household survey data quality report--which I pulled off this morning--indicating that it contains all the questions that StatsCan contemplated for inclusion in the 2011 long census form. NHS is therefore identical in content to what would have been collected in the 2011 long-form census.

In your view, why would we not simply continue with what we have and simply change the word “mandatory”? How difficult would that be? I appreciate the fact that some are saying we can't unscramble eggs, or whatever definition you want to put in there, but the reality is to simply change the top page and leave the census as it is, but in a mandatory form, particularly the long-form census. What would be the difficulty?

[Wayne Smith]

I think you have to imagine the scale of the operation. The census questionnaires began printing in August 2010—I think it was actually August 9. We are printing huge numbers of forms. We've started the assembly of those forms. We've built systems that have to be tested for high volumes and then locked down in order to support the Internet response, and also the—

[Dan McTeague, Liberal, Pickering—Scarborough East, ON]

Do they actually say “voluntary” on them? In the main frame of those questions, is it indicated that they have to be voluntary versus mandatory?

[Wayne Smith]

Well, we now have two completely separate operations. On the census form, it says that it is mandatory. On the national household survey and the accompanying letter, it says that it is voluntary.

[Dan McTeague, Liberal, Pickering—Scarborough East, ON]

Can you change that letter, sir, to say “mandatory”?

[Wayne Smith]

We could have maybe at the beginning of this month, but at this point, it is no longer possible. We simply don't have the time to make it happen by—

[Dan McTeague, Liberal, Pickering—Scarborough East, ON]

Okay. We only have a few minutes given to us here and I may want to share my time with my colleague, Mr. Rota.

You also state that the question of data quality...and I think this is coming from your own department, admittedly. The statement reads here.... Was it doomed to fail or an attempt to make a silk purse out of a pig's ear by saying the following? It states:

In its initial planning, Statistics Canada assumed a response rate for a mandatory 2011 Census long-form of 94%, identical to that achieved for the 2006 Census. Statistics Canada has assumed a response rate of 50% for the voluntary National Household Survey.

Short of the miracle that I think Mr. Fellegi has referred to, is it fair to say that what you're going to wind up with, at $30 million, is nothing less, sir, than a colossal waste of public funds, time, and reliability in terms of accuracy of information down the road?

[Wayne Smith]

On the contrary, we can in fact succeed. The national household survey will produce data that will meet many users' needs. On the effect of the declining response rate, there are really two aspects to what you need to take into consideration. One is that if there were no compensating change in the size of the sample, the sampling error would increase, and that would degrade the quality of the data, strictly from an accuracy point of view.

The sample size has actually been increased to compensate for that, so--

[Dan McTeague, Liberal, Pickering—Scarborough East, ON]

Assuming, of course, that you have people responding--

[Wayne Smith]

At a response rate around 60%, it would actually be a wash between the two.

[Dan McTeague, Liberal, Pickering—Scarborough East, ON]

Sixty per cent might be the best...?

[Wayne Smith]

If we were to achieve a 60% response rate, basically we would have the same size of responding samples, on average, across the country, that we would have had from the--

[Dan McTeague, Liberal, Pickering—Scarborough East, ON]

[Inaudible--Editor]...anticipate to fix the bias with a bigger sample. Is that what you're suggesting?

[Wayne Smith]

No. I said there were two factors, Mr. Chair.

The first factor, as I said, was the impact of a declining response rate. The first consideration is the impact on the accuracy of the data through sampling error. I said we've taken measures to correct that. The second issue is the issue of response bias. We've heard a great deal about that. The reality is that you cannot say, concretely and absolutely, that the decline in the response rate will result in a problematic degree of non-response bias. You can't scientifically say that.