Evidence of meeting #2 for Science and Research in the 45th Parliament, 1st session. (The original version is on Parliament’s site, as are the minutes.) The winning word was criteria.

A recording is available from Parliament.

On the agenda

Members speaking

Before the committee

Ljubicic  Professor, McMaster University, As an Individual
Pinker  Johnstone Family Professor of Psychology, Harvard University, As an Individual
Shariff  Professor, The University of British Columbia, As an Individual
Cobey  Scientist, University of Ottawa Heart Institute, As an Individual
Karram  Assistant Professor of Higher Education and Coordinator, Higher Education Graduate Program, University of Toronto, As an Individual
Larivière  Professor, Université de Montréal, As an Individual

Helena Jaczek Liberal Markham—Stouffville, ON

Thank you so much, Madam Chair.

Thank you to our witnesses.

The purpose of this committee is to make recommendations, as all three of you have done, on the type of evaluation criteria to be used for the allocation of federal funding through the tri-agencies. I was one of the members of this committee who was here in the previous Parliament when we studied this question. We heard a number of ideas.

Dr. Cobey, you mentioned the “publish or perish” imperative. One of the ideas we heard was that perhaps applications should be completely blinded as to who the proponent is. In other words, that potential bias would be removed, and the evaluation would be done simply on the quality of the proposal. That would perhaps be part of a stepwise review of the application. In other words, once various proposals were considered excellent, they could then proceed to more evaluation of the team. Since we're so interested in the whole EDI evaluation criteria, perhaps that could be part of the second step. It might relate to the training of researchers, etc.

I'd like some of your comments to see how, in a very practical way, you could remove some of the potential biases that have existed institutionally for a long time.

12:40 p.m.

Scientist, University of Ottawa Heart Institute, As an Individual

Kelly Cobey

Through you, Madam Chair, thank you for the question.

I think you point at issues or shifts in the potential way peer review is done at grant panels. I would agree with your suggestion. I think having blinded peer review at these committees could help address some of these issues, and then selecting for excellence in the second phase where perhaps it's not blinded.

I would encourage the committee to consider making peer review more open, generally speaking. While we may have blinded peer review initially, at the end phase, once selection is done, I think it would be of extreme value to open up the black box that is the peer review process in this country for federal funding and make those peer review reports as available and as open as possible. Sometimes there are trade secrets or things that need to be closed, but to me, in order to improve the system, we need to know how the system is working and we need to do active research, or metaresearch, on peer review to improve it. We don't want to go from one system that's clearly not working to another system that we think might be working better without actually having the evidence. As a researcher, I think we need research, metaresearch, to show that the goals for how we'd like to change peer review and select for excellence are actually being changed and achieved. Right now, across the board, I would say there's very little implementation in monitoring our policies and practices.

Helena Jaczek Liberal Markham—Stouffville, ON

Thank you for that.

Since you've raised yet again the whole issue of grant monitoring, how do you see that happening? Would the original peer review assessment team have a role in that? Would there be a time frame, with a sort of “let's take a look at where they're at”? It seems like a lot of extra work. Could you try to convince me that it would be really useful to have that piece?

12:40 p.m.

Scientist, University of Ottawa Heart Institute, As an Individual

Kelly Cobey

I will, absolutely.

Right now in Canada, for the vast majority of grants, once you get funding, there's essentially no monitoring until your final report. In other jurisdictions, there are grants officers assigned to funded projects to ensure that certain milestones are met and that overall outputs are delivered.

I'll use, from my area of expertise, the concept of open science. For instance, with clinical trials, we have federal policy to ensure that these trials are registered prospectively in an appropriate registry. We know that we're not doing that for metaresearch.

We have a policy and we need to monitor, when we do fund a trial, that those trials are indeed getting registered and that the results are subsequently being reported fully and completely. We know from an audit we've done that about half of the trials conducted in this country never see the light of day in terms of having their results reported in a public registry or even in a peer-reviewed publication. That suggests inefficiency. We want to make sure that there's monitoring to make sure that some of these basic science policies that we have—our science policies are quite strong and getting stronger—are being implemented on the ground.

Helena Jaczek Liberal Markham—Stouffville, ON

Thank you.

Dr. Karram wanted to give us a little bit more on DEI, so perhaps the remaining time would be an opportunity for her to contribute.

12:45 p.m.

Assistant Professor of Higher Education and Coordinator, Higher Education Graduate Program, University of Toronto, As an Individual

Grace Karram

Thank you so much.

We see this polarization that assumes there's a woke community on one side and then others who are feeling shut down on the other side. We really have to nuance that, because if we're looking at trying to change our labour market inefficiencies, we have to remember that we have a large number of recent Canadians and groups of students who are in graduate school who are soon to be our main researchers and our early career researchers, and they come from a rich wealth of communities that have cultural and religious heritages. They also see themselves as apart from the woke community, but they're not the top-tier researchers who are saying we have to just pick based on merit. I hope I can begin to point out the differences between these groups and say that if we include EDI in an effective way, we will actually bring all of those new researchers into the Canadian landscape of research production, making sure that we have the best talent from around the world that has chosen to live in Canada to do research. We cannot throw EDI out in this myth that we're not choosing the best research because of the groups of recent Canadians. They are highly educated and need to get into our labour market.

The Chair Liberal Salma Zahid

The time is up for MP Jaczek.

We will now proceed to MP Blanchette-Joncas.

Please go ahead. You have six minutes.

Maxime Blanchette-Joncas Bloc Rimouski—La Matapédia, QC

Thank you, Madam Chair.

Thank you to the witnesses for being here.

Professor Larivière, you have shown in your work that the group of Canada's 15 major research universities, or U15 Canada, has received about 80% of research funding in Canada over the past 20 years. Out of that group, five universities in particular receive nearly half of that amount.

Does this federal allocation criterion really promote excellence, or does it preserve a concentration that perpetuates institutional prestige without improving scientific production?

12:45 p.m.

Professor, Université de Montréal, As an Individual

Vincent Larivière

It's obviously a complex issue, and it's hard to find cause and effect relationships in all of this.

However, there's a well-known phenomenon in the sociology of science called the Matthew effect. Basically, the scientists or the institutions with the biggest amount of symbolic capital and prestige will receive even more, regardless of the intrinsic quality of it all. If two scientists discover the same thing at the same time, the discovery will most likely be attributed to the one who already has an enormous amount of capital. We know this to be one of the natural effects, say, of the scientific system, that is, giving more to those who already have it.

If future funding is based on past funding, that obviously leads to the concentration of funds, largely in the hands of researchers affiliated with U15 Canada.

Maxime Blanchette-Joncas Bloc Rimouski—La Matapédia, QC

According to your data, the more funding a university receives, the more the average cost per paper goes up, even with the impact taken into account.

Doesn't that prove that the current federal criteria, which amass funding, undermine excellence by reducing the scientific effectiveness per dollar invested?

12:45 p.m.

Professor, Université de Montréal, As an Individual

Vincent Larivière

Yes, that's what the data shows: The cost per paper is higher in large universities. You still have to question the data to be a good scientist, so we're trying to find the causes of all of that. I can't say that that's what it proves, but the data shows that it definitely costs more.

To put it charitably, we know that the big universities work extensively with the small ones. In all likelihood, it could be said that there are ripple effects on the smallest universities. More research is needed on the matter. It goes back to what my colleague said: We need data on the research system.

What we want to do in our labs is to find a way to fund it. Among other things, public policies should be created to fund research more fairly and generate the greatest collective benefits. For that to happen, the granting agencies have to share their data.

Maxime Blanchette-Joncas Bloc Rimouski—La Matapédia, QC

Yes.

You mentioned a little earlier that peer review committees can use bibliometric indicators as criteria for excellence—you just now said, “pull indicators out from nowhere”—such as citations, impact factors, the h-index and publication volumes. However, your research shows that the committees favour certain disciplines, as well as scientific publications in English, to the detriment of francophones, the humanities and emerging researchers.

Do those criteria really reflect excellence, or do they introduce new biases into research funding?

12:45 p.m.

Professor, Université de Montréal, As an Individual

Vincent Larivière

That definitely introduces biases into the research system. There's nothing inherently superior about publishing in one language or another. Almost all of our evaluation committees automatically give English publications higher prestige. That's how it is in Canada, and that's how it is in just about every country in the world.

In Canada, it's important to admit that this adds a particular dynamic to the country's linguistic balance. When Belgian scientists switch to English, that doesn't affect their language dynamic, because English isn't one of their national languages. In Canada, though, any switch to English automatically influences the country's linguistic balance.

Maxime Blanchette-Joncas Bloc Rimouski—La Matapédia, QC

Your work reveals discrepancies related to institution sizes, language—particularly for francophone researchers—and disciplines. However, the federal government's equity, diversity and inclusion, or EDI, criteria focus mainly on demographic diversity.

Shouldn't the criteria be broadened to include those measurable inequalities that determine scientific excellence?

12:50 p.m.

Professor, Université de Montréal, As an Individual

Vincent Larivière

Are you talking about language inequalities?

Maxime Blanchette-Joncas Bloc Rimouski—La Matapédia, QC

Yes.

12:50 p.m.

Professor, Université de Montréal, As an Individual

Vincent Larivière

Yes, that should happen, if EDI is being considered in a broader sense.

An official languages act should also apply to all of that. It's an aspect of diversity that brings a greater plurality of views to the scientific space. Canada gains an advantage. French has to be seen as an advantage for the Canadian scientific community, rather than a handicap.

Maxime Blanchette-Joncas Bloc Rimouski—La Matapédia, QC

International studies, such as the one by Aagaard and others, conclude that a more scattered distribution of funds produces greater collective results.

Should the Canadian federal criteria evolve along those lines to better serve excellence?

12:50 p.m.

Professor, Université de Montréal, As an Individual

Vincent Larivière

I think things have improved a lot in Canada. There was a time when the concentration of funds was such that the success rates were much lower. For example, there are currently two types of requests at the Social Sciences and Humanities Research Council. You can ask for a small amount, and the success rates will be higher, or you can ask for a large amount, and the funding rate will be lower.

I think the government can experiment more when it comes to spreading out research funding. Spreading out the funding allows for more chances for discovery. The more researchers there are doing research, the more discoveries there will be.

Maxime Blanchette-Joncas Bloc Rimouski—La Matapédia, QC

Since federal funds are public money, shouldn't the government require that the criteria for awarding them promote not only excellence, but also institutional, linguistic and regional diversity? What concrete reforms—such as caps per researcher, expanded core funding or better support for emerging researchers—would you recommend to better align excellence with the distribution of funds?

12:50 p.m.

Professor, Université de Montréal, As an Individual

Vincent Larivière

Those are all good suggestions in a way. What I want most is for us to give ourselves the room to experiment. At the moment, it's easy to imagine that the funding system generates inequalities and that there are biases. By trying things out, it will be possible to find slightly fairer ways to develop, to better measure or to better qualify scientific excellence.

As I said a little earlier, scientific excellence is multi-faceted. There's no one right way to do science.

The Chair Liberal Salma Zahid

Thank you. The time is up.

We will now proceed to the second round of questioning. We will start the second round with MP Singh Mahal.

Please, go ahead. You have five minutes for your round of questioning.

Jagsharan Singh Mahal Conservative Edmonton Southeast, AB

Thank you, Madam Chair.

Thank you to all the witnesses. Thank you to the members opposite as well.

Since this committee is about government funding for research and academic excellence, I will focus my questions on where the grant should go and how better use of that grant can be made. I will start with the heavily talked about question of DEI in this session.

My question is for Dr. Karram.

You said in your evidence that drawing a balance is an important thing that we need to keep in mind when federal funding comes into question. Here is my question for you. When there's DEI, it's a mandatory condition put by the government on educational institutions. How do you think a balance can be drawn for researchers who do not want to disclose that information? If they disclose that information, they risk not getting that funding or not having their application approved. How would you respond to that?

12:50 p.m.

Assistant Professor of Higher Education and Coordinator, Higher Education Graduate Program, University of Toronto, As an Individual

Grace Karram

We have years of research that shows that names, for instance, bias hiring with résumés.... This is well documented in sociology. When we have things like blind peer review as part of the process, that can be a way to evaluate the research without considering EDI. That's one option.

We also have really good streaming that we do for early career researchers. You say that you're an early career researcher, and you're in a particular pool that allows you to be evaluated accordingly. Why can we not have certain pools for people who are saying, “I want to be evaluated on merit”, and for people who say, “Look, I'm from a group that has historically been marginalized in the academy. I do not have the family resources. I went to a small institution, so I don't have the institutional resources to get these great research ideas off the ground”?

When I say that we're going to fund research infrastructure.... I work at a university where my hand is held from the inception of my research ideas right until the moment I click send on my SSHRC grant applications. I have such a robust community, offices that help me, and we want to make sure that groups of people who don't have access to that have access to that.

In their files, if we know they're from one of these groups and if they've been able to identify the reasons they have been on the outside of the research community, we want to bring them in, because that research that's getting missed is actually putting Canada behind. We know that international collaboration increases publications. The people who have natural ties to other parts of the world—because they're first-generation Canadians in the academy or have come here with a Ph.D. from another place—are a huge asset in the Canadian labour force, and we are missing it.

12:55 p.m.

Conservative

Jagsharan Singh Mahal Conservative Edmonton Southeast, AB

That was my next question, actually. You already mentioned that people from international jurisdictions who come to Canada with Ph.D.s in hand are also not being taken care of, and they are underemployed because they do not qualify for the grants that are available.

Don't you think that putting in a clause like DEI also eliminates those who want to be involved in education but don't want to come up because they don't want to disclose their political affiliations or because they have a fear of not being accepted if they disclose their political affiliations?