¥������

On July 13, Canada’s health scientists followed attentively a meeting of the Canadian Institutes of Health Research in Ottawa. The brought together about 50 researchers and agency staff to debate a way out of the CIHR’s peer-review reform debacle. That the meeting happened in the first place represented a victory for the research community, as did its outcome.

Fundamental questions remain, however. Was this emergency meeting a symptom of a more serious disarray in the management of Canada’s largest science funding agency?

The Peer Review Debacle

CIHR reformed its peer-review system for adjudicating research grant applications, moving towards an that matched proposals to reviewers through an algorithm and replaced face-to-face panels with virtual deliberation.

This occurred in the context of CIHR’s implementation of a new program architecture, resulting from put forward in 2011 by an international review panel.

During consultations,  and groups such as the raised concerns about the wholesale changes being implemented simultaneously at the agency. Still, CIHR proceeded to hastily alter its grant adjudication process without properly piloting and perfecting the new system. The agency’s president Alain Beaudet remained defiant and deflected criticism.

Expected problems with the new virtual peer-review system occurred and were widely covered in the media (e.g. , ). Reports highlight the rushed implementation and a number of operational flaws that denote lack of preparedness and poor planning.

However, the CIHR Summit only happened after Health Minister Jane Philpott ordered the council to convene it, expressly in response to the research community’s concerns, which were poignantly articulated in an penned by and signed by more than 1,300 health researchers.

The meeting’s  was a commitment to restoring face-to-face peer review panels, as requested in the letter. While this policy reversal has been celebrated, a working group still needs to work out how changes will be implemented. And health scientists will need to deal with the fallout of the funding decisions to be announced today (July 14), which were made under the “live pilot” of the online adjudication system.

A Broader Pattern

The peer review troubles were not an isolated incident of ineffective organizational change, when council decisions are at odds with expert input.

In 2015, following a review of the agency’s 13 institutes, the council downsized its structure of  scientific advisory boards from 13 to 5. It did so “ignoring  warnings and advice from the two expert panels it convened to lead scientific consultations on its reforms”, as by Paul Webster.

According to Webster, chairs of the old boards described these changes as  leading to “a dramatic narrowing of CIHR’s access to specialist, scientific guidance.”

Recent reviews of agency systems and processes also raised warning signs of how decisions have been made at CIHR. The first, an  of CIHR’s Research Reporting System (RRS), questions the very purpose of RSS:

the purpose of the RRS requires further clarification and communication to internal and external stakeholders

Despite the investment on RRS, operational details of implementation were lacking and the limitations of the system were unknown. Besides, auditors found “no systematic process to gather feedback on the RRS from internal and external stakeholders”, and noted that “a continuous improvement and communication regime has not been formalized.”

The second review – Reforms CRM Project Independent Third Party Review – similarly identified issues related to “communications and stakeholder engagement”, “information for senior level committee decision-making”; “project management and planning”; and “internal stakeholder engagement and impact on personnel.”

The agency was to claim action was being taking on all fronts, blending generic promises with specific but unclear initiatives. On communications and stakeholder engagement, for instance, actions included:

the creation of a new centralized Contact Centre, as well as a dedication to adopting new ways to communicate and work together, ensuring greater clarity and transparency.

Ironically, this was released 2 months before the CIHR Summit.

Time for a Leadership Review

Does the CIHR leadership still deserve the government’s confidence to manage this $1 billion organization?

A pattern of questionable design, implementation, and management of organizational systems and processes is apparent at CIHR. The peer-review debacle was the culmination of this pattern in a very consequential and public way.

At the helm of CIHR since 2008, Dr. Beaudet was reappointed in 2013 for a five-year term. Issues identified in the reviews fall squarely within his long tenure as president.

That the largest science funding agency in the country needed to go into firefighting mode to revert its stance on face-to-face peer review should raise eyebrows about the agency’s senior management. Minister Philpott’s statement on the outcome of the CIHR Summit concluded by asserting:

I recognize that today’s meeting is only a first step towards restoring a productive and respectful relationship between CIHR and Canada’s health researchers, and that further meaningful engagement will be ‎required to this end. Canadians should expect no less.

Considering the successive examples of opaque decision-making, a review of CIHR’s leadership would be a step in the right direction to restore trust in the council more broadly.