Internal Audit of the National Risk Profile

Table of contents

• Conformance with professional standards
• Background
• Audit objective, scope and methodology
• Processes
  • Finding 1 – Processes
  • Finding 2 – Lessons Learned
• Engagement
  • Finding 3 – Whole-of-Society Engagement
• Governance
  • Finding 4 – Governance Structure and Operations
  • Finding 5 – Monitoring and Oversight
• Conclusion
• Considerations
• Annex A – Audit Criteria

Conformance with professional standards

This audit conforms to the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit, as supported by the results of the Quality Assurance and Improvement Program.

Background

Canada is affected by many different types of disasters. These events can cause significant health, infrastructure, environmental, economic and social damage with greater impacts on vulnerable populations. Building resilience, and reducing the impact of disasters requires a proactive approach based on a clear understanding of risk and current information on the capacity of the national emergency management system.

The Emergency Management Act stipulates that the Minister of Public Safety and Emergency Preparedness is “responsible for exercising leadership relating to emergency management in Canada by coordinating, among government institutions and in cooperation with the provinces and other entities, emergency management activities”. The National Strategic Risk Assessment (NSRA), which is presented through the National Risk Profile (NRP), aims to strengthen resilience and improve the overall understanding of disaster risks in all sectors of society. Approved and funded for five years under Budget 2019, as part of the federal implementation of Canada’s Emergency Management Strategy for Canada: Toward a Resilient 2030 (Emergency Management Strategy for Canada), the NRP Round One was identified as an activity in the 2021-22 Federal, Provincial, and Territorial Emergency Management Strategy Interim Action Plan (the 2021-22 Interim Action Plan) as well as an initiative in support of Canada’s international commitments under the United Nations (UN) Sendai Framework for Disaster Risk Reduction (2015-2030) and a key action in support of the National Adaptation Strategy Disaster Reduction Pillar. The NRP was also a commitment under the December 2021 mandate letter for the President of the King’s Privy Council for Canada and Minister of Emergency Preparedness, as a means to inform decision-making and planning.

The NRP was Canada’s first strategic, national-level risk assessment that provided a Canada-wide picture of disaster risks and the existing measures and resources within the emergency management systems to address them. While the NRP is not a policy document, the information it presents could be used to inform future emergency management policy decisions and investments to address the identified gaps of the Canadian emergency management system.

A coordinated effort by departments across the federal government was used in the development of the NRP, with Public Safety Canada (PS) acting as the lead department responsible. Within PS, the Emergency Management and Programs Branch (EMPB) led the initiative and was responsible for the development of the methodology and processes used to conduct the NSRA as well as leading reporting products related to the initiative. EMPB also chaired and provided secretarial support for the main governance committee, the National Risk Profile Director General (NRP DG) Coordinating Committee.

In its development, the NRP leveraged two evidence-based methodologies: the All-Hazard Risk Assessment (AHRA) methodology and the Emergency Management Capability Assessment Methodology (EM-CAM). Previous federal risk assessment initiatives also helped to form the basis of this methodology, including the 2016-2019 NRP Pilot. The objective of the NRP Pilot was to design and test a sustainable approach for national risk assessments.

The NRP also collected expert evidence and data from across whole-of-society. As such, the methodology used for the NRP included gathering evidence from participants representing a range of national stakeholders from across the Government of Canada, provincial, territorial, and municipal governments, Indigenous organizations, research institutes, non-governmental organizations, and other emergency management organizations.

The NRP was designed to take a phased, all-hazards approach to disaster risk (Figure 1). This required research to be carried out in phases or rounds with a few hazards assessed at a time. Risk and capability assessments were conducted for the hazards identified in each round.

The first public report was released for Round One of the NRP in May 2023, which examined the three most concerning and costly hazards facing Canadians: earthquakes, wildland fires, and floods. It also contained a chapter on pandemics as a contextual factor that affected disaster risk and response.

Work on the second round of the NRP started in fall 2022 and is examining three different hazards: extreme heat events, hurricanes, and space weather events. These hazards were selected given their high impacts on public health, critical infrastructure, the economy, and ecosystems.

As of July 2024, work to finalize and publish the results from Round Two was underway. However, the NRP is considered to have sunset on March 31, 2024, as funding to extend the initiative beyond the initial five years was not obtained.

Audit objective, scope and methodology

Objective

The objective of this audit was to examine whether the NRP was governed effectively, incorporated whole-of-society stakeholder engagement and was guided by documented processes.

Scope inclusions

The scope of the audit included documentation, records, and processes from Rounds One and Two of the NRP (March 2019 to January 2024).

Scope exclusions

Although the NRP involved a coordinated effort across the federal government, the audit only examined activities for which PS was responsible.

Methodology

For each criteria established (Annex A), an audit methodology was developed to sufficiently and appropriately examine the area in support of the audit objective. To complete the audit, the following methods were used:

Interviews

Interviews were conducted with relevant stakeholders within PS who were involved with NRP related activities.

Document review

Review of relevant documentation, including those related to governance committee artifacts and NRP related methodologies, guidelines and procedures.

Processes

Overview of the Risk and Capability Assessment

The NRP used two evidence-based methodologies to assess Canada's current level of risk to all-hazards and inform the collective ability to mitigate their impacts: AHRA and EM-CAM.

The NRP risk and capability assessments are conducted separately but directly inform each other (Figure 2).

Risk Assessment Methodology

Risk assessments for the NRP were conducted using the AHRA methodology, which was tailored for the NRP. Based on the ISO-31000 Risk Assessment guidelines, the AHRA methodology was originally published by PS in 2012-13.

The AHRA methodology involves the following steps:

• Setting the context – threats and hazards that could impact Canada are identified.
• Risk identification – risk scenarios are prepared, describing cause and consequence.
• Risk analysis – defining the probability and severity of consequences to estimate risk.
• Risk evaluation – collectively, these risk estimates represent a picture of "all-hazards" risk.
• Risk treatment – identifying and recommending risk control or risk treatment options, particularly through the application of Capability-Based Planning (CBP).

In each round of the NRP, three hazards were selected for evaluation. Hypothetical representative scenarios were developed to assess their potential emergency management system gaps.

For each scenario, an impact assessment was conducted for the following impact categories: people, economy, environment, government, and social function. The AHRA methodology was also adapted to consider critical infrastructure, Gender Based Analysis Plus (GBA Plus), and future-oriented risk drivers (e.g., climate change, urban population density, and changing demographics) as part of the impact assessment.

For each impact category, participants evaluated the near-term (within the next five years) risk of the hazard by order of magnitude, which refers to the level of severity, on a scale from ‘limited’ to ‘catastrophic’. Likelihood ratings were assigned by hazard experts using historical data and were refined through facilitated discussions.

Capability Assessment Methodology

The NRP used a CBP approach to identify and measure capabilities that are required to prevent or reduce the impact of hazards on communities, before, during and after events. This approach can help highlight potential gaps within the emergency management system. The NRP’s application of the CBP approach was called the EM-CAM.

The capability assessments drew from the Canadian Core Capabilities List (CCCL), a list of 38 emergency management functions co-created with provinces and territories. The activities on the CCCL are grouped under the five priority areas of activity from the Emergency Management Strategy for Canada:

1. Enhance whole-of-society collaboration and governance to strengthen resilience;
2. Improve understanding of disaster risks in all sectors of society;
3. Increase focus on whole-of-society disaster prevention and mitigation activities;
4. Enhance disaster response capacity and coordination and foster the development of new capabilities; and,
5. Strengthen recovery efforts by building back better to minimize the impacts of future disasters.

In Round One, an analysis was conducted in order to determine which capabilities would be applied by scenario. However, in Round Two all 38 capabilities were assessed for each scenario in order to address an identified gap in information based on the results from the Round One approach.

Participants assessed capabilities via surveys to evaluate baseline capacity and competency related to:

• People and organization;
• Policies, processes, and practices; and,
• Infrastructure, technology, and tools.

A second assessment was used to establish the desired target state of each capability, as well as how it may evolve through a future lens. A validation exercise was also conducted with applicable stakeholders. Capability gaps were then identified from each capability by comparing the baseline and target scores.

Finding 1 – Processes

What we expected

The audit team expected NRP-related methodologies to be appropriately communicated, and consistently implemented.

The audit team expected to see formalized guidance documentation for the NRP-related methodologies (the AHRA for the risk assessment and the EM-CAM for the capability assessment) and for the approach to be up-to-date and reflective of current practices.

What we found

In Round One, it was noted that the NRP DG Coordinating Committee was provided with consistent updates on the development of the NRP risk and capability assessment methodology and approach. The Committee also received information on updated processes and approaches for Round Two risk and capability assessments, which were informed by feedback, best practices and lessons learned from Round One. 

In addition, the AHRA was presented during a federal workshop in December 2019, where over 100 participants informed its refinement. A second workshop, of similar size, was held in March 2020 which sought input from participants on the AHRA as well as the EM-CAM.

Documentation was found to support the Round One virtual risk assessments (e.g., presentations and scenario templates). Similarly, Round Two documentation included information sessions presentations, survey outlines, and scenario templates.

In Round One, it was noted that an external facilitator was used for the virtual risk assessment sessions. However, there were challenges with regards to time management and obtaining information on the long-term drivers of risk. Ahead of the report being published, additional academics were consulted and some academic literature was reviewed to address gaps identified during the process.

In response to some of the challenges identified in Round One, including criticism of the lengthy virtual sessions, the NRP team revised their Round Two approach to utilize a survey model supplemented by information and validation sessions. Additional changes were also made to the impact categories based on feedback received in Round One.

Detailed internal guidance documents were developed for the AHRA and EM-CAM in Round One and further updated in Round Two.

Why it is important

Ensuring that documentation outlining the methodology and approaches used for the NRP remains current will assist in retaining corporate knowledge.

Finding 2 – Lessons Learned

What we expected

Given the iterative nature of the NRP, the audit team expected feedback, lessons learned and best practices to be collected and considered as part of continual process improvement.

What we found

The audit team found that best practices were considered as part of the development of Round One and process improvements between Rounds One and Two.

The audit team found evidence that feedback and lessons learned were collected and considered as part of process improvements within Round One and between Rounds One and Two.

Lessons learned activities were found to be led by the respective teams responsible for various components of the NRP. Examples of these activities include:

• A lessons learned exercise took place following the conclusion of the contract with the Indigenous consulting company;
• Lessons learned were compiled following the virtual risk assessment sessions in Round One; and,
• A post-mortem exercise took place following the publishing of the Round One public report.

Lessons learned with regards to the risk and capability approach were applied as part of the NRP process. Although efforts were made to store corporate knowledge as a means of preserving NRP documentation, the audit team did not find evidence of a central repository of lessons learned to help facilitate access to lessons learned in support of knowledge management and transfer for future reference.

The audit team also found evidence of plans to complete lessons learned activities as part of Round Two. However, this had yet to take place when the Examination Phase of the Audit concluded in June 2024.

Why it is important

Without a centralized approach to documenting and retaining lessons learned, there is a potential for corporate knowledge loss due to difficulty in retrieving information and staff turnover.   

Consideration

1. Develop an approach to enable lessons learned and best practices to be documented and retained for future reference.

Engagement

Overview of Whole-of-Society Principle and Engagement

In 2015, Canada adopted the UN Sendai Framework for Disaster Risk Reduction (2015-2030) a non-binding international agreement, which establishes the importance of adopting an all-of-society (or whole-of-society) approach to disaster risk reduction as one of its guiding principles.

The Emergency Management Strategy for Canada builds on the foundational principles articulated in the UN Sendai Framework for Disaster Risk Reduction (2015-2030) as well as the An Emergency Management Framework for Canada in order to establish Federal, Provincial and Territorial (FPT) priorities to strengthen the resilience of Canadian society by 2030.

Under the Emergency Management Strategy for Canada, the 2021-22 Interim Action Plan was developed stating that “[t]he adoption of a whole-of-society approach seeks to leverage existing knowledge, experience and capabilities within [emergency management] partners, including those of Indigenous organizations and communities.”

Under the 2021-22 Interim Action Plan, the NRP is listed as an activity under priority area two, “improve understanding of disaster risks in all sectors of society” noting that FPT governments will continue to participate and provide input into the NRP risk and capability assessment sessions.

The public report provides the following list of participating stakeholders: federal departments and agencies, provinces and territories, municipalities, Indigenous organizations and communities, as well as the academic, private, volunteer, and non-governmental sectors, selected from across different communities living within Canada.

The use of a whole-of-society approach as part of the NRP, differed from previous risk assessment initiatives (i.e., the NRP Pilot) which primarily consisted of consultations and input being provided from other federal entities.

Finding 3 – Whole-of-Society Engagement

What we expected

The audit team expected participation in the risk and capability assessments to be reflective of the whole-of-society approach. As part of this approach, it was expected that guidance would be developed in support of planning for whole-of-society engagement.

What we found

Engagement Approach

The 2016-2019 NRP Pilot was limited to federal government participation with some academic representation. As such, Round One of the NRP represented the first time a whole-of-society approach had been used with the risk and capability assessment methodologies.

An initial Federal, Provincial, Territorial, Indigenous (FPTI) engagement document was established, which outlined the objectives, timelines and tables to engage and inform between July 2020 and fall 2021 for whole-of-society involvement in the development of the NRP. However, it was not clear if all of the identified tables were contacted, and whether the engagement planning took place as outlined in the document.

Potential participants were identified through a variety of channels (e.g., consultation with other government departments, word-of-mouth, postings on PS website, Senior Officials Responsible for Emergency Management (SOREM), and external research). Although outreach activities occurred, participation in the NRP was voluntary, as such, not everyone who was contacted chose to participate.

The audit team found documentation that identified potential participant groups at a high-level. However, further planning documents, which outlined the requirements to fulfill the expected whole-of-society consultation and identify adequate levels of participation were not found.

Engagement Through the Risk and Capability Assessments

In Round One of the NRP, 294 attendees participated in the risk assessment. Attendees represented the following groups: federal agencies; Indigenous communities and organizations; experts and academics; critical infrastructure owners and operators; provincial and territorial agencies; non-governmental organizations; municipalities and emergency management services.

Several challenges were identified in the conduct of the risk assessment workshops:

• Round One workshops were lengthy (i.e., 4 hours) and time consuming for participants.
• Time did not always allow for the discussion of risk drivers (e.g., GBA plus context, future lens, changing demographics).
• Participants felt that some of the questions asked were not related to their expertise.

To address the time constraints associated with a workshop format, surveys were administered for the Round One capability assessments. Participants (83 stakeholders) were drawn from a sub-set of risk assessment participants and included representatives from federal, provincial or territorial governments, Indigenous communities and organizations, professional associations, local governments, non-governmental organizations, academia, and consultants. However, the Round One NRP public report included a reference that the assessment reflected a low participant return rate of eight responses or less for nine identified CCCLs.

With regards to the capability assessments, in Round One, only the individuals who participated in the risk assessment sessions were included. Although familiarity scores were used to ensure only those participants with knowledge could score them, this created gaps as only certain capabilities were assessed per hazard and some capabilities had low participation rates. Consequently, to mitigate this short-coming, additional work (i.e., literature review) was undertaken to supplement the data collected via the Round One risk assessment sessions.

However, for Round Two, the identification of experts, operators and practitioners were linked to each capability, to ensure all capabilities could be appropriately assessed.

Surveys were also used to administer the risk assessments as part of Round Two. This was part of a revised approach to the Round Two risk assessments to focus on engaging the right people with the right knowledge and expertise. For example, Round Two included additional advocacy groups who were not contacted as part of Round One to complete a GBA Plus specific survey. Organizations were sought who represented various interest groups (e.g., visible minorities, children, people with disabilities, women, new Canadians, 2SLGBTQI+) which enabled narrative information for GBA Plus to be captured. This approach was an improvement to the Round One process used to obtain GBA Plus information where insights were collected as part of the broader risk assessment sessions and participants reported feeling they were being asked questions that were not related to their expertise. 

Engaging with Indigenous Peoples

As part of Round One of the NRP, an initial FPTI engagement document was created, which identified Indigenous organizations and related tables for initial engagement in order to provide general awareness of the NRP. However, there was no guidance for the unique approach to be adopted when engaging Indigenous organizations.

Through document review and interviews, it was observed that the process in place for engaging Indigenous individuals, organizations and communities changed over the course of the NRP initiative depending on the Round and assessment stage (i.e., risk assessment vs capability assessment).

For example, Indigenous organizations were invited to participate in the Round One Virtual Risk Assessment Sessions, however there were noted gaps with regards to capturing Métis and Inuit perspectives.

As such, PS contracted an Indigenous consulting company to address the under-representation of Inuit and Métis communities, as well as incorporate and reflect Inuit, Métis, and First Nations perspectives into the NRP public report. Six engagement sessions took place in 2022, which enabled PS to hear from a range of Indigenous communities, including status and non-status First Nations individuals, urban Indigenous people, Inuit and Métis. As part of these sessions, an effort was made to implement Ownership, Control, Access, and Possession (OCAP) principles, and steps were taken to address gaps identified in its application.

Following Round One, a lesson learned exercise was conducted and identified the following: 

• Pre-engagement with leadership should be considered as part of the planning, and key messages to explain the engagement and information sessions should be developed.
• There was notable confusion among stakeholders regarding the intent of the engagement.

In Round Two, Indigenous Peoples were identified as part of whole-of-society consultation in the risk and capability assessments. However, as of July 2024, an Indigenous consulting company for Round Two was not contracted given its dependency on funding availability.

Why it is important

Guidance documentation surrounding the identification of target groups to participate in the NRP and their associated consultation processes would better support the application of a whole-of-society approach. 

Indigenous knowledge in disaster risk reduction and recovery is important to ensure that perspectives are captured as part of a whole-of-society approach.

Considerations

2. Develop and implement overall guidance on whole-of-society and the achievement of sufficient participation.
3. In consultation with the Indigenous Affairs Branch, develop and implement an approach for consistently engaging Indigenous Peoples as part of the NRP process.

Governance

Overview of the National Risk Profile Governance Structure

The NRP governance structure, as outlined in the NRP DG Coordinating Committee Terms of Reference (TOR), consisted of two main committees (i.e., the Assistant Deputy Minister Emergency Management Committee (ADM EMC), and the NRP DG Coordinating Committee). The NRP DG Coordinating Committee was further supported in Round One by three DG Interdepartmental Risk Tables (DIRTs); one per hazard. The NRP DG Coordinating Committee was the lead mechanism for NRP governance and development. Figure 3 outlines the planned NRP governance structure.

Assistant Deputy Minister Emergency Management Committee

The ADM EMC was initially identified as the overall governance committee for the Emergency Management Strategy’s Horizontal Initiatives, which included the NRP.

Per the October 2019 version of the TOR, the ADM EMC was permanently co-chaired by the ADM of EMPB and a member from another federal organization on a rotational basis. The membership of ADM EMC represented 29 federal organizations. The Committee’s mandate included: exercising leadership and providing strategic guidance on a full range of horizontal emergency management policy and program initiatives; advancing a whole-of-society approach to emergency management; and, fostering an intersectoral approach across government departments to assess risk, prevent/mitigate, prepare for, respond to and recover from disasters.

Due to the COVID-19 pandemic response requirements, the ADM EMC was not active and was functionally replaced by the ADM Crisis Cell, which was more operationally focused and did not provide oversight over the NRP. As such, the ADM EMC did not provide oversight over the initiative as initially planned.

NRP DG Coordinating Committee

As outlined in the NRP DG Coordinating Committee’s mandate, the Committee was responsible for:

• Providing federal leadership on the development of the NRP;
• Overseeing the coherence of federal risk assessment work related to the NRP; and,
• Providing direction on the development of products designed to improve knowledge of hazards, risks, and capability gaps.

The Committee was composed of 18 individuals representing 11 federal organizations:

• Public Safety Canada
• Natural Resources Canada
• Environment and Climate Change Canada
• Finance Canada
• Indigenous Services Canada
• Defence Research and Development Canada
• Infrastructure Canada
• Treasury Board Secretariat
• Public Health Agency of Canada
• Privy Council Office
• Office of the Chief Science Advisor 

The NRP DG Coordinating Committee was established to provide federal leadership on the NRP. Given the ADM EMC was not convening during the NRP’s development, the NRP DG Coordinating Committee was the lead mechanism for NRP governance and development. Further, although DIRTs were established, through a written response it was noted that “direction and guidance for the NRP rested solely with the NRP DG Coordinating Committee”.

DG Interdepartmental Risk Tables

In Round One, the NRP DG Coordinating Committee was supported by three hazard specific DIRTs; one for each risk area: floods, wildland fires, and earthquakes. Each DIRT was chaired by the lead Department for their specific hazard; PS chaired the Floods DIRT and Natural Resources Canada chaired both the Wildland Fires DIRT and the Earthquakes DIRT.

The mandates for the DIRTs expanded beyond the NRP (NSRA) and included hazard-specific activities out of scope for this audit. DIRT activities related to the NRP (NSRA) included contributing to the hazard scenarios and providing updates to the DG NRP Coordinating Committee.

DIRTs were not established for the Round Two hazards. The audit team was informed that, through internal lessons learned discussions held by the NRP team following Round 1, DIRTs were deemed to not be the most efficient forum to support the NRP. As such, in Round Two, PS worked directly with the lead departments for each hazard area.

Finding 4 – Governance Structure and Operations

What we expected

Given challenges noted in the 2016-2019 NRP Pilot related to leadership and sustainability of similar initiatives, the audit team expected to find a governance structure that consistently engaged senior leadership across departments.

The audit team also expected to find a clear and precise TOR that accurately reflected the operations of the NRP DG Coordinating Committee and was updated and communicated on an as-needed basis.

What we found

The TOR for the NRP DG Coordinating Committee stated that the Committee will “report to the Assistant Deputy Minister (ADM) Emergency Management Committee (EMC) and provide updates to other ADM tables as required.” However, as noted, the ADM EMC was not active during the development of the NRP and did not provide oversight over the initiative, which created a gap in the identified governance structure.  

The audit team was informed that the lack of oversight from the ADM EMC did not impact the NRP and its oversight as ADMs would have been briefed by their respective DGs. However, the use of less formal communication channels removed the opportunity for consistent briefings within a committee forum to allow for formal monitoring to take place at the ADM level. 

At the onset of the NRP DG Coordinating Committee’s formation, the mandate, roles and responsibilities of the Committee, its members and Secretariat were documented. The TOR also established that the document would be reviewed on an annual basis, however this did not occur and the document was not kept up-to-date. As such, the TOR included responsibilities and deliverables that were initially intended to be completed by the Committee that went unfulfilled. Of the six deliverables identified in the TOR, only two were partially met (i.e., annual Cabinet reports, biennial public reports). Only one public report was published before funding sunset in March 2024. Further, the audit team was informed that the focus of the Committee narrowed to the specific NRP product (i.e., the result of the NSRA).

Through interviews, it was explained that the use of alternates at the NRP DG Coordinating Committee was prevalent.

In communicating with PS members, one DG was not aware they were a member of the Committee. The DG informed the audit team that limited communication was distributed on the NRP since becoming a member in April 2022. Further, the nature of the communication led the DG to believe they were receiving it as part of a wider distribution list and not as a member of the Committee.

There were instances where the NRP DG Coordinating Committee was used to facilitate planning and decision making, however, it was primarily used to enable members to provide feedback when consulted.  As the NRP progressed, and members became more informed on the initiative, a more selective approach was taken in determining the type of information that was shared with the Committee.

Why it is important

The TOR is a communications product that establishes the roles and responsibilities of the Committee and also sets expectations related to its workings and deliverables. An outdated TOR results in a communications product that can lead to confusion and misunderstandings surrounding the committee’s priorities and the collaboration of stakeholders; which may impact the initiative’s strategic direction and success.

As noted in the NRP Pilot, similar initiatives have suffered due to an overall lack of leadership and sustainability. Given the initiative utilizes a phased approach that leverages expertise from other stakeholders both outside and within government, and is future oriented in its application by providing information on disaster risks to inform future policies, having appropriate senior management oversight could assist in contributing to the sustainability of the initiative.

Consideration

4. Assess the governance structure to ensure it reflects the operational and oversight needs of the NRP and is supported by appropriate documentation.

Finding 5 – Monitoring and Oversight

What we expected

The audit team expected governance committees to remain informed on the progress of deliverables and emerging risks through regular reporting.

What we found

As previously noted, the ADM EMC was initially expected to provide oversight over the NRP. However, as the Committee was not active during Rounds One and Two of the NRP (as of January 2024), the initiative relied on individual DGs to brief their ADMs. This was not the ADM-level oversight model that was initially envisioned and created a situation whereby briefings at the ADM-level were siloed in nature, done outside of a committee forum and broader governance structure.

The NRP DG Coordinating Committee received updates on Round One of the NRP project at scheduled meetings (e.g., updates from DIRTs, updates from drafting teams, risk and capability assessments). However, there was a notable decrease in Committee meetings following 2020 as Round One progressed and during Round Two only two committee meetings occurred (i.e., October 25, 2022, and June 22, 2023). The audit team was informed that:

• Meeting frequency was adjusted to align with the input requirements of the project stage, as some stages, such as the development of the NRP methodology and approach, required more frequent input from members than others.
• There were instances earlier in the initiative where some items, which were deemed more appropriate to discuss at the working level (e.g. line-by-line demonstrations of surveys), were brought to the Committee unnecessarily.
• Secretarial communication was used in lieu of meetings, however, limited supporting documentation was provided of secretarial communication and as such a regular frequency of reporting was not identified.

Through interviews and document review, the audit team noted the use of planning and monitoring documents. These included critical paths for reporting and Gantt Charts in Round Two for risk and capability assessment. However, it was noted in interviews that these products were not always provided to the governance committee due to their level of detail. Instead, updates were integrated into presentation materials and presented to the Committee and focused on high-level next steps.

Why it is important

Without a determined reporting format and frequency that provides a clear and concise overview of the initiative, including work underway and work outstanding, it is difficult to effectively monitor against timelines and deliverables.

The use of informal communication channels removed the opportunity for consistent and collaborative briefings to allow for more generative discussion and formal monitoring to take place at the ADM level.

Consideration

5. Assess the reporting needs of the committee and establish and implement a reporting mechanism in support of monitoring its progress and deliverables.

Conclusion

Overall, the audit found that the NRP in its development was guided by documented processes, informed by whole-of-society stakeholders’ engagement, and supported by a DG-level governance committee. Opportunities remain to improve the documentation of processes and practices, increase clarity around engagement and strengthen some governance activities.

More specifically, the audit concluded the following:

• Processes related to the risk and capability assessments were found to be documented, communicated, and consistently followed. Importance should be placed on ensuring that documentation surrounding the methodology, approach and lessons learned continues to remain up-to-date and accessible for possible future rounds of the NRP.
• The incorporation of stakeholder engagement was evident in the NRP process. Clarity surrounding whole-of-society expectations and the approach to engaging Indigenous Peoples could be beneficial to future rounds of the NRP.
• The NRP DG Coordinating Committee effectively oversaw the development of the NRP during Round One. However, a reduction in governance activities as part of Round Two reduced the level of oversight over the initiative. Further, there was a gap in the intended governance structure over the NRP given the lack of an active ADM-level committee.

Considerations

Recommendations will not be issued given the NRP initiative sunset on March 31, 2024. However, the audit team has identified considerations that could be actioned should the initiative continue. Considerations do not require a management action plan (MAP) and are not subject to the Internal Audit and Evaluation Directorate’s MAP follow-up process. The following actions have been identified for the ADM, EMPB to consider.

Consideration 1:

Develop an approach to enable lessons learned and best practices to be documented and retained for future reference.

Consideration 2:

Develop and implement overall guidance on whole-of-society and the achievement of sufficient participation.

Consideration 3:

In consultation with the Indigenous Affairs Branch, develop and implement an approach for consistently engaging Indigenous Peoples as part of the NRP process.

Consideration 4:

Assess the governance structure to ensure it reflects the operational and oversight needs of the NRP and is supported by appropriate documentation. 

Consideration 5:

Assess the reporting needs of the committee and establish and implement a reporting mechanism in support of monitoring its progress and deliverables.

Annex A – Audit Criteria

Criterion 1

An effective governance structure is in place to provide direction and support timely decision-making, monitoring and reporting of the NRP initiative.

Criterion 2

Engagement processes are in place to receive input from whole-of-society stakeholders to support the NRP initiative.

Criterion 3

The NRP initiative is guided by documented processes that are consistently applied and incorporates lessons learned as well as best practices.

Date modified:

2024-11-28