Evaluation of the Northern Projects Management Office (NPMO) 2016-2017 to 2018-2019

1.1 Program Overview

In Canada's territories, natural resources remain the region's economic backbone and large-scale driver for development and employment. Accounting for approximately 40 percent of Canada's landmass, the North's mineral and oil and gas assets have vast economic potential but remain largely undeveloped.

In the North, environmental review processes are founded in the framework for land claims and are guided by federal territory-specific and territorial (where devolution has occurred) legislation which govern waters, surface rights, environmental and socio-economic assessment, and land use. Specifically:

• In Yukon, the Yukon Environmental and Socio-economic Assessment Act (YESAA);
• In the Northwest Territories, the Mackenzie Valley Resource Management Act (MVRMA) in part of the Northwest Territories (with the exception as noted below); and
• In Nunavut, the Nunavut Project Planning and Project Assessment Act (NuPPAA) and the Nunavut Land Claim Agreement (NLCA).

As an exception, the Inuvialuit settlement region (with land in the Northwest Territories, and the North Slope Region of the Yukon) is governed by the Inuvialuit Final Agreement^{Footnote 3} and the Canadian Environmental Assessment Act (CEAA). The CEAA also applies to certain other federally regulated areas in the including offshore waters of the North.

These territorial regulatory regimes are based on a co-management approach to resource development and other major projects such as infrastructure. Reviews are conducted by co-management boards with responsibility for the planning, environmental and regulation of land and water use on Crown, territorial, settlement and private lands, and the examination of potential socio-economic impacts. Co-management boards are independent public institutions comprised of appointed board members based on recommendations from the federal and territorial governments, as well as Indigenous organizations and governments.

Federal government departments with regulatory authority on major projects in the North include the following (please see Annex A for additional information on the role of each department and agency in the Northern regulatory processes for major projects):

• Crown Indigenous Relations and Northern Affairs Canada (CIRNAC);
• Natural Resources Canada (NRCAN);
• Fisheries and Oceans Canada (DFO);
• Environment and Climate Change Canada (ECCC);
• Transport Canada (TC);
• Parks Canada (PC);
• Canadian Environmental Assessment Agency (CEAA);
• National Energy Board (NEB); and
• Canadian Nuclear Safety Commission (CNSC)

Aside from CanNor, all federal partners in the NPMO have their costs covered within their existing financial resource structure and are not receiving additional funding through the NPMO Initiative. As a result, while the NPMO Initiative involves horizontal coordination between federal entities, it is not formally considered a Federal Horizontal Initiative.

The NPMO project portfolio consists of projects planning on entering the environmental assessment process, projects currently within that process, and approved projects moving to implementation via the permitting phase.

As of November 2018, the NPMO project portfolio consisted of 34 projects worth approximately $36.6 billion. The projects include 24 mining projects, eight infrastructure projects, and two oil and gas initiatives. Of those, 15 are in what is called pre-planning, that is planning to enter the environmental assessment (EA) phase, eight are in the EA process; and 11 have had EA approval and are in the permitting phase. It is important to note that approximately 20 of those 34 projects have been inactive for more than two years. Inactive projects will be discussed in more detail later in the body of this report.

The NPMO works collaboratively with federal regulatory departments and agencies to manage regulatory requirements for major resource projects in the North. Specifically, the NPMO maintains a number of governance mechanisms to help advance resource development and major projects. These include:

• Territorial Project Committees for each territory, which can meet quarterly each year to review the state of resource development and emerging issues in the territory. These groups can include regulatory boards, territorial and federal regulatory departments and agencies, and Indigenous organizations on issues of common interest;
• Resource Development Advisory Groups (RDAGs), which include project proponents, territorial and federal regulators, and Indigenous representatives, that review the parameters for the project before it enters environmental assessment, providing an opportunity for early issues identification, the building of relationships, guidance to proponents and adjustment of the project design; and
• Project Working Groups, specific to each project, that coordinate federal input into the regulatory process and can include involvement of territorial governments when acting as regulators. These groups review project documents, assess impacts, coordinate the collection and dissemination of information, and oversee the federal decision process.

1.2 Program Objectives

The NPMO Initiative is part of a larger Government of Canada commitment to create a more effective and efficient northern regulatory system that encourages increased investment in sustainable resource development while enhancing environmental protection.

The NPMO was established within CanNor in 2010 based on a recommendation in the Road to Improvement: Review of the regulatory systems across the North Report. The NPMO's mandate is to improve the environmental review process for proposed major^{Footnote 4} resource development and infrastructure projects in the three territories by increasing federal coordination and policy capacity in the northern regulatory review process and improving the coordination of Crown consultations with Indigenous groups in the regulatory review process.

The NPMO is a service organization that facilitates and coordinates various activities throughout the life-cycle of a proposed resource development or regional infrastructure projects. NPMO takes a project management approach to ensure that activities and processes are well defined, transparent, timely and predictable and that federal agencies and departments involved in the environmental assessment (EA) and licensing and permitting processes clearly understand their role and are accountable for their performance.

The NPMO has been subject to three cycles of funding: from 2010-11 to 2012-13; from 2013-14 to 2015-16; and from 2016-17 to 2019-20 via an extension of funding for an additional four years.

NPMO delivers services to project proponents when proponents request assistance from NPMO; when more than one federal department has a role in the review process for the proposed project; and if there is a Crown consultation obligation. As the single window federal coordinator in the North, the NPMO's mandate includes improving the timeliness, predictability and transparency of northern regulatory processes to foster a stable and attractive investment climate for major projects in the North.

NPMO has three main activities and related outputs:

1. Supporting robust and thorough environmental assessment processes through coordination and issues management of major projects in the North (approximately 53% of resources requested);
2. Supporting Crown consultation duties^{Footnote 5} including early engagement and meaningful participation of Indigenous peoples and northern communities (approximately 26% of resources requested); and
3. Providing technical expert capacity to deliver evidence-based assessments (approximately 21% of resources requested).

According to the NPMO Logic Model (see Annex B),^{Footnote 6} the immediate outcomes of the NPMO Initiative are that:

• Movement of major projects through the regulatory system is effective, comprehensive and transparent;
• Gaps in the regulatory system are filled through maximizing the use of existing programs and tools and the development of new mechanisms; and
• Partnerships are established with northern governments and organizations.

The intermediate outcome of NPMO Initiative is that:

• Approved projects that are implemented spur significant economic and socio-economic growth in nearby communities.

The ultimate outcome of the NPMO Initiative is the achievement of:

• Strong, stable territorial economies for the benefit of Northerners and all Canadians.

As indicated in the NPMO Logic Model, the economic and socio-economic outcomes to which NPMO is contributing are also influenced by external factors such as commodity prices, availability of investments, existing infrastructure (e.g. roads, ports, etc.) as well as other economic development and social factors.

1.3 Partners and Stakeholders

The NPMO Initiative was initially created to increase capacity to support the Northern regulatory system in response to the growing number and complexity of major projects in the North, while ensuring that objectives of environmental and regulatory processes are achieved within established timelines.

In addition to CanNor, where the NPMO is located, the regulatory departments and agencies outlined in Annex A are responsible for supporting northern environmental assessment and regulatory permitting regimes. In addition to federal government departments and agencies involved in the regulatory approval process for major projects, the NPMO works with:

• Northern Environmental Assessment/Review Boards, as well as Land and Water Boards;
• Indigenous organizations;
• Indigenous governments;
• Industry representative organizations;
• Territorial governments;
• Municipalities;
• NGOs; and
• the public.

1.4 Alignment with Government and Agency Priorities

The NPMO Initiative aligns to government priorities as outlined in the December 4, 2015 Speech from the Throne under the priority of "a clean environment and strong economy", with a specific commitment that "public input will be sought and considered". The Speech from the Throne further articulated that: "Environmental impacts will be understood and minimized. Decisions will be informed by scientific evidence. And Indigenous peoples will be more fully engaged in reviewing and monitoring major resource development projects."

The NPMO Initiative is one of four programs under CanNor and represented in the Performance Information Profile as "Efficient and Predictable Environmental Review Process in the North".^{Footnote 7}

1.5 Governance

A Major Projects Deputy Ministers' Committee (DMC) was established to act as the governance body for the implementation of the Cabinet Directive that established the Major Projects Management Office (MPMO) and the NPMO, with a mirror committee at the Assistant Deputy Minister level. A previously existing DG Committee was reconstituted at the end of 2018.

Description of Figure 1: Governance Model

• CanNor President (DM) chairs the Major Projects Deputy MInister's Committee
• CanNor Vice-President (ADM) chairs the Major Projects ADM Committee
• NPMO DG chairs the Major Projects DG Committee

The membership of the DMC includes the Deputy Heads of:

• Natural Resources Canada (Committee Chair);
• Environment and Climate Change Canada;
• Fisheries and Oceans Canada;
• Crown Indigenous Relations and Northern Affairs Canada;
• Indigenous Services Canada;
• Transport Canada;
• Justice Canada;
• Innovation Science and Economic Development Canada;
• Parks Canada;
• Canadian Environmental Assessment Agency;
• National Energy Board;
• Canadian Nuclear Safety Commission;
• Canadian Northern Economic Development Agency; and
• other members added at the discretion of the Chair.

The major activities of the DMC include:

• Coordinating the implementation of the Cabinet Directive as well as related Memoranda of Understanding (MoU);
• Upholding the objective of improving the performance of the regulatory system for major resource projects;
• Providing coordination and guidance for the resolution of issues related to specific projects in the regulatory system; and
• Serving as authorizing signatories for each Project Agreement.

1.6 Resources

Financial resources for the program consist of time-limited funding (C-base) 2016-17 to 2019-20 of $9,275,844 (excluding PWGSC charges) over the four-year period, and ongoing funding from CanNor (A-base). Total funding for the three years of the period under evaluation were $11,458,404.

The following table provides a financial breakdown over the prescribed time period.

2.1 Evaluation Scope and Objectives

The purpose of this evaluation is to examine the relevance, effectiveness and efficiency of the NPMO Initiative and assess whether improvements could be achieved going forward by making recommendations as required in alignment with the three evaluation issues.

The evaluation covers all objectives of the NPMO Initiative and core activities of the NPMO outlined in Section 1.1.2 of this report and is focused on the three-year timeframe taking place since the last evaluation of the NPMO, from 2016-17 to 2018-19.

This evaluation was conducted as per the Treasury Board Policy on Results (2016) and focuses specifically on commitments made in the Treasury Board Submission that provided funding for the 2016-17 to 2019-20 period. The evaluation issues explored within the scope of this evaluation are provided below.

Relevance

1. Continued need for the program.
2. Alignment with Government of Canada (GoC) priorities.
3. Alignment with federal roles and responsibilities.

Performance: Effectiveness

4. Achievement of Expected Outcomes.

Performance: Efficiency

5. Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes.

The full evaluation matrix including performance indicatorscan be found in Annex C.

2.2 Evaluation Methodology

The evaluation was conducted in four phases: planning, data collection, analysis and reporting. The planning phase included development of the evaluation plan, which included a work plan and data collection instruments, and the evaluation matrix (see Annex C). The evaluation consisted of three main lines of evidence: key informant interviews, document review and case studies. Data collection was carried out between November 2018 and early January 2019, including field visits to Iqaluit, Whitehorse and Yellowknife to interview NPMO staff and stakeholders.

For document review, the NPMO office provided the evaluators with a selection of documentation supporting NPMO outputs. The evaluators also reviewed CanNor, NPMO and other government documentation including legislation related to the NPMO mandate for a total of 130 documents. Internet searches were also conducted as related to the case studies with an emphasis on identifying relevant media articles.

Information obtained through the document review was collected in a document review technical report. Findings were developed for each evaluation question where relevant evidence was identified. The scope of the document review for the general evaluation consisted of 55 individual documents.

Case studies were selected based on the following criteria: geographic coverage (i.e., one case study per territory); materiality; maturity of the projects with respect to the latter stages of the EA process; and variation in the types of projects (i.e., infrastructure and mining). The NPMO provided a master list of projects from which the evaluation consultants highlighted case study candidates, with a final selection made in consultation with the Project Authorities.

A total of 75 documents were reviewed for the case studies and 13 interviews were conducted with relevant personnel. Evidence was collated and analyzed, and findings consolidated by evaluation question for each case study. Internal reports were created for each case study.

An initial interview list, categorized by the range of stakeholder groups, was compiled by the NPMO and Project Authority and submitted to the evaluators. For the general evaluation, 32 interviews were conducted, with a further 13 interviews conducted for case studies. An internal technical report was developed that analyzed the interviews by stakeholder group and provided findings by each evaluation question.

The analysis phase included developing an evidence matrix that included the findings for each line of evidence, the summary findings for the evaluation, and potential recommendations. The evidence matrix was shared and validated with the Project Authority and NPMO staff. A preliminary findings presentation was also provided to an ad hoc Interdepartmental Evaluation Advisory Group composed of representatives from seven federal departments and agencies with a role in the review of major projects in the North. Upon receiving feedback, the evaluators developed a draft and final evaluation report.

2.3 Limitations and Mitigation Measures

There were certain limitations encountered during the evaluation which should be taken into consideration when reading this report.

3.1 Is there a continued need for the NPMO Initiative and its core activities?

There was a general consensus among stakeholders on the continued relevance of the core functions of NPMO, namely federal government coordination, advice and issue management, and maintaining the Crown consultation record.

Industry and territorial stakeholders had a more diverse opinion on the relevance of pathfinder services (e.g., advice and referrals to help navigate the northern regulatory process), with some considering that either the private sector or territorial governments were capable of providing that service. Socio-economic assessments and promoting investment in the North were ranked as the least relevant NPMO activities to stakeholders.

3.2 Is the NPMO program aligned with government priorities and CanNor strategic objectives?

As outlined in the December 4, 2015 Speech from the Throne 2015 and the Ministerial mandate letters, NPMO is aligned to the federal priorities of major resource developmentthat is grounded on robust oversight and environmental assessment processes with decisions based on science, facts and evidence. NPMO is directly aligned to the CanNor priority of "advancing major project development in the territories", and the CanNor core responsibility of "Economic Development in the Territories".

With respect to reconciliation and meaningful participation of Canadians and Indigenous groups, the NPMO is aligned with federal priorities. However, the NPMO's role is limited to monitoring and maintaining the Crown consultation record, developing consultation models, and sending consultation and notification letters to Indigenous organizations and communities to solicit their participation in the EA processes, in fulfillment of Canada's legal commitments as per Section 35 of the Constitution Act, 1982.

3.3 Is the NPMO Initiative consistent with existing and proposed federal roles and responsibilities?

The policy authority for the NPMO mandate and activities is derived from numerous acts and government priorities and its role is further defined in the MoU entitled "MoU Defining Terms and Scope of cooperation between federal departments, agencies and NPMO for coordination of Northern Project (2012)." The NPMO has further defined its role through the elaboration of individual MoUs with territorial governments.^{Footnote 10}

NPMO activities and outputs are consistent with the federal role as outlined in its foundational program documentation. Those activities are focused on the environmental assessment process for major projects in the North, and include:

• Coordination and issues management, including single window support, policy and advocacy and horizontal coordination;
• Crown consultation duties with respect to Indigenous peoples and northern communities;
• Pan-territorial coordination and issues management; and
• Provision of technical expertise for socio-economic assessments of major projects.

According to interviews, stakeholders were often not clear on the NPMO mandate despite the MoUs that are in place. All lines of evidence indicated that NPMO is seldom active during the post-EA permitting and licensing phase of the regulatory process despite its stated role. Interviews indicated that this can be a result of there being fewer federal government departments involved in the licensing and permitting phase.

3.4 Does the program bring value-added to the regulatory review process for major projects in the North?

Interviewees were asked to rank the theoretical value-added of NPMO core activities along a five-point Likert scale. The services were ranked by interviewees as follows (based on percentage categorizing this the activity as high or very high value-added):

• Single window coordination of federal activities (79%);
• Support for Crown consultation (76%);
• Advice and issues management across stakeholders (76%);
• Pathfinder services (69%);
• Promoting investments in the North (62%); and
• Socio-economic assessments (24%).

The responses received indicated that there is value-added to some of the core activities with respect to coordination of federal activities, advice and issue management and support for Crown consultation, being the primary activities ranked the highest. There was considerable variation of perspectives across stakeholders on different core activities, reflecting the diverse interests of stakeholders. As an example, pathfinder services were ranked relatively high by industry and Review Boards but ranked lower by federal and territorial government departments.

Some stakeholders had difficulties separating theoretical value-added from the reality of service delivery, with many pointing out that the NPMO is not meeting their expectations regarding delivery of core services in some jurisdictions over the period of the evaluation and is not realizing potential value added of enhanced core activities that may require a stronger presence and mandate.

The ongoing staffing profile of NPMO has an undeniable impact on service delivery. During the period of the evaluation project, there was only one person on active duty in the Nunavut office, out of a complement of three, and the Yukon office had undergone a complete staff turnover with both persons leaving within three months of each other. Overall, the NPMO had only seven of 12 positions filled, with two additional individuals on an extended leave of absence.

If the NPMO role is intended to be purely administrative in nature as a secretariat function for federal departments, NPMO offices need to be staffed accordingly. However, at present, the positions are staffed at a fairly senior level (CO-2 and CO-3), which would align with the expectations of a staff that can provide greater value-added services. Increased value-added was described as being able to "stick-handle" through complex issues and working with federal departments to develop a unifying Government of Canada perspective and position.

4.1 To what extent has the program produced expected outputs?

As stated previously in this report, the timing of the evaluation coincided with a substantial low-point in terms of staffing for NPMO. However, according to interviews, difficulty in staffing is a perennial problem with NPMO as it is with most organizations in the North. NPMO has consistently underspent its budget by approximately 30% during this period, largely attributed to less than planned personnel expenditure.

Interviewees were asked to rank the effectiveness of service delivery along a five-point Likert scale. When interviewees were asked to what extent has the program produced expected outputs across the core activities, "Don't Know" was the most frequent answer for four out of six activities: advice and issue management among stakeholders; pathfinder services; promoting investments in the North; and technical expertise on socio-economic assessments.

This result is not necessarily surprising given that NPMO's diverse stakeholders would not necessarily be fully aware of its suite of activities if those activities do not involve the stakeholder directly. In addition, the NPMO conducts little to no work in the socio-economic assessment area.

The effectiveness of NPMO core activities was generally ranked low by interviewees who gave answers other than "Don't Know". The following ranking is based on percentage categorizing delivery and effectiveness as high and very high:

• Support for Crown consultation (38%),
• Single window coordination of federal activities (34%),
• Advice and issue management among stakeholders (31%),
• Pathfinder services (21%),
• Promoting investments in the North (7%),
• Technical expertise on socio-economic assessments (3%).

Interviewees indicated that there is considerable variance in the effectiveness of service delivery across regions, with particular concern for the Nunavut office. Individuals did identify some NPMO activities that have improved. For example, Crown consultation support has new processes and templates that have improved coordination and record keeping, although there are cases where NPMO is not fulfilling this function as well as expected.

Outputs are being produced, including notification and consultation letters, issue tracking, Crown Consultation Adequacy Reports, briefing materials, meeting minutes and workplans. Stakeholders did observe inconsistencies in the production of these outputs over time in some jurisdictions, most notably in Nunavut.

There are also clear gaps in what is expected according to NPMO's own standard operating procedures (SOPs) and what is produced. Examples include:

• Lack of evidence of socio-economic assessments being produced or the presence of any technical expertise in that area;
• Lack of evidence of functional databases despite a list of IM/IT platforms;
• Outdated NPMO Project Tracker website, taken down during this evaluation, but duplicative of more accurate and detailed information available on Review Board websites.
• Other items not produced but included in the NPMO SOPs, such as Lesson Learned reports on each project, specific profiles on Indigenous groups, territorial land information, and Northern Project Agreements.

The individual case studies reaffirmed these findings. Stakeholders contacted through the case study process have varying views on the usefulness and quality of the outputs, from very low usefulness to some usefulness. In the case of the Back River project, a planned gold mine in the western Kitikmeot Region of Nunavut, there is evidence of some outputs being produced but also gaps in what would be expected according to the NPMO SOPs. Stakeholders associated with the Tlicho All Season Road (TASR) project were unaware of NPMO activities and outputs. There is however evidence of outputs being produced during the last seven months of the 28 month project. Outputs included joint letters from NPMO and the GNWT to seek input from Indigenous groups, a consultation issue tracker, the Crown Consultation Adequacy Report, workplans, participation in one meeting and briefing materials.

The Casino Gold Mine project in the Yukon has been on hiatus since 2016, but there is evidence of outputs being produced during the active EA phase, including meeting notes and email correspondence, briefing materials, issue tracking, numerous letters to stakeholders, consultation assessment tools, and a consultation plan.

4.2 To what extent is the NPMO capacity for single-window project management and coordination of federal activities throughout the regulatory life-cycle of major projects contributing to a more effective, comprehensive and transparent regulatory system?

According to all lines of evidence, NPMO's involvement in the regulatory life-cycle of major projects is largely limited to the EA phase with rare contributions to the licensing and permitting phase. As a result, the phrasing of this outcome (NPMO capacity for single-window project management and coordination of federal activities throughout the regulatory life-cycle of major projects contributes to a more effective, comprehensive and transparent regulatory system) is misleading and can be improved.

Regarding the EA phase, the system is designed and driven by the legislation^{Footnote 11} and processes of the duly established Review Boards. The federal Ministers responsible for the legislation are either the Minister of Crown-Indigenous Relations or the Minister of Environment and Climate Change in the case of the CEAA. It is therefore unlikely the NPMO, under CanNor and the Minister of Innovation, Science and Economic Development (ISED), could directly impact the legislation of another federal Minister when it comes to the EA processes. NPMO would have no role whatsoever in influencing the licensing and permitting system as this is the purview of the regulatory authority.

NPMO does however interact and engage with that system. Despite the noted inconsistencies in delivery over time and across jurisdictions, the work of NPMO in the area of federal government coordination can improve the effectiveness with which the federal government engages with the EA system. Structured communication improves the coordination across departments that avoids contradictory positions on issues and duplicative or unnecessary information requests.

The issue of timely response to EA timelines can be a sensitive issue. NPMO has no authority over other departments to ensure timeliness of their submissions. Interviews indicated that there is often push-back from departments in response to attempts by the NPMO to reinforce timelines. NPMO occupies a complex position, as departments have to follow their own approval processes and lines of authority and are committed to meeting timelines as best as they can without outside pressure. A clearer understanding of the time requirements for processes undertaken by each department would assist in setting expectations, but other government departments should understand they will be held accountable to their commitments.

Despite these issues, some interviewees commented that since the start of NPMO, the number of time extensions requested by federal departments to the various Review Boards has decreased. Many factors may be attributable for the decrease including the coordination role performed by NPMO which makes the process more effective. However, some industry interviewees highlighted their frustration with the federal government in establishing an EA system with timelines that federal authorities are unable to meet. Such delays are also not welcomed by the territorial governments or Review Boards.

The case studies further support the perspectives generated by the other lines of evidence. Interviewees contacted as part of the Back River Case Study held mixed views on whether the single window approach had been realized. One interviewee stated that NPMO did seek input from groups during the second round of consultations^{Footnote 12} and that this process was helpful. Another interviewee stated that for a single window approach to work, it requires the support of all federal departments which NPMO does not have.

One area where NPMO has played a substantial role is in facilitating the establishment of the Pan-Territorial Assessment and Regulatory Board Forum. The first forum was held in January 2016 and three more meetings have been held since the inaugural session, with the most recent meeting taking place in September 2018. The purpose of the Forum is to bring together representatives from each of the EA and regulatory boards in the North to share best practices and to discuss opportunities for collaboration. Topics vary each year but may cover issues such as consultation, linkages between the assessment and regulatory phases, and incorporating traditional knowledge. As such, the Forum can have an impact on the comprehensiveness of the assessment and regulatory phases. It is recognized that NPMO did help initiate and facilitate the meetings.

4.3 To what extent is the NPMO capacity to support Crown Consultation contributing to meaningful engagement and participation of Indigenous peoples and northern communities?

The regulatory system for major projects has been established such that consultation activities are predominately undertaken by the proponents and the Review Boards during the EA phase and by Licensing Boards during the licensing and permitting phase. The NPMO approach is consistent with the federal guidelines on Duty to Consult^{Footnote 13} and NPMO's own consultation models^{Footnote 14}, and the NPMO procedures^{Footnote 15} which state that the NPMO relies on the regulatory review process to generate the bulk of the Crown consultation record. The consultation process is further augmented by additional specific efforts including any correspondence or meetings with specific federal departments. A large part of the record is comprised of industry engagement with the communities and any commitments made by project proponents.

In conjunction with other groups, NPMO contributed to the development of consultation models and associated training of federal staff. According to interviews, these models have been adopted to some extent by some territorial governments in their own consultation processes. There is evidence of NPMO activity and outputs for engagement, including the Crown consultation record, tracking of issues, engaging in direct consultation through letters, and assessing the adequacy of the consultation in terms of Section 35 commitments. There have been some issues, however, with the quality and consistency of the outputs, both across and within the three territories.

Given its limited role, NPMO is not in the position to contribute directly to meaningful engagement and participation. The NPMO role is strictly to ensure adequacy of the consultation in terms of Section 35 requirements. There were suggestions from some interviewees that early engagement is preferred. Given NPMO's limited role, it is also not possible for NPMO to unilaterally undertake such early engagement. The RDAGs address that need to some extent, but early engagement requires the participation of all federal departments involved in the EA process, and other interviewees voiced a reluctance to engage in consultation processes outside of a formal EA process.

4.4 To what extent have Indigenous knowledge and perspectives informed decisions for major projects?

Given that the amount of direct consultation done by NPMO is limited, NPMO's contribution to the stated outcome is equally limited. During interviews, very few instances were documented where NPMO engagement letters generated a response from an Indigenous group.

To put the consultation process in context, the proponent of the Back River project claimed to have undertaken over 250 consultation activities, including week-long consultations at the community level. The Nunavut Impact Review Board (NIRB) conducted two Panel Reviews, each of several days. By contrast, NPMO had sent out a total of 12 engagement letters. As stated in the previous section, this is not a criticism of NPMO, rather it is aligned with its role and mandate. It should also be noted that the position of Crown Consultation Coordinator in NPMO has been vacant as of March 2018.

Overall, Indigenous knowledge and perspectives influence decisions on major projects. The overall purpose of consultation is to inform decisions on projects and there is evidence to support that it does. As an example, the project certificate (i.e., the authorization for the project to proceed after the completion of the environmental impact review) issued by the Nunavut Impact Review Board for the Back River Project has 92 terms and conditions^{Footnote 16} attached to it, many of them raised by the Indigenous communities affected.

4.5 To what extent are partnerships established and nurtured with northern governments and organizations?

While the NPMO was involved in establishing a number of formal agreements until 2015, there were no new formal partnerships established during the evaluation period, from April 2016 to March 2019.

The pre-existing agreements are:

• Memorandum of Understanding Defining Terms and Scope of Cooperation between Federal Departments, Agencies and the Northern Projects Management Office for Coordination of Northern Projects (May 2012);
• Memorandum of Understanding between CanNor and the Kitikmeot Inuit Association (November 2012);
• Memorandum of Understanding between CanNor and the Department of Executive, Government of the Northwest Territories (June 2013);
• Memorandum of Understanding between the Canadian Northern Economic Development Agency and the Qikiqtani Inuit Association with respect to Cooperation for the Coordination and Management of Major Projects in the Qikiqtani Region (September 2013);
• Memorandum of Understanding between CanNor and the Department of Economic Development, Government of the Yukon (January 2015).

Historically, NPMO also developed Northern Project Agreements, which were project-specific agreements with other federal departments on the roles and responsibilities for the environmental assessment and regulatory phases of the project. Only three such agreements were completed.

In 2016, NPMO made the decision to halt the development and implementation of project specific agreements because their value-added was limited in comparison to the time required to finalize them and also because the relationships among northern regulatory federal partners are already defined in the 'Memorandum of Understanding defining Terms and Scope of Cooperation between Departments, Agencies and the Northern Projects Management Office for Coordination of Major Projects'.

Regarding the nurturing of relationships, most relationships with NPMO are generally viewed as positive. The relationships vary, with a more limited and as-required interaction with Review Boards and Territorial Governments. There can be tensions in relations with other federal departments, but these relationships are generally viewed positively. According to interviews, tensions can arise around process and timelines. Some interviewees viewed that turnover of NPMO staff is detrimental to the relationships that are established.

Evidence from the case studies varies, with some proponents stating there was a limited relationship with NPMO during the EA process and others reinforcing the need for the NPMO. Common to the case studies, however, was the fact that Indigenous groups and communities stated they had no relationship with NPMO.

4.6 To what extent is the NPMO capacity in the area of socio-economic assessments contributing to a better understanding of the socio-economic impacts of major projects?

The NPMO's position related to socio-economic assessment has been vacant for some time, and there was no evidence of any activities related to socio-economic assessments.

At one point there was an unsuccessful attempt to draw on a range of sources and data and consolidate that in a database for socio-economic information on northern communities, but no evidence of that work was available for review. In any event, development of the database does not necessarily translate into conducting socio-economic assessments or contribute to a better understanding of the impacts of major projects.

4.7 To what extent are gaps in regulatory system filled (by program, tools, mechanisms) by NPMO?

NPMO rarely engages in the licensing and permitting processes and therefore does not contribute to the regulatory system in that regard. With respect to the EA process, there was no gaps in the system filled by NPMO during this evaluation period.

4.8 To what extent are approved projects implemented?

There is no attribution of approval of projects to NPMO, so there can be no attribution of implementation of projects to NPMO. Implementation of approved projects is primarily dependent upon the financing and economics of the project. The licensing can take time, but it is a process that can be navigated by industry. Of the three case study projects reviewed by the evaluation, two were approved, with one progressing through the licensing and permitting process and the other in implementation as of March 2019.

According to NPMO, there were a total of 14 projects that initiated or ended their assessment process over the course of the evaluation period (i.e., from 2016 to 2019). Of those, nine completed their assessment processes during this period, with two moving to licensing and permitting, and one initiating implementation.

However, as noted by one interviewee, by the time a company has reached the end of an EA process, there has already been investment and spin-off economic benefits. One industry representative had estimated that they had invested approximately $40 million in the EA process alone. That investment does not include the infrastructure already developed for the base camp operations and other investments.

4.9 To what extent do approved projects spur economic growth and socio-economic growth in nearby communities?

The existing performance indicators included in the NPMO performance measurement framework were deemed unviable for data collection.^{Footnote 17} Furthermore, as previously reported, the NPMO does not undertake socio-economic assessments and, therefore, no evidence was available for review. No other studies on the socio-economic benefits of communities located by major projects were referred to the evaluation by stakeholders. While natural resource projects contribute to the GDP of each territory, the amount contributed by individual major projects is not clear and varies from territory to territory. In 2017, the territories reported for mining, quarrying, and oil and gas extraction as a percentage of territorial GDP as 13% for Yukon, 36% for NWT and 16% for Nunavut.^{Footnote 18}

Interviewees were able to provide anecdotal evidence for economic benefit to nearby communities, mostly from jobs and supply contracts. There is contention however with some interviewees who pointed to experiences where the socio-economic benefit is hard to define. Some reasons identified by interviewees that raise questions on socio-economic growth include the fact that sometimes labour is imported which itself can cause other issues including increases in the cost of living and housing; the level of employment positions that are available to local labour; social issues that may arise from increased income in communities; law and order and public safety issues; and the lack of tangible projects that have occurred in local communities as a result of benefit or cooperation agreements between Indigenous groups and industry.

4.10 What are the chief internal and external factors influencing achievement of the NPMO's objectives?

Internally, the expertise, capacity and turnover of NPMO staff members is considered a limiting factor to achievement of NPMO objectives. The recent period has seen a particularly high level of turnover. Staffing in the North has commonly been identified by interviewees as a challenge.

Externally, a range of factors were identified by interviewees as influencing outcomes, with changes in the legislative and regulatory framework, either nationally or regionally, considered to have the most impact on NPMO. Other issues can create more complexity in the working environment, such as unsettled land claims, new approaches such as phased developments of major projects, declining caribou herds and transboundary issues.^{Footnote 19} However, the most critical challenges to the effectiveness of the NPMO are internal.

4.11 What are the key lessons learned (best practices and areas for improvement) in the design and delivery of the NPMO?

In terms of best practices, the evaluation considered certain practices to be evergreen, meaning they are constantly in a state of review and revision for improvement. As such they can be considered to qualify under best practices and included:

• Work Plans for the Project Specific Working Groups (jointly developed with federal departments on review processes); and
• Consultation models including the consultation letters,^{Footnote 20} and other supporting tools such as the Issues Tracking Table and Crown Consultation Assessment Report.

Areas for improvement have been identified throughout this report, but when specifically asked this question interviewees commented most often on the need for improved consistency in services and a better and clearer articulation of NPMO role and services.

5.1 To what extent is the design of the NPMO Initiative appropriate for achieving its expected outputs and outcomes?

This question was further broken down into three sub-questions for which the findings are outlined below.

5.2 Is the NPMO governance structure clear and are other federal partners actively engaged?

For issues impacting major projects, the governance structure for the NPMO remains the Major Projects Deputy Minister (DM) Committee which was first established with the Major Projects Management Office (MPMO). It is therefore a shared governance structure.

The effectiveness of that committee in providing oversight and support to NPMO issues is uncertain. Interviews with stakeholders indicated that certain major projects in the MPMO portfolio are dominating the attention of the Committee, with the NPMO portfolio left to the end of the committee meetings with whatever time is remaining. There is also a lack of understanding on the Committee of the northern regulatory regime as most of the departments are more familiar with the CEAA which applies to all major projects in the provinces (and only the Inuvialuit settlement region in the Yukon and Northwest Territories in the North).

In the initial design of the MPMO there was also an ADM and DG Committee for Major Projects. No record of those groups meeting was shared with the evaluators for this evaluation period, but it was reported that the DG Committee was reinstated at the end of 2018.

5.3 How was NPMO performance information used in decision-making?

There is some tracking of activities and the progress of major projects through the EA processes on a monthly basis, but it falls short of using performance information for decision-making. The NPMO provides a summary of the projects in process and in the pipeline (i.e., anticipated) to the DM Committee on a regular basis, but there was no evidence of decisions being sought from the Committee on any issue.

More fundamentally, the NPMO logic model and resulting performance measurement framework is not well-aligned to its role and functions. As has been highlighted in this report, socio-economic assessments are an area where activities and outcomes are identified but for which the NPMO has conducted no activities to date. Similarly, some of the other outcomes are not aligned to NPMO activities so performance information can be of little value even if it was collected, which it is not.

6.1 Conclusions

Conclusions derived from analysis of the available data for all three lines of evidence have been developed and are presented by evaluation area.

Relevance
The NPMO delivers relevant and important services for the Government of Canada including federal government coordination, maintaining and monitoring the adequacy of the Crown consultation record, and providing support for issue management.

Other external services (i.e., services intended for stakeholders outside of the Government of Canada) such as single-window access to the federal government and pathfinder services may be relevant and have value, but their importance varies across stakeholders and across the three involved jurisdictions. The "single window" aspect of NPMO services currently requires better definition. While often implied as an external service that incorporates pathfinding, issue and advice management, and potentially other services, there is a lack of clarity regarding what the single window actually provides. NPMO activities in the post-EA, licensing and permitting phase is an area worth further exploration regarding possible value added of NPMO involvement, even if on a case-by-case basis.

Additional services that NPMO offers are considered lower priority by stakeholders, namely promoting investment in the North and socio-economic assessments. While still necessary activities in the North, NPMO should consider deprioritizing the delivery of these services given its current understaffed complement, competing priorities that this work introduces, and the inconsistency of service delivery on more critical core services.

Assessment of the relevance of the mandate, role and services of NPMO was often blurred by a lack of understanding on the part of stakeholders. A more in-depth analysis of NPMO stakeholders, including Indigenous groups and communities, and their associated needs and expectations is warranted to better define and clarify the NPMO service offering to those groups. Redefinition and clarification of the roles and responsibilities of NPMO is an activity that requires continuous engagement with partners and stakeholders due to persistent and ongoing changes in the operating context, including staffing changes, the introduction of new stakeholder organizations and individuals, and fundamental transformation of the overarching working environment including devolution of land and resource management responsibilities in the Yukon and the Northwest Territories as well as the negotiations toward devolution in Nunavut.

Effectiveness
As the NPMO moves forward, it will be important to ensure the quality and consistency of service delivery to its Government of Canada partners. Equally, when offering services to external clients, the needs of those stakeholders must be understood and addressed within their individual contexts. Based on a co-management model, the territorial regulatory regimes are different, as are the territorial government structures, Review Boards and industry sectors. As such, a "one-size-fits-all" offering is not feasible, and some degree of customization is required.

NPMO is in the complex position of having no regulatory role while being held responsible for coordinating federal regulatory departments without formal authority over those departments. This model can only work under certain conditions. One condition is undoubtedly the proper functioning of the NPMO and consistency in service delivery. The other necessary condition is an updated formalization of the relationships between NPMO and other federal departments and agencies (e.g. through a MoU) and the goodwill of the federal regulatory departments involved in the process. Most interviewees commented that such goodwill has, in many cases, been absent. Managing these challenging relationships with the other departments to engender better support is a role envisaged for the NPMO Director General, a position that has been filled on an interim basis for a large part of this evaluation period.^{Footnote 23}

The assessment of NPMO's performance has been hampered by a logic model and performance measurement framework that are largely unaligned to the actual roles and activities of the NPMO, with outcomes that are set at too high a level for attribution. A more appropriately aligned performance framework would provide better decision-making information and assist the organization on focusing on its core functions.

As noted in the report, maintaining adequate human resources within the NPMO has been a challenge and is the main factor contributing to its performance. While staffing in the North is a challenge, there is an opportunity for CanNor, as Canada's economic development agency for the region, to show leadership on alternative human resource strategies for ensuring the right capacity is in place as needed.

Efficiency
Effective governance is a critical success factor for the NPMO as it is in a position of having responsibility for federal government coordination but is not a regulatory body and does not have authority over the other departments or the process itself. When issues arise, there needs to be fair and transparent means to seek solutions. The current reconstitution of the DG Committee for Major Projects is a step in the right direction, but its effectiveness will need to be monitored and assessed.

The NPMO organizational design should have adequate flexibility to be able to respond to the ebbs and flows of resource development projects in the North. It should also maintain adequate management and performance monitoring of its staff and satellite offices. The NPMO would benefit from an organizational review to identify potential areas for improving efficiency or effectiveness and to allow for flexibility in its staffing of the HQ and satellite offices, as well as reconfiguration of management structures.

6.2 Recommendations

Following assessment and analysis of the evidence, the evaluation has produced four recommendations across three thematic areas.