Evaluation of the Inclusive Diversification and Economic Advancement of the North Program, Northern Business Relief Fund, and Northern Isolated Community Initiatives Fund 2019-2020 to 2022-2023

IDEANorth exists to address an ongoing need to diversify the Northern economy

The economic landscape of Canada's territories is largely shaped by natural resource extraction and development projects, which, while contributing significantly to Canada's and the local economies' growth, also expose them to boom-and-bust cycles due to their resource dependency. As stated in the Arctic and Northern Policy Framework, while sustainable resource development is the cornerstone of the territorial economies, there is also a need to grow other areas to help insulate these resource-based economies and to provide more diversity of opportunity for Arctic and northern peoples.^{Footnote 7} Challenges such as sparse and widely distributed population, a significant infrastructure deficit, high energy costs, difficulty accessing capital, fluctuations within the natural resource extraction sector due to cyclical commodity pricing, and a shortage of skilled labour pose major hurdles to economic growth and diversification.^{Footnote 8} These conditions underscore the importance of sustaining economic development programs, such as IDEANorth, to foster the growth and diversification of Northern economies.

Key informant interviews with internal CanNor and other government representatives indicated that IDEANorth remains very relevant. Participants shared that IDEANorth is well-designed to address:

• Economic development in the territories through economic diversification and innovation (n=11).
• Creating a more robust economy that has higher quality, better paying jobs, and an improved skilled workforce (n=4).
• Major sector development for SMEs (n=3); and,
• Infrastructure gaps that are barriers to economic development (e.g., housing) (n=3).

Funding participants also agreed that there is an ongoing need for the IDEANorth Program. Without the CanNor funding provided through the IDEANorth Program, participants shared that their project would not have been realized. As a result, business expansion, growth in revenue streams, and increased profits would not have occurred. Additionally, CanNor funding streamlined participants' ability to complete their project all at once, decreasing the time before they were operational. Recipients of IDEANorth funding who completed the evaluation survey indicated that the funding allowed for increased industry capacity (e.g., investments in training, development of new projects, increased technology, economic development) (n=21) and that the funding allowed for the purchase of needed equipment (n=4).

Food security challenges in the North show that there is an ongoing need for the NICI Fund

Northern and remote communities in Canada, particularly in Indigenous communities, face drastically higher food prices and challenges in satisfying their nutritional needs through traditional methods. Harsh environments, a lack of land suitable for agriculture, the cost of energy and transportation, and the expense of food production solutions are all significant barriers. Statistic Canada reports that while 16% of Canadian households experienced food insecurity in 2021,^{Footnote 9} the rate across the territories was 26.4%, including 46.1% in Nunavut, 22.2% in the Northwest Territories and 12.8% in Yukon.^{Footnote 10}

Initiatives, like the Northern Isolated Food Initiative (NICI) Fund, support improving food security in northern and Indigenous communities. Further, the NICI Fund promotes economic growth and diversification of the northern food sector, which is needed to improve the health and sustainability of communities to enhance growth in other sectors.

NICI recipients agreed that without the funding provided by CanNor through the NICI Fund and the Northern Innovation Food Challenge, their projects would not have been implemented. In addition, recipients shared that the funding provided by CanNor was instrumental in securing additional funding needed to complete their projects. For example, one recipient shared that without the CanNor funding, they would have continued to make small improvements to their business however, they would not have been able to improve the availability of food in the north to the degree that they have since receiving the funding. In addition to addressing a funding gap for implementing innovative food solutions in the north, recipients felt that the nature of the Northern Innovative Food Challenge provided them the space to think differently about food security in the north. This allowed recipients to develop innovative approaches that they would not have considered otherwise.

All internal CanNor representative key informants indicated that there is an ongoing need for the NICI Fund. Specifically, because the NICI Fund is designed to address:

• Economic growth and developments of the northern food sector to improve food security in the north (n=8 ) and,
• Developing sustainable food security systems (n=3).

The unprecedented impact of the COVID-19 pandemic required government support for SMEs to reduce the overall impact on the northern economy.

The territories are home to more than 4,000 SMEs, 80% of which have fewer than 20 employees.^{Footnote 11} The economic and social context is unique in the North as all communities identify as rural and/or remote and contend with issues specific to the territories (e.g., significant physical and social infrastructure deficit, overcrowded housing). Lack of economic diversification means that the territories are particularly impacted by external drivers, as well as, a general higher cost of living and doing business. Economic development in the territories faces several inherent challenges including small, dispersed populations (i.e. 124,000 across three territories; 38 communities are fly-in, including all of Nunavut.

Businesses in the North were significantly impacted by the limits placed on physical distancing and gatherings because of COVID-19. These businesses are often the lifeblood of many small and isolated communities as they frequently represent the only one of their kind in the community and play a critical role in ensuring the safety and well-being of Northerners. Preliminary analysis from the Yukon Chamber of Commerce shows that only half of SMEs expect to be able to weather the COVID-19 pandemic shutdown. If businesses close, many owners are indicating they will not have the financial resources to restart them. Should the close permanently, they would leave a significant gap in locally available services and have a long-term negative effect in communities.

Therefore, on April 20, 2020, CanNor launched the Northern Business Relief Fund (NBRF), which made available existing funds of $15 million in non-repayable support to assist with operating costs not already covered by other federal measures. This Fund, which was unique among RDAs, prioritized businesses with less than 20 employees, and provided up to $100,000 per recipient to cover eligible fixed expenses between April and July 2020. In the first week of its launch, CanNor received over 120 applications for funding, representing approximately $5 million in funding requests.

All CanNor representatives who participated in a key informant interview agreed that the NBRF was extremely relevant in response to the COVID-19 pandemic, as this program responded to an urgent need to protect the northern economy. The funding was said to be complimentary to other government supports and saved SMEs from bankruptcy.

NBRF funding recipients who completed a survey provided a range of business expense-related reasons for why the funding program was necessary:

• Funding allowed for the maintenance of business costs so they could continue successfully through and/or after the pandemic (n=28)
• Funding supported businesses who noted they lost revenue while closed, but still had high overhead costs (n=27). A subset of these respondents were in the tourism industry and lost all revenue (n=8).
• Funding was crucial due to forced government shutdowns (n=14)

NBRF funding was complimentary to other supports.

In addition to the NBRF funding, participants received funding from other sources including the bank, the Canadian Emergency Business Account (CEBA) loans, and the Canadian Emergency Response Benefit (CERB) to further support their business. The NBRF was seen as particularly effective because the funding received was unique in that it allowed businesses to cover operational expenses and was non-repayable if the business did not turn a profit. A funding recipient included in the case studies carried out to inform this evaluation shared that their revenue dropped by more than 75% the following year and that it would have been very hard to cover those operational costs without the funding. Furthermore, the non-repayable nature of the program allowed them to work on re-building their business after the pandemic, rather than being stressed about paying back their loans.

IDEANorth appears to be well-aligned with the core mandate of CanNor, the Arctic and Northern Policy Framework, and the Pan-Territorial Growth Strategy and the high degree of alignment between project purpose and program design, confirm the relevance of program funding.

IDEANorth supports CanNor's core responsibility of economic development in the territories and is directly linked to the following departmental result of:

• Businesses are developing in the territories. Specifically, CanNor tracks the number of businesses by business locations in the territories, which enables the Agency to identify whether interventions are supporting small and medium sized enterprises (SMEs). Under IDEANorth, CanNor also tracks support to SMEs, in addition to SMEs owned by underrepresented groups, including women, youth and Indigenous peoples. IDEANorth also measures the level to which recipients can leverage funds to demonstrate the commitment of other organizations to their project.
• People and communities participate in the economy of the territories. Specifically, the Agency tracks the labour participation rates in the territories, as well as the number of women, Indigenous peoples and youth engaged in IDEANorth projects. This enables the CanNor to identify whether interventions are helping to increase participation of targeted groups in the economy.
• Innovation and Skills Plan, pursuing results that align with all three of the Plan's Action Areas (People & Skills; Research, Technology & Commercialization; and Companies). Specifically, IDEANorth provides the foundational investments necessary for economic development in the territories with the goal to help communities diversify and transition to new forms of economic activity and promote a clean growth economy. It also targets the inclusion of under-represented groups to optimize their energy and contributions to ensure innovation has a broader reach throughout Canada's diverse communities.

The Arctic and Northern Policy Framework (ANPF) is a comprehensive co-developed strategy by the Government of Canada to address the unique challenges and opportunities in the Arctic and northern regions. Its goals include nurturing healthy communities, investing in infrastructure, creating jobs, supporting research, addressing climate change, ensuring safety and security, restoring international leadership, and advancing reconciliation between Indigenous and non-Indigenous peoples.^{Footnote 12} The IDEANorth program is well aligned with the ANPF by focusing on fostering economic growth and diversification in the territories and through an emphasis on involving Indigenous people in the development and implementation of projects.

Internal CanNor representatives agreed that the IDEANorth program aligns well with the federal government roles, responsibilities, and priorities (n=12). Specifically, for CanNor, whose mandate is to grow territorial economies, these programs help to address many of the gaps that will help to fulfill this mandate. CanNor representatives discussed how CanNor, and all programs, are linked back to the Arctic and Northen Policy Framework (ANPF), which is linked to the Pan-Territorial Growth Strategy (PTGS); CanNor's response to providing a funding tool that is linked to the ANPF. IDEANorth is embedded into all aspects of the PTGS. IDEANorth creates jobs, improved job quality, and works to develop sectors from the group up.

The programs support GBA Plus outcomes by supporting Indigenous and women-owned businesses.

Program data tracking the number of projects supported by IDEANorth with Indigenous, women and youth (under 30 years of age) proponents. Based on this data, over the evaluation period, IDEANorth supported at least 96 projects with an Indigenous proponent, 27 with women, and 8 where the proponent was a young person under 30 years of age.

IDEANorth funded at least 235 projects over the evaluation period, with distribution across the 3 regions and project types

The following table shows the descriptive information for all IDEANorth projects listed in a project database provided by the Project Authority. As shown, over the evaluation period 235 projects were funded by IDEANorth including 96 based in Yukon, 90 in NWT, and 49 in Nunavut. Projects were distributed by type of activity, with the most common being product and market development (65 projects), targeted investments (59), and research and planning (28).

IDEANorth proponents leveraged other funding sources at a rate of $2.59 for every $1.00 of CanNor funding

As shown in the following table, the leveraged amount ranged between $1.64 and $3.75 for every dollar of CanNor investment over the four years of the evaluation period.

IDEANorth made progress towards its short-term results and objectives over the evaluation period

This includes making progress towards the outcome of "territorial economic growth and capacity is supported" though the 1,957 people who attended training sessions, capacity building activities or workshops between 2020-2021 and 2022-2023. The program also notes funding 50 projects over the evaluation period that support small scale infrastructure, which contributes to progress made towards the intended outcome of "gaps in territorial small-scale economic infrastructure are addressed".

Most IDEANorth recipients reported meeting or exceeding their expected results

The following table shows results from the survey of IDEANorth funding recipients to the question "please indicate how the actual results of the project/funding compared to what you had expected". As shown, most indicated that their results had met (42%) or exceeded (36%) their expectations. No respondents indicated that their results were less than expected, however some (22%) said they did not know, or the question was not applicable.

IDEANorth funding recipients shared that their actual results surpassed their expectations for various reasons:

• Despite challenges such as staffing limitations and the need for more investment in areas like affordable charter air transportation, companies saw increased revenues and profits, and stronger balance sheets than in previous years.
• Funds helped ease significant purchases for businesses and allowed for infrastructure improvements that benefited both companies and local communities. Investments made were not just temporary but provided continuous benefits and capacity for innovation.
• The development of tools designed to assist Indigenous businesses has led to recognition and opportunities in the mining sector.
• The quality of construction and renovations funded by these grants resulted in superior facilities and innovative solutions, like improved heating systems.
• Training programs funded by grants have had a lasting impact, enabling individuals to acquire new skills and even surpass their instructors, leading to the creation of viable businesses.
• Educational programs were enriched by culturally relevant materials and the inclusion of Elders, enhancing the learning experience for students.

IDEANorth has an impact on employment and investment in the region, by generating, maintaining, and expanding jobs over the evaluation period

Funding recipients who completed a survey were asked to select from a list of impacts to indicate which occurred because of their project. A majority indicated that CanNor funding resulted in new jobs (76%), partnerships established with Northern governments (74%), investments to economic infrastructure (56%), and maintained jobs (56%). However, it is not clear if the 74% of respondents who indicated that partnerships were established with Northern governments were describing a partnership other than the one with the IDEANorth program.

Furthermore, IDEANorth projects examined through case study and interview with proponents found that the funding allowed proponents to expand businesses, purchase needed equipment, hire contractors that supported market expansion, hire new staff, and increase profitability.

Most IDEANorth project impacts can be attributed to the program funding

Funding recipients were also asked to indicate the extent to which project outcomes were attributed to CanNor funding, compared to all other factors that may have also contributed. The findings are shown in the following table, including that the majority (58%) of respondents credit the funding as being the primary source of project impacts and outcomes. No IDEANorth recipients who completed the survey said that their outcomes would have occurred in the absence of CanNor support.

IDEANorth recipients tend to report a range of intended and unintended project impacts

Among the n=13 IDEANorth recipients that said their project generated unintended impacts there were 8 mentions of positive unintended consequences and 5 mentions of negative unintended consequences, as shown in the following table. Positive impacts included increases to internal capacity and skills. Negative impacts related to higher-than-expected costs and administrative burden.

NBRF/RRRF projects were distributed across the 3 regions

As shown in the following table, NBRF/RRRF grants went primarily to for-profit companies based across the North.

The NBRF, together with the RRRF, supported 565 SMEs to maintain over 1,685 jobs

The purpose of NBRF was to respond to a timely and unique need to provide support and funding to businesses in the North to keep their doors open during the pandemic. NBRF was unique in that it provided operational funding as an eligible expense.

Survey participants were asked to describe the impact receiving the NBRF had on their business. The results are shown in the following table. Most reported positive impacts, such as remaining viable (66%) or maintaining staff or hiring new staff (54%). A minority of respondent to the survey reported negative outcomes such as losing their business or having to repay amounts received though the NBRF.

Most NBRF recipients reported meeting or exceeding their expected results

The following table shows results from the survey of NBRF funding recipients to the question "please indicate how the actual results of the project/funding compared to what you had expected". As shown, most indicated that their results had met (52%) or exceeded (21%) their expectations. Some respondents (25%) indicated that their results were less than expected.

Most NBRF project impacts can be attributed to the program funding. Recipients were more likely to credit CanNor with positive outcomes as opposed to blame the department for negative results.

Funding recipients were also asked to indicate the extent to which project outcomes were attributed to CanNor funding, compared to all other factors that may have also contributed. The findings are shown in the following table, including that many (39%) highly attribute their project results to CanNor, and most others (47%) say impacts resulted because of a combination of CanNor and other factors.

The table also shows that while 50% of those with results that exceeded expectations say the outcomes were primarily because of CanNor funding, just 24% of those with unmet expectations say their outcomes were primarily from the funding. The external factors that impacted on businesses during the pandemic were substantial in scale and scope, and this finding demonstrates that NBRF recipients who experienced negative outcomes are aware that it is not CanNor, but a range of many other factors, that is likely to blame for the outcomes they have experienced.

NBRF case study participants highly attributed the CanNor funding provided through the NBRF program as the driving factor that kept their business opened during the pandemic. Participants shared that without the funding, they would have been forced to close their doors and shut down their business, permanently. Furthermore, the NBRF funding was seen as instrumental because business owners were not required to pay back the loan if they did not turn a profit. Participants felt that not having to pay back their loan was helpful in re-opening their business, allowing them to focus on operations, rather than worrying about more expenses.

Unexpected results reported by NBRF recipients tended to relate to unexpected costs or negative financial outcomes

Of the NBRF participants that said their project generated unintended impacts, most provided negative examples. A small number provided positive examples of unintended consequences, including a reduction in emotional stress, but most others were focused on the experience of unexpected costs or negative financial outcomes, as summarized by the following three themes:

• Administrative and/or financial cost of adhering to the contract with CanNor was higher than expected (6 mentions).
• Experienced a negative financial impact because of receiving the NBRF (e.g., limited ability to receive other relief funds such as CEBA; having to pay tax on the amounts received) (5 mentions).
• Was unaware that funds may need to be paid back and experience financial strain while repaying (4 mentions).

The NICI Fund supported at least 50 projects in NWT, Nunavut and Yukon during the evaluation period.

The following table shows descriptive information for all NICI projects listed in the program's tracking sheet.

NICI projects have had an impact on local food production and supply, and the fund has been able to support innovative projects being carried out by organizations that have not historically accessed CanNor funding. However, these impacts have yet to be scaled to the degree necessary to address food insecurity.

NICI funding recipients who completed a survey shared that it is too soon to quantify the success NICI in terms of improving overall food security in northern and Indigenous communities. However, 71% of recipients (n=5) noted that individual projects were able to improve security in their local communities. Others (n=3, or 43%) said that their project resulted in the sharing of solutions related to food across the region or territories.  Those individuals were asked for more information about how results were shared:

• "We take a food ecosystem model in all our food sovereignty work, and so whenever we do a food related project, we are engaging with our relations outside of the Traditional Territory."
• "Through a report on our website and a presentation at our annual conference. A further discussion was had at our AGM."
• "Our focus was and continues to be providing foundation documents that communities and Inuit organizations can utilize in aligning of mutual objectives and bringing insight as to the technologies, we are also able to share at large with our sister organizations across the Territory."

In addition, internal CanNor representatives indicated that the Northern Food Innovation Challenge was successful in supporting communities to test and implement innovative projects. The fund provided the opportunity for new partnerships with community organizations who have not previously received CanNor funding. In addition, CanNor representatives detailed many successes related to testing and implementing innovative solutions related to food, including:

• The challenge provided an avenue for community organizations to think outside of their normal scope to come up with innovative ideas. In phase I, these ideas were meant to be ideas that could be thought about and researched. Successful projects were then moved to phase II where these innovative ideas could be implemented.
• There were three streams of funding through the NICI Fund which was seen as a success because it allowed more diversification and flexibility in spending the funding.
• Increased food production was also measured as a success of the program. Although no long-term data is available to substantiate this, participants shared that projects were able to increase food production and supply within their local communities.

They also noted challenges, including the following:

• Timelines were shared as a challenge with the NICI fund as time is needed to test and implement solutions related to food and that short timelines impede progress.
• Funding is a challenge because food security in the north is such a large issue, more funding must be made available to support these types of programs.
• Although the program is relevant and needed, that this program alone won't be sufficient to address the need in the territories.

NICI recipients report their projects were successful and exceeded or met their expectations, and attribute those results somewhat or primarily to CanNor funding

Survey respondents were asked to compare the actual results of their projects with their expectations. Most (71%) NICI recipients indicated their project results were as expected and the rest (29%) said that results were far better than expected. Most (71%) indicated that the results were due to a combination of CanNor funding and other factors, and the rest (29%) said that results were highly attributable to the funding received from CanNor.

Funding recipients who participated in a key informant interview highly attributed the impacts generated by their project to the funding provided by CanNor. These participants felt that without the CanNor funding their projects would not have occurred. The other participant felt that the impacts that occurred were the result a combination of CanNor funding and other factors. This participant shared that although CanNor funded most of the project, there were contributions from other's as well.

NICI participants agreed that there were positive unintended impacts associated with the implementation of their projects

NICI funding recipients who completed a survey were then asked about any unintended impacts, whether positive or negative. The four NICI survey participants who indicated that there were unintended impacts resulting from their project described positive impacts including interest being generated among the community and project benefits that extended beyond the initial set of stakeholders. One recipient indicated that the administrative burden of the project application and approval was higher than expected. In addition to addressing food security, funding recipients shared that the success of their project created long-term jobs. One participant reported that to sustain their business post-project, they had to hire 6 full-time staff members. Similarly, the other participant reported that their business model was so success that they created 3 full-time and 5 part-time positions, in addition to turning a profit that exceeded their expectations. Lastly, two recipients shared that a negative unintended impact of undertaking their project was related to funding. Recipients shared that the lack of long-term funding should be addressed to better support their projects.

The flexibility of funding provides the opportunity to design efficient projects that respond to unique challenges. Additional focus on strategic planning and performance measurement may increase efficiency.

Programs at CanNor are tailored to the specific challenges of northern business and have been designed to be broad in scope and eligibility, specifically because the gaps are greater, expenses are higher, and challenges are more difficult to overcome. The program allows CanNor the flexibility needed to assist and respond to unique challenges. CanNor key informants indicated that this approach supports the efficient delivery of project funds. Re-gaining delegated authority was also highlighted by internal interviewees as a strong contributor to improving efficiency of the programs. Interviewees highlighted challenges that relate to efficiency, including:

• A lack of directional investment plans for IDEANorth. Some indicated that a 5 to 10-year investment plan for each territory would improve the efficient and effective distribution of project funds.
• Lack of an appropriate IT or data management system for tracking projects, clients, and performance measurement indicators.

IDEANorth recipients indicate that most aspects of the program are efficient, easy to understand and that they were well supported by program officials. Timeliness of project approval was the biggest concern.

Survey participants were asked to indicate their level of agreement or disagreement with statements related to program operations and efficiency based on their interactions with CanNor. The statements are included in the following table, along with the proportion of respondents who agreed (strongly or somewhat), disagreed (strongly or somewhat), or provided another response (neither agree nor disagree, do not know, not applicable). The statements have been ordered according to the proportion of the recipients in agreement.

The high level of satisfaction with operational processes was also found among proponents interviewed for the case studies. Recipients shared that they felt the application process was easy to complete and reasonable in length and they appreciated the nature of the application whereby they were required to submit an expression of interest first, before moving to the next stage of project approval which required a more detailed and rigorous application. Recipients felt that this process values their time. Additionally, participants felt well supported by their CanNor official when completing their application. The timeliness of project approval and receiving funding were reported by recipients as quick and efficient. Recipients felt that their project received approval quite quickly and that the two-tiered application process facilitated this efficiency. One participant, who had a large project, felt that the approval process was also efficient although due to the size of their project their application required additional levels of approval which did delay the process somewhat. Furthermore, participants shared that there was a funding schedule laid out at the outset of their project that was accurate. Participants felt that the transparency of the funding scheduled allowed them to plan their project timelines accordingly. Furthermore, all funding recipients who participated in a case study shared that the reporting requirements were reasonable and relatively easy to complete. Participants understood the importance of reporting on project progress throughout the project, as well as submitting a final report so that officials can verify that the money provided was spent as intended. One participant noted that the only issue with reporting was that sometimes it as difficult to get information from vendors that was needed to complete their reports.

Overall, NBRF recipients indicated that most aspects of the program were efficient, however some disagreed that it was easy to understand and felt the time and effort for applying and reporting back was not appropriate.

Most NBRF recipients appeared satisfied with the efficiency of operational processes and strongly agreed or agreed with most statements asked about in the survey. However, some disagreed that the process was easy to understand (25%), that the reporting requirements were appropriate in terms of time and effort required (22%), and that the application process was appropriate in time and effort required (21%).

Funding recipients who participated in a case study shared that they felt the overall process of the NBRF program functioned efficiently agreeing that the application process, timeliness of approval, and timeliness of receiving funds were sufficient and reasonable. The exception was that participants agreed that the reporting requirements were lengthy and produced a large amount of work. Being asked to report on every individual operational expense was out of the norm for the participants and they shared that this level of reporting was quite time consuming. Although both participants notes that the reporting was a less efficient component of the program, both also agreed that they understood why the reporting needed to be detailed and that they didn't believe that there would be any way CanNor could reduce the administrative load on the participants.

NICI Fund recipients agreed that the program operates in an efficient manner and that they understand it well. However, timeliness of project approvals was noted as a major concern.

Recipients who completed the evaluation survey indicated that most aspects of the program were efficient, however nearly all (6 of 7) said that they project approval was lengthy and not appropriate. NICI recipients indicated that more time is needed to test and implement solutions related to food security and that short timelines impede progress.

However, NICI funding recipients who participated in a case study shared that overall, the steps to apply for, be approved, receive funding and reporting requirements were efficient and reasonable. With respect to the application process, participants felt that the initial expression of interest that screens the project is efficient. It is possible that issues affecting the responses of survey participants have been addressed and were not experienced by case study participants.

CanNor representatives shared thoughts about overall challenges to program efficiency.

• The Agency is lacking an efficient IT system for data collection. Currently excel spreadsheets are being used to collect and report data. Participants shared how this can create a lot of room for error.
• Client capacity. Participants shared that a lot of their clients in the territories do not have the knowledge and know-how to plan, implement or report on their budgeting process.
• COVID pandemic and the cost of doing businesses. The high cost of doing business was further enhanced by the COVID pandemic, which meant that clients inaccurately assessed how much money they would need for their projects.

Performance measurement data is collected and integrated into contribution agreements. It appears that despite issues with performance indicators and data management, performance data is used by senior management in the design and implementation of the programs.

CanNor representatives indicated that while PMIs are established and used by management, they need to be reviewed and updated because the current state does not show the true impact of the programs (n=7). Representatives shared concerns that the current PMIs do not tell the full story of the territorial economic landscape and feel that they were developed with the southern, urban landscape in mind. Furthermore, CanNor representatives shared that the current PMIs are also challenging for clients and that clients are given little guidance on how to navigate data collection (n=4). This creates problems and increases workload for both the clients and the internal staff.

In addition, internal representatives were split on whether PMIs are collected in a reliable and consistent manner. While two participants shared that they are collected well, the rest of the participants felt that there is a lot of room for improvement (n=11). Some suggestions included:

• Giving more information to clients when they are signing their contribution agreements about what is expected with data collection and reporting so that they are aware at the outset (n=4).
• Develop an IT system that is used for PMI collection and monitoring (n=3). The current system of using excel spreadsheets presents too high of a risk of errors and has limited functionality.

These concerns appear to be a continuation of those identified in the 2017 Evaluation of the Strategic Investments in Northern Economic Development, including that performance indicators may not fully capture the program's long-term impact, highlighting a need for revised project evaluation criteria and improved reporting coordination with territorial governments to better track and understand the impact of these initiatives. Related recommendations of the 2017 Evaluation included:

• Review and define the Performance Measurement Indicators so that they are more relevant to the Northern context and provide the required insights on the program's impacts/results.
• Develop a reporting template to support consistent reporting and easier tracking of SINED outputs (i.e. program results).
• Develop a database to track project outputs and outcomes (i.e. project results), by region.

The way in which some performance data is captured and recorded limits its usefulness in assessing program effectiveness and progress made towards intended outcomes.

Of note, this includes the number of jobs created, maintained or expanded as a result of IDEANorth fundings. While program data shows the expected and actual results for this category of impact, the data as reported by proponents and maintained by the program area. For example, the following table presents data collected by the program for three of the four years covered by the evaluation. However, the data does not reliably present the actual jobs created, maintained, or expanded despite appearing to do so. "Expected" values may include jobs to be created, maintained, or expanded over a multi-year project but that are reflected only in the first year of the project. "Actual" values are self reported by proponents annually. The actual values reported by project proponents reflect the number of project jobs at that moment, but do not necessarily reflect the net new amount each year. The result is an overcounting of these outcomes.