NABEP Evaluation Summary
The Evaluation
The Northern Adult Basic Education Program (NABEP) was established in 2011-12 as an education support program. NABEP is designed to help Northerners get targeted training so they can participate more fully in the labour market. The Program objective is to expand the territorial post-secondary educational institutions' basic workplace skills programs, particularly in remote communities. NABEP is also intended to improve access to adult basic education.Footnote 1 Three territorial colleges in Canada's North: Aurora College, Yukon College (now Yukon University), and Nunavut Arctic College (NAC) deliver the program.
This evaluation reviewed NABEP's relevance to federal government priorities and goals; its achievement of expected short-, medium-, and long-term outcomes; and its cost effectiveness and efficiency. It also makes recommendations for future programming. The evaluation assessed the five-year period since the last evaluation of the Program, from 2015-16 to 2019-20.
The evaluation included a document review, interviews and a case study. The evaluation team completed 28 interviews with program stakeholders. A total of 66 documents were submitted or otherwise identified by the evaluators for review. The evaluation team conducted a Curriculum Development case study.
What the Evaluation found
Relevance: The evaluation concluded that there is a continued and ongoing need for Adult Basic Education (ABE) services in the North, and that NABEP fills that gap. While skills development can be a component of ABE, it should not replace ABE, which is at the core of needs and NABEP. Improved literacy should be the core outcome of the Program. Differences across the three territories, and changes in socio-economic conditions in each territory require flexibility in design and delivery.
The program is appropriate for both the federal government and CanNor, but improvements could be made. Over the longer term, the responsibility for future ABE services should be transitioned to territorial governments. In the absence of this option, CanNor has the responsibility for continuing to fill the gap to the best of its ability.
What was recommended
The evaluation findings led to specific recommendations for improvements.
Recommendations for CanNor
1. Continue support with an improved program management approach.
2. Seek federal partners to engage territories on a more appropriate solution to long term ABE services in the North.
Recommendations for Delivery Partners
3. Nunavut Arctic College: Re-examine the drop in NAC student enrolments, including an assessment of the NAC needs assessment process.
Effectiveness: NABEP is delivered in a complex operating environment across all three territories. Each territory presents unique challenges but all have target audiences that are remote, and often vulnerable. Prospective students face a range of barriers that impacts their ability to access and complete learning opportunities. The Program requires stable funding, effective monitoring and evaluation for continuous improvement, and the flexibility to adapt to changing needs and opportunities.
The evaluation shows that progress has been made on all expected outcomes. The Delivery Partners are experienced educational institutions with the expertise to deliver the program. Program management varies across institutions. Improvements could be made to the timeliness of funding decisions, sharing of best practices and overall program improvements, and data collection and analysis for decision-making. The complex operating environment requires a clear result framework, that aligns with the scope of the Program and reflects the expected outcomes.
Recommendations for CanNor
4. Improve program management by choosing between a centralized or decentralized model, with adequate resources (quantity and expertise) to ensure accountability.
5. Improve the program results framework.
Recommendations for Delivery Partners
6. Require all partners to document their approach to curriculum development, and where appropriate, implement a collaborative, formal approach to the curriculum development process that involves Adult Educators and learners.
Efficiency: The Program has expended funding as per budgets (-6% variance overall), although Nunavut Arctic College significantly under-spent in two of the five years under review. CanNor has operated the program at a low cost but with limited program oversight and accountability, which can impact Delivery Partners in delivering services and achieving results. The timing and length of the funding agreements could be better aligned to the operational realities of delivering ABE services in the North.
Recommendations for CanNor
7. Request five-year funding periods with on-going planning supported by standardized reporting and templates such as work plans and performance indicators, within a realistic results framework.
8. Ensure resources for professional monitoring and evaluation are included in each agreement, with commitments for data collection. Third-party evaluation of each of the Delivery Partners' programs should be conducted in year 4 of any five-year cycle. CanNor should oversee the evaluations to support a comprehensive evaluationFootnote 2 of the three programs.
9. At a corporate level, improve training of CanNor management and staff on the use of the CanNor document management systems so that there can be more effective use and access to historical, program records.
Best Practices: Delivery Partners are identifying lessons learned and best practices and adopting them for continuous improvement of the program, although this has at times been interrupted by uncertain funding. Caution should be exercised when moving to online learning, as some data indicates that completion rates are lower than community-based, face-to-face learning. Online learning may not be appropriate for general application across the territories.
Recommendations for CanNor
10. Recommence the Annual Tri-Territorial Meeting with a clear focus on information sharing of best practices and lessons learned, results monitoring and evaluation, and identification of possible collaborations on curriculum/course delivery. Decisions should be recorded and reported upon. This would also be an opportunity to engage relevant federal departments (e.g., ISC, CIRNAC, ESDC) and relevant territorial departments to explore a more stable federal-territorial agreement on adult basic education for a more appropriate longer-term solution.
What was CanNor's response
CanNor partially accepted all of the recommendations and developed a response to explain how recommendations will be implemented. Additional details can be found at the following link: