Nepal Ex Kamaiya Education and Poverty Alleviation Project

Independent Final Evaluation process

Terms Of Reference

1.1 Background Information

Backward Society Education (BASE) has been implementing DFID funded project titled Nepal Ex-Kamaiya Education and Poverty Alleviation Project (NEKEPA) Since January 2013. This project has been envisioned and implemented by BASE in partnership with World Vision Advocacy Forum (WVAF). The project is intervened in mid and far- western district namely Dang, Banke, Bardiya, Kailali and Kanchanpur of Nepal.

1.2 Project Description (Aim of the Project)

The project supports over 186,000 of Nepal’s ex-kamaiya community (freed bonded laborers). In 2000, following peaceful civil society protests, the Government of Nepal abolished the kamaiya system but the ex-kamaiyas are extremely poor, without land or livelihoods. Almost 15% have not received their rehabilitation packages because most are unaware of their entitlements and because bureaucracy, inefficiency and corruption obstruct accessing them. Un- and under employment is high, with 69% experiencing income/food insecurity for at least nine months/year. Evidence suggests that 11% of children are out-of-school and likely to be in child labour.

The project aims to support ex-kamaiyas to gain their entitlements, to mobilize them for savings and credit, and to provide vocational training. The ex-kamaiyas will increase their incomes and gain productive employment; children (with a focus on girls) will be supported to attend school and complete primary education.

NEKEPA was designed by a group of staff from both BASE and WVAF, in close consultation with intended beneficiaries.  The design was influenced by recent research undertaken by BASE into the educational, health, employment and living standards of ex-Kamaiya communities (NEP Baseline Survey Report, 2011).  In the context of helping Nepal meet MDGs 1, 2 and 3, the team recognised the contribution that NEP was already making towards MDGs 2 and 3, and considered ways in which these could be improved through integrating livelihoods, leadership training and holding duty bearers to account, thus forging a convergence of activities.  Initial project concepts were shared with a small group of ex-Kamaiya leaders through BASE’s grassroots network of members living in the target areas.

DfID Nepal’s Country Plan aims to create new jobs in Nepal, of which two-thirds will be for women; make government services more transparent and accountable, particularly by supporting civil society organizations to advocate for this; and improve educational services.  The NEKEPA project supports all these strategic aims.  In the context of Nepal, the GoN School Sector Reform Plan 2009-15 places emphasis on access for out of school populations and guaranteed learning for all.  The GoN Three Year Interim Plan similarly supports further efforts to meet MDGs 1, 2 and 3, and emphasises the need for employment generation; it cites rural Mid-Western Nepal as the area of Nepal most requiring attention. The NEKEPA project aligns with all these plans.

1.3 Purpose of the independent final evaluation

The  main purpose of the independent final evaluation is to:

• Identify and verify the impact of the project.
• Meet the target Indicator as per logical framework
• Identify and Verify the data with Independent research through its annual report and logical framework
• Allow BASE to learn about what has been achieved through the project and the challenges encountered in implementation.
• Allow BASE to share lessons learnt with internal and external stakeholders.
• Allow BASE to account with local stakeholders and funders for the project’s achievements.
• Ascertain whether funds were used effectively and efficiently to deliver results (though the evaluator will not conduct a full audit).

1.4 Key objectives of the evaluation

The evaluation has two explicit objectives that are explained below:

1        To independently verify (and supplement where necessary), grantees’ record of achievement as reported through its Annual Reports and defined in the project log frame;

2                 To assess the extent to which the project was good value for money, which includes considering:

• How well the project met its objectives;
• How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;
• What has happened because of DFID funding that wouldn’t have otherwise happened; and
• How well the project aligns with DFID’s goals of supporting the delivery of the MDGs.

3. Include other learning objectives as part of the final evaluations as well but at minimum, the two objectives above should be addressed.

1.5 Verification of reporting

The first task of the final evaluation is to verify grantee achievement. The record of achievement will be presented in past Annual Reports and progress against the project logframe. This exercise could include verifying information that was collected by the grantee for reporting purposes and possibly supplementing this data will additional information collected through primary and secondary research.

Verifying the results from the project log frame will begin to capture what the project has achieved. However, there will be other activities and results that occur outside of the logframe that may require examination in order to respond to the different evaluation questions. Verifying reporting will also necessarily include a review of the data and systems that were used to populate results.

1.6 Assessment of value for money

Each final evaluation should assess the extent to which the delivery and results of the project are good value for money. Value for money can be defined in different ways, but at minimum the evaluation report should include an assessment against:

• How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;
• What has happened because of DFID funding that wouldn’t have otherwise happened; and

The NEKEPA project offers excellent value for money in three ways:

• It is a new project which builds on the ASI project; the NEP. The NEKEPA project is therefore able to take advantage of: a completed comprehensive baseline survey; trained and experienced staff; and established project systems and procedures, which would otherwise need to be initiated.  The synergy of activities which address child labour, education, legal and political rights, and income generation will support each other and result in a greater overall impact than if these activities were being addressed in isolation.  This is particularly true in relation to gender equality where a convergence-approach means that women/girls are less likely to ‘fall between the gaps’ of a single-themed approach.
• It is a project which encourages the ex-Kamaiyas to claim their entitlements and rights from the bottom-up, which is lower-cost than top-down methods and, as development experience has shown, is often more successful and sustainable.
• It is a 100% Southern-led project, being run by two local Nepali NGOs. The costs required for Nepali NGOs – particularly in relation to salaries, transport, communication etc – are significantly lower than for UK NGOs operating in the UK or Nepal. Furthermore, this approach will ensure that all DfID funding remains in Nepal, indirectly contributing to Nepal’s economy.

1.7 Evaluation questions

To ensure comparability across the final evaluation reports, the evaluator(s) should respond to the questions below. Projects are welcome to include additional questions based on their own learning needs, however this is not required. Please note that the attention given to each evaluation question may vary depending on the objectives of certain projects and the availability of data, so the independent evaluator(s) should use his/her discretion in the level of effort used to respond to these questions.

All evaluators are encouraged to structure their research questions according to the OECD-DAC criteria of relevance, effectiveness, efficiency, sustainability and impact.

Relevance

• To what extent did the grantee support achievement towards the MDGs, specifically off-track MDGs?
• To what extent did the project target and reaches the poor and marginalized?
• To what extent did the project mainstream gender equality in the design and delivery of activities (and or other relevant excluded groups)?
• How well did the project respond to the needs of target beneficiaries, including how these needs evolved over time? 
Effectiveness
• To what extent are the results that are reported a fair and accurate record of achievement?
• To what extent has the project delivered results that are value for money? To include but not limited to:

 How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;

What has happened because of DFID funding that wouldn’t have otherwise happened; and

To what extent has the project used learning to improve delivery?

• What are the key drivers and barriers affecting the delivery of results for the project? 
Efficiency
• To what extent did the grantee deliver results on time and on budget against agreed plans?
• To what extent did the project understand cost drivers and manage these in relation to performance 
requirements? 
Sustainability

To what extent has the project leveraged additional resources (financial and in-kind) from other sources? What effect has this had on the scale, delivery or sustainability of activities?

INDEPENDENT FINAL EVALUATION – GUIDANCE FOR GPAF GRANTEES

• To what extent is there evidence that the benefits delivered by the project will be sustained after the project ends? 
Impact
• To what extent and how has the project built the capacity of civil society?
• How many people are receiving support from the project that otherwise would not have received support?
• To what extent and how has the project affected people in ways that were not originally intended?

1.8     Time frame

The report should be presented to BASE in hard copy and electronic formats by 15 jan 2015.

1.9     Methodology

 Develop a mixed methodology framework and questionnaire for data collection (e.g. semi-structured interviews, focus group discussions, etc.) for approval by project partners. DFID will consider Bond’s evidence principles when reviewing candidates’ approach papers and we would like proposed methodologies to meet the principles as far as possible

• Conduct a desk based review of the project documentation and other relevant materials. For example:

Project documentation may include:

• The approved project proposal document.
• The original project log frame and the last approved project log frame.
• Annual Project Reports, including financial information.
• Any case studies produced by the project.
• Baseline studies.
• Other evidence of impact that the project team thinks is important. This could include anecdotes of decisions having been taken, policies or programmes that have changed or communication material that may have an impact on decision-making.

Other documentation may include:

• Legislation
• Government plans and policies, including the Ministry of land reform report with plan 2013 -2015, Ministry of forest report with plan 2013 -2015,Ministry of Education report with plan 2013 -2015, School Sector Reform Plan 2009-2015
• Relevant ILO reports
• Relevant reports of other bodies

• Visit at least 6 districts (out of 10) and interview a range of stakeholders including: Ex Kamaiya leader, Ex kamaiya saving credit groups member, Watchdog committee members, District Education office representative, District land and reform office representative, District forest office, District women and child welfare council, school focal teacher, former and current school children that have been assisted to access formal schools; Ex kamaiya saving credit group members and children that have received vocational training; Paralegal training, leadership and skill development training, Micro enterprises support, livelihood support, financial support to initiate agro based micro enterprises support, and advocacy development training, former and current members of Child Rights Advocacy Groups; parents; teachers, district education officers; members of watchdog committees; vocational skills providers; and programme staff in-country.
• Prepare (four) detailed case studies for at least two districts highlighting the impact of the project.
• As part of assessing the impact of advocacy relating to the project, interviewing a range of stakeholders at the district and national level (national stakeholders may include Ministry of Education, Ministry of Land reform, Ministry of Agriculture, Ministry of forest, Ministry of Women and Child welfare, BondedLabor Task Force, international and national NGOS and international agencies).
• Conduct interviews with partner staff members: DFID, WVAF and BASE.
• Present a preliminary overview of findings to partners in Nepal and DFID and receive comments from stakeholders before preparing the draft evaluation report.
• Prepare the final evaluation report in English, complete with a summary and recommendations.
• Submit a draft to BASE for written comment before finalizing the report, to minimize the chance of inaccuracies and to maximize ownership of the findings.

Point of contact throughout the evaluation will be Roshu Raj Chaudhary, Programme Coordinator-NEKEPA.

1.10   Content of the Report

 Results: Support findings with examples.

• Consider the stated outputs of the project as laid out in the most recent version of the project log frame; based on the evidence you have collected and analyzed, to what extent have each of the outputs been delivered and targets/milestones reached as envisaged?
• What were the results of the individual outputs? What changes were brought about?
• Taken together, to what extent did the outputs achieve the desired outcome?

Beneficiaries: Support your findings with examples and recommendations.

• Which target groups did the project work with? How did they benefit?
• What is your evidence that the project reached the intended target group(s) as described? For example, specify any sample surveys used in the evaluation, numbers of people interviewed etc.
• Did the project seek input from beneficiaries? At what stages? Describe how this input was sought. Are there any examples of how the project changed course as a result of feedback from beneficiaries?
• Are there other children in the community that suffer from lack of access to education due to forced labor and slavery like practices that did not form part of the target group? Would it be possible to include these groups in the future if the project was able to continue?

Impact: Support your findings with examples and recommendations.

• How did the project identify and assess impact?
• What evidence is there of impact on the lives of the Haliya and Kamaiya as a result of this project?
• Are the changes relevant and favorable to the communities’ needs in relation to both education and overall eradication of slavery-like practices?
• Identify any gaps in the project that has resulted in the community’s needs/issues not being tackled comprehensively.
• Have there been any others changes in the community that are related to the project? How are these changes manifesting? (E.g. have occurrences of early marriage, child labor decreased?)
• Have there been any changes in policy, practice, attitude of decision makers – at both national, district and local levels – which have benefitted the communities? How are these changes manifesting? Assess the extent to which these changes are a result of the impact of advocacy?
• What are the advocacy needs and opportunities going forward?

Unintended consequences:  Support your findings with examples and recommendations.

• Explain if the project produced any unanticipated consequences or outcomes that were not intended? For example: Were there any positive unexpected benefits?  Did something negative happen as a result of the project?
• Were these learning’s documented and shared?

 Programme management: identifying effective methods

• What was the overall theory of change for this project? Was it suitably effective in bringing about lasting change in this area or are there gaps in this process?
• What were the major challenges and gaps in the planning and implementation of the project?
• How effective has project management been?
• Assess the quality of monitoring undertaken throughout this project: were regular monitoring tools easy to use and did they capture all relevant information?
• To what extent were the indicators for measuring outcome appropriate?
• How could monitoring be improved upon?
• Was communication between partners sufficient and appropriate? If not, how could this be improved upon?
• Assess whether the working relationship between partners, including whether relationshipshave helped or hindered the delivery of outcomes and lasting changes. Assess any positive or negative impacts.
• What have been the most effective methodologies and approaches used by all partners to bring about the changes in peoples’ lives? What has worked and what has not and why?
• Did the project identify and manage risk effectively?

Value for money

• Have there been any financial issues, which have hindered the delivery of the programme?
• Has the project been cost effective, delivering value for money? Consider the below:

Economy

What has the project management done to buy and use inputs at a value-for-money price?  What did the organization do to drive down unit costs but maintain quality?

Efficiency

How did the project ensure that resources (inputs) were used efficiently to maximize results?

Effectiveness

Do you consider the project has been effective in bringing about the anticipated changes for beneficiaries and target groups?

Equity

To what extent have the project’s services been made available to/reached all the people that they are intended to?

Gender and social Inclusion

To what extent have the project’s services been made available to reach all the people?

Monitoring, Evaluation and Learning

• What is the key lesson learned from this project? How were these lessons identified by partners, and could this process be improved upon?
• How have lessons been documented and shared? Have any parts of the project changed in light of lessons learnt along the way? Suggest any ways that this process (identifying, sharing, implementing learning’s) could have been improved upon.
• Was the mid-term evaluation helpful? Were the recommendations of the review implemented, and if so, did this have a positive impact on the project?

Capacity building/Empowerment and Advocacy

• Assess if the capacity of partners increased throughout the project term.
• Provide an assessment of partners’ capacity needs going forward.

• Please assess whether other stakeholder’s capacity was built throughout the project and how. Identify if the beneficiary found the capacity building useful, and how this capacity building led to change in the project.

Examples include:

• End-beneficiaries
• Local leaders or change agents
• Other Civil Society Organizations / networks
• Local community-based organizations
• Local government
• National government
• Other (please specify)

Challenges and enablers

Summarize in bullet points the main challenges faced by the project as well as things that helped it along the way.

 Sustainability and scaling up

• In what ways has the project ensured sustainable outcomes?
• Does the project have a clear strategy for ensuring sustainability beyond DFIDs funding?
• In what ways has the project contributed to the achievement of broader national goals of “Education for All by 2015”? (see the School Sector Reform Plan 2009-2015)  Considering this plan and the additional government education plan for post-2015, is there a way to ensure the project is sustainable in the long-term by using government initiatives?
• Do elements of this project require future funding? Which aspects of the project do stakeholders wish to continue with, and why? Has funding been secured?
• Please assess the need, if any, for a follow-on or related project and what the objectives of such a project should be. Please link to impact (particularly the overall aim of eradicating slavery like practices), community needs not addressed, etc.

 Recommendations

Summarize five to ten key recommendations to aid future programming by DFID.

1.11   Specification of the Consultant

Senior consultant

Essential selection criteria include:

• Extensive experience in evaluation
• Knowledge of bonded labor and forced labor
• Work experience on the issue of bonded and/or child labor, or more broadly in human rights
• Experience of participatory approaches in general, and specifically with children, Education, Exkamiaya and their rehabilitation process
• Fluent in English
• Independent of all partners
• Fluent in Nepali (at least one member of the team must be fluent)

 Desirable selection criteria include:

• Specific knowledge about bonded labor in Nepal and related legislation and policy
• Specific work experience on the issue of bonded and/or child labor/education in Nepal

Junior consultant

Essential selection criteria include:

• Fluent in Nepali (at least one member of the team must be fluent)
• Fluent in English
• Knowledge of bonded labor in Nepal and related legislation and policy
• Work experience on the issue of bonded and/or child labor in Nepal
• Experience in evaluation
• Independent of all partners

Desirable selection criteria include:

• Experience of participatory approaches in general, and specifically with children