Save the Children’s Health Program in Timor
Leste
Terms of Reference
Independent Consultant to conduct ‘end of
program’ evaluation
This Terms of Reference (TOR) sets
out the requirements to conduct the end of program evaluation of the Save the
Children Health Program in Timor-Leste which is funded by AusAID. An
independent consultant will be contracted on behalf of Save the Children to
carry out the tasks described in this TOR.
1.
Background Country background: The 2009 Human
Development Index (HDI) ranks Timor Leste 162 out of 182 countries and the low
overall living standards are reflected in the average life expectancy of 59.7
years. Although there have been major developments in the country there are
still major obstacles to overcome to improve health outcomes for the population,
including children who are most vulnerable.
Improving the health
situation of women and children is a priority for Timor Leste and the country is
committed to achieving the Millennium Development Goals (MDG) by 2015. The key
MDGs selected for the Save the Children (SC) Health Program are MDG 4, reducing
child and infant mortality and MDG 5, reducing maternal mortality.
The
rate of child mortality in Timor Leste is 64 deaths per 1000 live births; the
country aims to reduce this to 53 deaths per 1000 live births by 2015 (MDG 4).
For maternal mortality (MDG 5), the current estimate is 557 death per 100,000
live births, and the aim is to reduce this to 252 by 2015. Although there has
been progress over the past seven years, there remains a gap between the trend
demonstrated in current indicators and 2015 targets. SC’s Health Program aims to
contribute to achieving these targets and permit women and their children their
right to life, as per article 6 in the Convention on the Rights of the
Child.
Overview of the Health Program of Save the Children in
Timor Leste:
Save the Children’s vision is a world in which every
child has a healthy and safe childhood, the opportunity to learn, and a voice to
speak for them. Save the Children’s (SC) Health Program in Timor Leste aims to
reduce the morbidity and mortality among East Timorese newborns and their
mothers in alignment with their rights.
In so doing, SC is contributing
to the Millennium Development Goals (MDG) through reducing child and maternal
mortality. Infants, young children and their mothers are the main focus of the
Program. A primary health care approach focusing on maternal, newborn and child
health is being implemented including behavior change communication (BCC),
community mobilisation, and health systems strengthening. The program is in the
implementation phase covering 8 sub districts of two rural and remote
mountainous districts of Timor-Leste (Ainaro and
Manufahi).
Implementation of the program is through partnerships with:
o local NGOs (Alola Foundation, Health Net and Sharis Haburas Comunidade
(SHC)),
o the Government (Ainaro and Manufahi District Health Services)
and
o Community Based Organizations (Mother Support Groups (MSG) and
Children’s Clubs (CC))
This model provides an integrated approach to
improving maternal and child health outcomes.
Program objectives are
outlined below:
a) Improve the nutritional status of pregnant and
lactating women, and children under 2 years (through improved breastfeeding and
appropriate complimentary feeding strategies);
b) Increase level of
knowledge and skills of target communities regarding aspects of healthy living
(i.e. nutrition, hygiene, physical wellbeing, free from communicable diseases
etc.);
c) Improve practices of birth preparedness of caregivers
including strengthening referral mechanism for complicated pregnancies;
d) Strengthen the capacity of district health systems including
community based health workers to provide quality primary health care services;
and
e) Strengthen Village Health Committee members to enable them to
monitor and advocate for quality health services delivery at the community
level.
2. Purpose of Evaluation This
program requires the services of an independent consultant to undertake an end
of program evaluation to assess the quality of implementation and outcomes, and
make recommendations for future programming.
Save the Children will be
withdrawing operations from Timor-Leste in September 2013 and the Health Program
evaluation will be an important part of SC reflecting on success, challenges and
lessons learned. SC will feed this learning into future SC health programming
and into the Timor health sector analysis and programming.
This
evaluation will also be intricately linked with an overall program learning
review which will be contribute to a final three year learning publication.
This health evaluation is to be a learning process for SC and partner
staff, with an emphasis on building the capacity of SC and partner staff
regarding data collection, collation, analysis, and interpretation
skills.
AusAID requires this evaluation as part of their ANCP
performance reporting process – and during the in-country component, initial
findings will be presented to AusAID and other relevant stakeholders in Dili,
Timor-Leste.
3. Objectives and key evaluation
questions
The evaluation is to achieve the
following:
a) Assess the extent of achievements of program
objectives (what was the outcome/ impact of the program);
b) Assess the
quality of program implementation including evaluating partnership processes and
the overall management of program activities; and
c) Provide lessons
learned and recommendations for future programming for SC and the health
sector.
Key evaluation questions will be developed with the team members, as
part of the evaluation capacity building focus.
4.
Scope of evaluation The scope of the evaluation will
include:
a) Desk review of the Health program design documents,
M&E frameworks, financial budgets, annual plans, past reports by
SC;
b) Developing an evaluation process plan that will effectively
evaluate program outcomes and impact, program effectiveness, and quality of
partnership and management.
c) Developing a plan for data collection
and analysis with the methodology using a mix of quantitative and qualitative
methods, and encouraging beneficiary & stakeholder participation (including
field visits to program sites (Ainaro and Manufahi));
d) Providing a
written evaluation report at the conclusion of the evaluation review outlining
findings, learning and recommendations;
e) Presenting these initial
findings with SC and other key stakeholders in Dili, Timor
Leste.
5. Evaluation design and
methodology
This evaluation should use both quantitative and
qualitative methods consisting of structured questionnaires, observations,
semi-structured in-depth interviews (IDIs), focus group discussions (FGDs) and
any other relevant tools.
Subjects to be involved (e.g. interviewed,
complete questionnaire) during the evaluation will include: SC health staff,
local NGO implementing partner staff, government health staff, health volunteers
(PSF), health committee (PDSS) members, direct beneficiaries of the health
program, caregivers of children under-2, school children, community leaders
(Chefe Sucos & Aldeia), and other stakeholders like INGOs, national NGOs and
CBOs working in the sector and/ or in the geographical locations.
Where possible and appropriate, the evaluation tools (questionnaires
etc.) should be finalised and translated into Tetun (with support from the
evaluation team members).
6. Organisation, roles
and responsibilities
The evaluation team will be led by the
independent consultant and will comprise of SC and partner NGO team members.
Team members |
Independent Consultant | |
TEAM LEADER |
Health Program Manager (SC) | |
Team member |
Health Program M&E Officer (SC) | |
Team member |
Senior Health Officer Ainaro (SC) | |
Team member |
Senior Health Officer Manufahi (SC) | |
Team member |
Health Program Community Development Officer (SC) | |
Team member |
Partner NGO representatives (Alola, Healthnet & SHC) | |
Team member |
Child Rights & Governance Program Manager (SC) | |
Team member overall program |
Senior Coordinator Child Rights & Governance (SC)
| |
Team member overall program |
Program Director (SC) | |
Team member overall program
|
The evaluation team and other
program staff will travel with the independent consultant to the field. There
may be some overlapping themes and questions etc. with target groups being
interviewed. I.e. it is envisaged that there will be some interviews,
questionnaires, focus group discussions etc. that are conducted to collect data
for both evaluations (health and the broader overall program one) – but the
health evaluation will be of a deeper nature and have a stand-alone analysis and
report. The independence of the health evaluation and consultant will be
respected, but the consultant should be aware that the health evaluation is part
of a larger reviewing process.
The team will undertake the following
tasks:
a) A desk review of program design documents, reports, budgets,
M&E framework, Demographic health survey, and other relevant
documents;
b) Developing an evaluation process plan that outlines and
summarizes the data collection, collation and analysis methodology including
reporting and presentation;
c) Travelling to the district field to
visit program locations;
d) Conducting the actual review/ evaluation
questions and other identified tools
e) Provide final evaluation report
and presentation to key stakeholders and including recommendations for future
programming.
Defining the differential roles
Specific roles of
the independent consultant:· Desk review of the relevant
project documents
· Develop evaluation plan and questionnaires etc.
with evaluation team members
· Facilitate training for the data
collectors
· Facilitate stakeholder workshops as relevant (Dili and
district based)
· Guide and supervise the data collectors to carry
out their respective activities
· Conduct data analysis with the team
members, identifying key issues, trends, outcomes and learning
· Take
the lead in drafting the summary report on the evaluation
· Present
the study purpose, methodology, findings, and recommendations to Save the
Children and other stakeholders (including AusAID) at the national level
· Finalise the evaluation report, incorporating stakeholder
feedback
The specific roles of the team
members:· Accompany the independent consultant and
participate fully in the evaluation
· Provide insight into the SC
health program
· Participate in tasks and assessment as required by
the independent consultant
· Provide inputs into the evaluation
planning and implementation
7.
Deliverables
The Evaluation team led by the Independent Consultant
must provide the following:
a)
Evaluation Plan and
MethodologyThe evaluation plan must address program logic (log frame
and objectives etc.); key evaluations questions and outline the proposed
methodology. The methodology must be culturally appropriate for the target
audience.
b) Final Report The independent
consultant must provide a detailed final report to SC and key stakeholders and
include the following information:
o Executive Summary
o Program
Description
- Program background and rationale
- Program goal,
objectives and outputs (logical framework)
- Cross cutting
themes
- Key stakeholders
o Program relevance analysis
o
Output and Activity analysis
o Outcome and Impact analysis
o Program
management analysis (partnership and program management)
o Sustainability
o Budget and expenditure variance report
o Case study
compilation
o Lessons Learned
o Recommendations
o
Conclusion
c) PresentationPresent final report
to SC, AusAID and other key stakeholders
8.
Timeline
The in-country component of the
health evaluation will be for a period of 24 days (to be completed within 30
days) from the date of arrival in Dili, Timor-Leste. Evaluation needs to start
on 8 July, 2013.
Days | |
Activity |
5 days | |
In-country desk review of all relevant documents, develop evaluation plan in
collaboration with Save the Children staff. |
1 day | |
Meet with stakeholders in Dili (partners and Ministries) to conduct
workshop/ FGD about overall program activities, outputs, outcomes.
|
1 day | |
Travel to field (Dili to Manufahi) if the plane is available then travel
will be one hour instead on one day |
3 days | |
Manufahi Visit project sites, collect data through consultation/ focus
group discussions with district stakeholders (including partners/ district
health services/ beneficiaries). |
1 day | |
Travel in field (Manufahi to Ainaro) about three hours
|
3 days | |
Ainaro Visit project sites, collect data through consultation/ focus group
discussions with district stakeholders (including partners/ district health
services/ beneficiaries). |
1 day | |
Travel from field (Ainaro to Dili) if the plane is available then travel
will be one hour instead on one day |
2 days | |
Analysis workshop in collaboration with Save the Children staff and
representative from partners. |
3 days | |
Write up report. |
1 day | |
Presentation of briefings to ministries, partners and other key
stakeholders. |
3 days | |
Finalisation of report
|
9.
Budget/Resources
Total budget for this evaluation is US$ 14,000
which will include independent consultant fee, his/her accommodation, and any
per diem and travel costs.
10. Plan for
dissemination and learning The evaluation report will be
disseminated among key stakeholders including community members, partner NGOs,
nutrition working group, Government departments (national and district level)
and other stakeholders working in the Health sector.
The Health Program
Evaluation report will be a component of a larger review of all SC programs and
activities in Timor-Leste; so the main points from this health program
evaluation will be captured in the final overall SC program review
publication.
------------------------------
------------------------------
------------------------------
------------------------------
---
Competency
and expertise required: We are interested in person with strong
field-based research skills (both quantitative & qualitative data
collection, analysis and reporting) and experience conducting research and
evaluation in the health sector, especially in the areas of Nutrition and
Maternal & Child Health (MCH) care. The consultant must be committed to
working with national staff as this evaluation will be a capacity development
experience for some of the staff. Proficiency in Tetun is preferred.
Day to day liaison:
Joydip Ghosh, Health program
Manager, Save the Children in Timor
Leste
------------------------------
------------------------------
------------------------------
------------------------------
---
Interested
candidates are requested to apply for this position with their CVs, capability
statements, list of research work and publications, and references.
Applications must be submitted by 31 March, 2013. Please send all
correspondence and supporting documents by email to Joydip Ghosh:
joydip.ghosh@savethechildren.org.au
