Evaluation Questions. The following is a list of indicative and generic evaluation questions. A more specific set of evaluation questions shall be drafted and validated during the inception phase of the evaluation, especially taking account of the results described in ‘COVID delays mitigation’ above, so that the external evaluation efforts are complementary to those already evident from the project progress reports produced in December 2022. The CLIMWAR project (started 2017) also provided an extensive report, including self-assessment, at the end of 2022. In line with the main purpose presented above, the evaluation will focus on the following two main aspects. ⮚ Adequacy of ▇▇▇▇ Governance, management and coordination mechanisms: • In what ways could the deployment of resources under ▇▇▇▇ be optimized to enhance the potential impacts of the invested financial resources (e.g., ensuring complementarities with other UNESCO donors in order to multiply effects and avoid duplications)? (Question AC-1) • How effective and efficient are the governance, coordination, advisory, management frameworks established under the trust fund by the two sides? This includes the meetings of the Steering Committee and bilateral consultations between UNESCO’s relevant services and the Flemish Government (in particular with BSP, the Natural Science Sector, IOC and concerned field offices) and the project selection process in Flanders. (Question AE-1) ⮚ Assessment of projects funded under the Trust Fund, in particular the following selected Large-Scale projects: OTGA 2, OIH, PACMAN, CLIMWAR and BE- RESILIENT - Relevance: • To what extent was the timeframe, the geographic coverage and thematic coverage of the projects adequate within the context of the overall programmes? (Question R-1) • What is the current and future potential of the projects to contribute to relevant UNESCO’s/ the Flemish government policies in the light of the 2030 Agenda? (Question R-2) • To what extent did the selected projects meet stakeholder and beneficiaries’ needs in consideration of regional, national and basin scale (local) priorities? (Question R-3) Has there been particular attention to consideration of how scientific and science-based activities can be true enablers, how the projects related to specific area-based disadvantaged groups, indigenous peoples, and any social and environmental concerns specific to the project and, more generally, of gender equality? (Question R-4) • How has the support provided through ▇▇▇▇ complemented/reinforced UNESCO’s programmes, in connection with the contribution of these programmes to strengthening UNESCO’s C/4 (medium term strategy) and C/5 (programme and budget)? (Question C- 1) - Efficiency: • Have the selected projects produced the outputs in the project documents in a timely manner (considering delays due to the COVID-19 pandemic)? (Question E-1) • Were the human and financial resources used efficiently? (Question E-2) • Were opportunities seized to develop potential synergies with relevant networks and partners? (Question E-3) • To what extent have partnerships and cooperation with implementing partners been effective? (Question E-4) - Effectiveness/Signs of Impact: • To what extent have expected results for the selected projects been achieved? (Question I-1) What were the key enablers and key challenges for such achievements? (Question I- 2) • Was an adequate monitoring framework/methodology put in place in order to achieve the expected results and maximize the success of the projects? (Question I-3) • If there were differences between the real and the expected outputs, were these discussed between UNESCO and the Government of Flanders? (Question I-4) • What have been the longer-term effects of the projects within the respective institutional, country, regional and international contexts (including potential for replication and multiplier effects)? (Question I-5) - Sustainability: • What mechanisms (including in terms of communication) have been put in place to ensure that the projects and/or their effects are sustainable over time and or scaled up/ replicated elsewhere? (Question S-1) • In what ways have the project activities (and their outputs and effects) contributed to the visibility of the selected projects, of Flanders and of UNESCO? (Question S-2) • Did the project lead to new opportunities for international scientific cooperation, and if so, did this involve scientists from Flanders? (Question S-3) Specific questions related to the selected large-scale projects to be examined by the evaluators are included as annex of the present ToRs. In responding to the main evaluation questions presented in this chapter, the evaluation team will need to consider the indicative list of questions in the annex (as sub-evaluation questions or suggested questions to be included in their data collection tools).
Appears in 1 contract
Sources: Trust Fund Agreement
Evaluation Questions. It is expected that the evaluation questions will be guided by the OECD DAC criteria for evaluating development assistance, which are: relevance, efficiency, effectiveness, impact and sustainability. The following is a additional criteria of coverage and inclusivity are also relevant here, given the results promised for women. The final list of indicative and generic evaluation questions. A more specific set of evaluation questions shall will be drafted and validated agreed during the inception phase of the evaluationevaluation manager’s assignment. However, especially taking account of the results described in ‘COVID delays mitigation’ above, so it is anticipated that the external evaluation efforts are complementary to those already evident from the project progress reports produced in December 2022. The CLIMWAR project (started 2017) also provided an extensive report, including self-assessment, at the end of 2022. In line with the main purpose presented above, the evaluation will focus on address the following two main aspects. ⮚ Adequacy of ▇▇▇▇ Governance, management questions: How effective was the programme as a whole in delivering the promised outcomes and coordination mechanisms: • In what ways could impacts? (Impact in the deployment of resources under ▇▇▇▇ be optimized DAC criteria) Were the outcomes and impacts delivered relevant to enhance beneficiaries needs? (Relevance in the potential impacts of DAC criteria) Were the invested financial resources (e.g., ensuring complementarities interventions really catalytic? How did the interventions interact with other UNESCO donors factors in order to multiply effects the local and avoid duplications)regional economy? (Question AC-1Impact and sustainability in the DAC criteria) • How effective and efficient are were individual interventions in delivering the governanceintended outcomes? If some interventions were more successful than others, coordination, advisory, management frameworks established under why? Were these the trust fund by the two sides? This includes the meetings right combination of the Steering Committee and bilateral consultations between UNESCO’s relevant services and the Flemish Government (in particular with BSP, the Natural Science Sector, IOC and concerned field offices) and the project selection process in Flanders. (Question AE-1) ⮚ Assessment of projects funded under the Trust Fund, in particular the following selected Large-Scale projects: OTGA 2, OIH, PACMAN, CLIMWAR and BE- RESILIENT - Relevance: • To what extent was the timeframe, the geographic coverage and thematic coverage of the projects adequate within the context of the overall programmesinterventions? (Question R-1Impact, effectiveness and efficiency in the DAC criteria) • What How effective is the current market system analysis approach in delivering more relevant interventions that improve incomes and future potential of the projects to contribute to relevant UNESCO’s/ the Flemish government policies reduce poverty? (Impact, effectiveness, relevance in the light of the 2030 Agenda? (Question R-2DAC criteria) • To what extent did the selected projects meet stakeholder programme’s interventions stimulate the continuation in the market of exiting players and beneficiaries’ needs the entry of other players into the market in consideration of regionala long term, national positive and basin scale (local) prioritiessustainable way? (Question R-3Sustainability in the DAC criteria) Has How effective was the programme in meeting its results promised for women and girls, who were only 7% of beneficiaries in PropCom 1? such as: What kinds of interventions were most effective in increasing the incomes of women and girls? Were there been particular attention to consideration trade-offs between the aims of how scientific and science-based activities can be true enablers, how increasing the projects related to specific area-based disadvantaged groups, indigenous peoplesparticipation of poor people in markets, and any social increasing the incomes of women and environmental concerns specific to the project and, more generally, of gender equalitygirls specifically? (Question R-4Impact, effectiveness, efficiency and coverage and inclusivity in the DAC criteria) • How has What is the support provided through ▇▇▇▇ complemented/reinforced UNESCO’s programmes, in connection with effect of insecurity on the contribution of these programmes to strengthening UNESCO’s C/4 (medium term strategy) and C/5 (programme and budget)interventions? (Question C- 1) - Efficiency: • Have the selected projects produced the outputs Sustainability in the project documents in a timely manner (considering delays due to OECD DAC criteria) What is the COVID-19 pandemic)effect of climatic and food shocks on the programmes interventions? (Question E-1) • Were Sustainability in the human and financial resources used efficiently? (Question E-2) • Were opportunities seized OECD DAC criteria)
2.1.2 The evaluation is also expected to develop potential synergies with relevant networks and partners? (Question E-3) • To what extent have partnerships and cooperation with implementing partners been effective? (Question E-4) - Effectiveness/Signs of Impact: • To what extent have expected results for the selected projects been achieved? (Question I-1) What were the key enablers and key challenges for such achievements? (Question I- 2) • Was provide an adequate monitoring framework/methodology put in place in order to achieve the expected results and maximize the success analysis of the projects? (Question I-3) • If there were differences between the real and the expected outputs, were these discussed between UNESCO and the Government of Flanders? (Question I-4) • What have been the longer-term effects of the projects within the respective institutional, country, regional and international contexts (including potential for replication and multiplier effects)? (Question I-5) - Sustainability: • What mechanisms (including portfolio composition in terms of:
1. Spread of communication) have been put in place to ensure that the projects and/or their effects are sustainable over time and or scaled up/ replicated elsewhere? (Question S-1) • In what ways have the project activities (and their outputs and effects) contributed to the visibility of the selected projects, of Flanders and of UNESCO? (Question S-2) • Did the project lead to new opportunities risk
2. Value for international scientific cooperation, and if so, did this involve scientists from Flanders? (Question S-3) Specific questions related to the selected largemoney/cost-scale projects to be examined by the evaluators are included as annex of the present ToRs. In responding to the main evaluation questions presented in this chapter, the evaluation team will need to consider the indicative list of questions in the annex (as sub-evaluation questions or suggested questions to be included in their data collection tools).benefit analysis
Appears in 1 contract
Sources: Call Down Contract