Monitoring
Monitoring is the systematic and routine collection of information from projects and programmes for four main purposes:
- To learn from experiences to improve practices and activities in the future;
- To have internal and external accountability of the resources used and the results obtained;
- To take informed decisions on the future of the initiative;
- To promote empowerment of beneficiaries of the initiative.
Monitoring is a periodically recurring task already beginning in the planning stage of a project or programme. Monitoring allows results, processes and experiences to be documented and used as a basis to steer decision-making and learning processes. Monitoring is checking progress against plans. The data acquired through monitoring is used for evaluation.
Types of Monitoring
Monitoring can take various forms including:
- Results monitoring which tracks whether a project is on target towards its intended results (outputs, outcomes, impact) e.g., Have all the children in a target community actually been immunised?
- Process (activity) monitoring tracks whether inputs and resources have been used efficiently. For example, have the correct medicines been delivered on schedule to the correct medical unit?
- Compliance monitoring checks compliance with regulations, contract requirements, laws, and ethical standards. For example, has any electrical equipment that is being donated been properly tested?
- Context (situation) monitoring tracks the setting in which the project operates, especially insofar as this poses a risk to the project. For example, how might civil unrest in a neighbouring village impact upon the outcomes of your project or lead you to change the assumptions on which it is based?
- Financial monitoring focuses on costs and ensures implementation is within budget. Have school uniforms been purchased and delivered at the projected price?
- Organisational monitoring tracks institutional aspects of a project’s implementation: how have project partners worked together and communicated with each other?
- Beneficiary monitoring tracks beneficiary perceptions of a project. It includes satisfaction or complaints with the project and partners. What benefit have they perceived from the digging of a village well?
In general, monitoring is integral to evaluation. During an evaluation, information from previous monitoring processes is used to understand the ways in which the project or programme developed and stimulated change. Monitoring focuses on the measurement of the following aspects of an intervention:
- On quantity and quality of the implemented activities (outputs: What do we do? How do we manage our activities?)
- On processes inherent to a project or programme (outcomes: What were the effects /changes that occurred as a result of your intervention?)
- On processes external to an intervention (impact: Which broader, long-term effects were triggered by the implemented activities in combination with other 10 types of evaluation to put you on the right track
Evaluation
Evaluation is assessing, as systematically and objectively as possible, a completed project or programme (or a phase of an ongoing project or programme that has been completed). Evaluations appraise data and information that inform strategic decisions, thus improving the project or programme in the future. The evaluation process is thus, an analysis or interpretation of the collected data which delves deeper into the relationships between the results of the project/programme, the effects produced by the project/programme and the overall impact of the project/programme.
Evaluations should help to draw conclusions about five main aspects of the intervention:
- relevance
- effectiveness
- efficiency
- impact
- sustainability
Information gathered in relation to these aspects during the monitoring process provides the basis for the evaluative analysis.
The evaluation process is an analysis or interpretation of the collected data which delves deeper into the relationships between the results of the project/programme, the effects produced by the project/programme and the overall impact of the project/programme.
Types of Evaluation
· Formative evaluation: This is generally conducted before the project implementation phase. But depending on the nature of the project, it may also continue through the implementation stage. Its main purpose is to generate baseline data to investigate the need for the project, raise awareness of the initial project status, identify areas of concern and provide recommendations for project implementation and compliance.
· Process evaluation: It is conducted as soon as the project implementation stage begins. It assesses whether the project activities have been executed as intended and resulted in certain outputs. Process evaluation is useful in identifying the shortcomings of a project while the project is still ongoing to make the necessary improvements. This also helps to assess the long-term sustainability of the project.
· Outcome evaluation: This type of evaluation is conducted once the project activities have been implemented. It measures the immediate effects or outcomes of the activities in the target population and helps to make improvements to increase the effectiveness of the project.
· Summative evaluation: This occurs immediately after project conclusion to assess project efficacy and the instant changes manifested by its interventions. Summative evaluation compares the actual outcome data with baseline data to determine whether the project was successful in producing the intended outcomes or bringing about the intended benefits to the target population. It provides evidence of project success or failure to the stakeholders and donors to help them determine whether it makes sense to invest more time and money for project extension.
· Impact evaluation: Impact evaluation assesses the long-term impact or behavioral changes as a result of a project and its interventions on the target community or population. It assesses the degree to which the project meets the ultimate goal, rather than focusing on its management and delivery. These typically occur after project completion or during the final stage of the project cycle. However, in some longer projects, this can be conducted in certain intervals during the project implementation phase, or whenever there is a need for impact measurement.
· Real-time evaluation: Real-time evaluation is undertaken during the project implementation phase. It is often conducted during emergency scenarios, where immediate feedback for modifications is required to improve ongoing implementation. The emphasis is on immediate lesson learning over impact evaluation or accountability.
· Participatory evaluation: This type of evaluation is conducted collaboratively with the beneficiaries, key stakeholders and partners to improve the project implementation. Participatory evaluation can be empowering for everyone involved as it builds capacity, consensus, ownership, credibility and joint support.
· Thematic evaluation: Such type of evaluation focuses on one theme across a number of projects, programs or the whole organisation. The theme could be anything, ranging from gender, migration, environment etc.
· Cluster or sector evaluation: Just as the name suggests, this evaluation is implemented by larger development and humanitarian sectors, including a group of different organisations, programs or projects that are working on similar thematic areas. It assesses a set of interconnected activities across different projects and entities. As a result, it strengthens partnerships within these key sectors, while improving their coordination, accountability, predictability, and response capacity.
· Meta-evaluation: This is used to assess the evaluation process itself. Meta-evaluations could be useful to make a selection of future evaluation types, check compliance with evaluation policy and good practices, assess how well evaluations are utilized for organizational learning and change, etc.
What is the difference between Monitoring and Evaluation?
Monitoring & Evaluation
M&E is an embedded concept and constitutive part of every project or programme design (“must be”). M&E is not an imposed control instrument by the donor or an optional accessory (“nice to have”) of any project or programme. M&E is ideally understood as dialogue on development and its progress between all stakeholders.
In general, monitoring is integral to evaluation. During an evaluation, information from previous monitoring processes is used to understand the ways in which the project or programme developed and stimulated change. Monitoring focuses on the measurement of the following aspects of an intervention:
- On quantity and quality of the implemented activities (outputs: What do we do? How do we manage our activities?)
- On processes inherent to a project or programme (outcomes: What were the effects /changes that occurred as a result of your intervention?)
- On processes external to an intervention (impact: Which broader, long-term effects were triggered by the implemented activities in combination with other 10 types of evaluation to put you on the right track
1. Formative evaluation: This is generally conducted before the project implementation phase. But depending on the nature of the project, it may also continue through the implementation stage. Its main purpose is to generate baseline data to investigate the need for the project, raise awareness of the initial project status, identify areas of concern and provide recommendations for project implementation and compliance.
2. Process evaluation: It is conducted as soon as the project implementation stage begins. It assesses whether the project activities have been executed as intended and resulted in certain outputs. Process evaluation is useful in identifying the shortcomings of a project while the project is still ongoing to make the necessary improvements. This also helps to assess the long-term sustainability of the project.
3. Outcome evaluation: This type of evaluation is conducted once the project activities have been implemented. It measures the immediate effects or outcomes of the activities in the target population and helps to make improvements to increase the effectiveness of the project.
4. Summative evaluation: This occurs immediately after project conclusion to assess project efficacy and the instant changes manifested by its interventions. Summative evaluation compares the actual outcome data with baseline data to determine whether the project was successful in producing the intended outcomes or bringing about the intended benefits to the target population. It provides evidence of project success or failure to the stakeholders and donors to help them determine whether it makes sense to invest more time and money for project extension.
5. Impact evaluation: Impact evaluation assesses the long term impact or behavioral changes as a result of a project and its interventions on the target community or population. It assesses the degree to which the project meets the ultimate goal, rather than focusing on its management and delivery. These typically occur after project completion or during the final stage of the project cycle. However, in some longer projects, this can be conducted in certain intervals during the project implementation phase, or whenever there is a need for impact measurement.
6. Real-time evaluation: Real-time evaluation is undertaken during the project implementation phase. It is often conducted during emergency scenarios, where immediate feedback for modifications is required to improve ongoing implementation. The emphasis is on immediate lesson learning over impact evaluation or accountability.
7. Participatory evaluation: This type of evaluation is conducted collaboratively with the beneficiaries, key stakeholders and partners to improve the project implementation. Participatory evaluation can be empowering for everyone involved as it builds capacity, consensus, ownership, credibility and joint support.
8. Thematic evaluation: Such type of evaluation focuses on one theme across a number of projects, programs or the whole organisation. The theme could be anything, ranging from gender, migration, environment etc.
9. Cluster or sector evaluation: Just as the name suggests, this evaluation is implemented by larger development and humanitarian sectors, including a group of different organisations, programs or projects that are working on similar thematic areas. It assesses a set of interconnected activities across different projects and entities. As a result, it strengthens partnerships within these key sectors, while improving their coordination, accountability, predictability, and response capacity.
10. Meta-evaluation: This is used to assess the evaluation process itself. Meta-evaluations could be useful to make a selection of future evaluation types, check compliance with evaluation policy and good practices, assess how well evaluations are utilized for organizational learning and change, etc.
Why do we need M&E?
The value of Monitoring and Evaluation (M&E) is now widely recognised in international development circles. Most Non-Governmental Organisations (NGOs) and volunteer organisations (VOs) are expected to engage in M&E, particularly when they receive funding from public funding bodies.
There are many reasons why NGOs should embrace M&E, not least since a lukewarm attitude towards evaluation has a negative impact on the ability of NGOs to learn, store, and disseminate formal lessons (Cumming, 2008b).
Even so, despite the many benefits of M&E, it can often be neglected by NGOs , partly because it is perceived to be too challenging and costly. Remember that evaluation is not about getting to know everything about a project or programme but about identifying some key markers of success or failure.
M&E can be beneficial…
For Your Beneficiaries
- M&E can help you show that you are not doing harm and not ‘killing with kindness’. It helps to evidence the ‘good’ you are doing.
- It can be part of a participatory process that requires you to listen to the opinions of local partners and beneficiaries.
For Yourself
- It assesses how and where a project has achieved its intended goals, how/ where it has not and how/ where it could be improved.
- It identifies how efficient the project was (or could be) in converting resources (funded and in-kind) into activities, objectives and goals.
- It highlights how sustainable and meaningful the project was (or was not) for participants.
For Your Donor
- It helps you meet organisational reporting and donor requirements.
- It can help to show value for money at a time when the press and other bodies are scrutinising/ criticising donor funding of NGOs.
- It can open the door to future funding from public bodies.
Final points to remember:
- Evaluation can be done one time, at the end of a project, or it can be an ongoing study.
- In all cases, evaluation is a way to learn from what has happened, look at it objectively, and use that knowledge to make informed decisions about the future of your work.
- Evaluation is not just about demonstrating success, it is also about learning why things don’t work. As such, identifying and learning from mistakes is one of the key parts of evaluation. Greater honesty on the part of the entire development industry would channel everyone’s efforts into more productive action!