Ebooks

Evaluation has become an essential component of effective programme planning and implementation across development, education, agriculture, health, and allied sectors. In today’s knowledge-driven society, the success of any programme is measured not only by its execution but also by its outcomes, impacts, and long-term sustainability. Recognizing this need, this book, Programme Evaluation, has been prepared to provide learners with both theoretical foundations and practical approaches to evaluation. The book is organized into five blocks for systematic learning. The first block introduces the fundamental concepts and theories of evaluation. The second block explains the process of conducting evaluations in real-world contexts. The third block focuses on management techniques such as SWOT analysis, bar charts, and networks, which aid in programme planning and decisionmaking. The fourth block presents programme evaluation tools and models, including the Logic Framework Approach (LFA). The final block deals with impact assessment, offering insights into indicators, methodologies, and specialized areas such as Environmental Impact Assessment (EIA).

Concept of Evaluation: Meaning and concept in different contexts The term “evaluation” originates from the Latin word “Valupure,” which signifies the value attributed to a particular entity, concept, or action. Evaluation is a systematic process that involves the periodic comparison of actual outcomes and impacts against those initially planned or anticipated. It is designed to assess the overall worth of an endeavor, thereby facilitating learning and improvements for future initiatives. This process encompasses the structured and objective assessment of an ongoing or completed project, program, or policy, including its design, execution, and results. The primary objective of evaluation is to determine the relevance, achievement of objectives, efficiency, effectiveness, impact, and sustainability of a given intervention. Through this approach, evaluation seeks to provide credible and useful information that supports informed decision-making for stakeholders, including beneficiaries and donors. By integrating lessons learned from past experiences, evaluation enhances the strategic planning and implementation of future projects, ensuring continuous improvement and better resource utilization. Evaluation is an essential tool in both public and private sectors, allowing organizations to measure their performance against predefined benchmarks. It serves as a crucial mechanism for accountability, ensuring that resources are utilized effectively to achieve intended goals. Moreover, evaluation fosters transparency by offering objective insights into the successes and shortcomings of initiatives, thereby reinforcing stakeholder trust and engagement. There are several types of evaluation, each serving a distinct purpose.

Evaluation theories refer to the conceptual frameworks, models, and principles that guide the systematic assessment and analysis of programs, policies, interventions, and other social phenomena. They provide a set of organizing principles and methodologies for evaluating the effectiveness, efficiency, relevance, and sustainability of various interventions and initiatives in different domains, including education, healthcare, social services, environmental protection, and public policy. Evaluation theories draw from various disciplines, such as psychology, sociology, economics, statistics, and management, and they may emphasize different aspects of the evaluation process, such as the role of stakeholders, the criteria for success, the methods for data collection and analysis, and the use of evaluation results. i. Programme theory Program theory defines as the mechanisms that mediate between the delivery (and receipt) of the program and the emergence of the outcomes of interest. Programme theory, variously referred to as programme theory, programme logic, theory-based evaluation or theory of change, theory-driven evaluation, theory-of-action, intervention logic, impact pathway analysis, and programme theory-driven evaluation science refers to a variety of ways of developing a causal modal linking programme inputs and activities to a chain of intended or observed outcomes, and then using this model to guide the evaluation. The term ‘logic model’ is used to refer to the summarized theory of how the intervention works (usually in diagrammatic form) and ‘programme theory evaluation’ is used for the process of developing a logic model and using this in some way in an evaluation. Most approaches to building logic models have focused on simple, linear models, but some have explored how non-linear models might be used to better represent programmes and guide their evaluation. In particular, a number of evaluators have incorporated concepts of complexity in their discussion and use of logic models. 

Ten Steps in programme evaluation i. Identify and describe programme you want to evaluate: The first step of the evaluation process is to identify the object or programme of evaluation. For this purpose, a short written description of the object or programme to be evaluated must be drafted. This description must be formulated in a way that is also comprehensible to outsiders. Relevant items to be included in this description are, for example, the name of the object or programme of evaluation, the expert responsible for it, the employees taking part in it, the goals, scientific references, the contents or topics, the elements, methods, scope, available resources, key figures, past developments, etc. ii. Identify the phase of the programme • Design • Start-up • Ongoing • Wrap-up • Follow-up

SWOT Analysis – Concept, origin and evolution While working on a research project at the Stanford University sometime around 1960s to 1970s, Albert Humphrey developed an analytical tool to evaluate the strategic plans and find out why corporate planning failed. He coined this technique as SOFT analysis where – • S stood for what things are Satisfactory at present, • O denoted what Opportunities can be explored in the future, • F meant the Faults in the present and • T signified the Threats that could surface in the future. While the majority agrees that SOFT is the predecessor of SWOT, some people believe that the concept of SWOT analysis emerged separately and has nothing to do with SOFT. How SWOT evolved from SOFT? The first mention of the term SWOT can be traced back to the Long Range Planning seminar held in Zurich in 1964. In this seminar Urick and Orr proposed the concept of SWOT analysis which was derived from SOFT by replacing the F for faults with W for Weaknesses. With its initial promotion in the Britain, the concept soon gained recognition among strategic planners and management consultants the world over. The new interpretation of SWOT, the way it is used these days, differs slightly from that of SOFT.

Networks – Introduction, origin and widely used networks Network is a technique used for planning and scheduling of large projects in the fields of construction, maintenance, fabrication, purchasing, computer system instantiation, research and development planning etc. There is multitude of operations research situations that can be modelled and solved as network. Some recent surveys reports that as much as 70% of the real-world mathematical programming problems can be represented by network related models. Network analysis is known by many names PERT (Programme Evaluation and Review Technique), CPM (Critical Path Method), PEP (Programme Evaluation Procedure), LCES (Least Cost Estimating and Scheduling), SCANS (Scheduling and Control by Automated Network System), etc. It is a graphical representation of logical and sequentially connected activities and events of a project. Network is also called arrow diagram. PERT (Programme Evolution Review Technique) and (Critical Path Method) are the two most widely applied techniques.

An important decision that the management has to take during implementation of the process of evaluation is about which model to choose. The choice of a single model will mainly depend on the nature of the problem or the situation in an organization. However an integrative approach using more than one model may also be used. Before implementation of any of these models and before the actual process of evaluation starts it is important to make suitable preparations and create a conducive environment in the organization for successful conductance of the evaluation. The following points are to be taken in to consideration before and at the time an evaluation is carried out in an organization:

Introduction to LFA – Background and description The Logical Framework Approach (LFA) is a methodology mainly used for designing, monitoring, and evaluating international development projects. Variations of this tool are known as Goal Oriented Project Planning (GOPP) or Objectives Oriented Project Planning (OOPP). The logical framework Approach is a method to develop the project’s goals and activities and identify key information (such as risks) through a participatory approach. Basically, this means that instead of having a single person who designs the project all on his own, you seek the involvement of all concerned parties – also known as the stakeholders. The most important stakeholders are the potential beneficiaries of the project, or the target group.Getting the involvement of your target group and other stakeholders not only ensures that your project is well designed. It also makes sure that everybody is aware of the project and what it is supposed to change. It allows people to get involved, and develop a feeling of responsibility for the project and its results.This is known as developing ownership for the project’s outcomes, and it is important to make sure that the positive realizations of the project are embedded locally and durable over time. People will care about the project and take care about its results.

Introduction Impact Assessment (IA) is a systematic process used to evaluate the potential or actual effects of a project, policy, or program on society, the economy, and the environment. It helps decision-makers understand the implications of their actions and ensures that interventions achieve their intended outcomes while minimizing negative consequences. Meaning of Impact Assessment Impact Assessment is a structured method of determining the significance of an action or intervention in various sectors. It involves analyzing both the intended and unintended effects of policies, projects, or programs. IA provides insights into the sustainability, feasibility, and effectiveness of an initiative before or after its implementation. By assessing social, economic, and environmental impacts, IA helps stakeholders make informed decisions. Concept of Impact Assessment The concept of IA is rooted in sustainable development and evidencebased policy making. It is used in multiple fields, including environmental management, social development, economic planning, and corporate governance. The key principles of IA include:" i. Comprehensiveness: Evaluating a wide range of impacts, including social, economic, and environmental factors. ii. Transparency: Ensuring that the assessment process is open and inclusive.

Indicators for impact assessment Meaning and concept of Indicator The term indicator is often used interchangeably with data, targets, standards for evaluation, and even various modes of data collection, which undoubtedly inhibits a productive dialog on the subject. Such confusion is unwarranted, since the term has a precise connotation- in the literature on monitoring and evaluation of development. Simply stated, indicators are variables designed to measure change in a given phenomenon or process. They are analytical tools which facilitate the measurement of change and provide summary data for project design, implementation, and evaluation. Indicators are defined as “specific (explicit) and objectively verifiable measures of changes or results produced by an activity or intervention” (United Nations: 1985; p. 37). A few observations about indicators can be useful in further clarifying their nature. First, as the above definition implies, indicators provide quantitative data which can be analyzed using statistical techniques to produce estimates of a phenomenon. In cases when only qualitative information is gathered, it should be converted into numerical data for the purposes of analysis and presentation. Second, an indicator can capture one or more dimensions of a phenomenon or process depending upon the purpose of the study. We can, for example, construct an indicator for urbanization based on a wide range of variables such as number of school, higher educational institution, road connectivity, number of financial institution, health infrastructure, income etc. On the other hand, one can use each of the above items as a distinct indicator. Indicators focusing on different dimensions of a subject are difficult and more costly to construct. Third, several indicators can be used to capture the same phenomenon.

Impact assessment approaches Impact assessment approaches can be classified into four broad categories i.e. quantitative, qualitative, participatory and mixed methods. i. Quantitative impact assessment approaches • It focuses on assessing the degree and extent of the impacts quantitatively. • Some degree of quantification may be necessary in all impact assessments, in order to evaluate the success of the Intervention and the extent of any adverse effects. • Largely depends on micro-economic approaches following econometric models • Involving baseline studies: The precise identification of baseline conditions, definition of objectives, target setting, rigorous performance evaluation and outcome measurement • Costly and limited scope: Limited in the types of impacts which can be accurately measured and may pose difficulties for inference of cause and effect.

Introduction Environmental Impact Assessment (EIA) is a process of evaluating the likely environmental impacts of a proposed project or development, taking into account inter-related socio-economic, cultural and human-health impacts, both beneficial and adverse. UNEP defines Environmental Impact Assessment (EIA) as a tool used to identify the environmental, social and economic impacts of a project prior to decision-making. It aims to predict environmental impacts at an early stage in project planning and design, find ways and means to reduce adverse impacts, shape projects to suit the local environment and present the predictions and options to decision-makers. Environment Impact Assessment in India is statutorily backed by the Environment Protection Act, 1986 which contains various provisions on EIA methodology and process. Evolution & History of EIA EIA is termed as one of the best policy innovations in the 1900s. The main aim of EIA is to conserve the environment and bring out the best combination of economic and environmental costs and benefits. Read the below-mentioned points to understand the Environmental Impact Assessment evolution and history: • The birth of EIA is dated back to the 1970s.