Evaluation

JSI is dedicated to improving the quality of health information and increasing evidence-based decisionmaking in the health sector. In response to the growing demand for information-driven health services planning and management, JSI applies sound research and evaluation approaches. Through evaluation, JSI's goal is to strengthen collection and the use of high-quality data to improve health program and system performance. Following we provide copies of JSI's evaluation tools as well as reports of research and results.

Please note the following are listed in chronological order starting with the most recent publication.

Evaluation Tools | Evaluation Reports

Evaluation Tools

A Guide to Monitoring & Evaluation of NGO Capacity Building Interventions in Conflict-Affected Settings (PDF, 1.32 MB)

The guide provides a conceptual framework for the role and importance of high quality monitoring and evaluating (M&E) capacity-building interventions for NGOs operating in conflict-affected settings. Drawing on case studies, past research in capacity measurement and field testing, this guide offers principles for planning and implementing capacity building M&E provides a selection of tools by which to measure progress against capacity-building objectives; and offers a set of indicators which NGOs can draw upon and adapt. Participatory approaches to capacity building M&E are emphasized. Fitzgerald, Molly; Posner, Jessica; Workman, Anna. Reproductive Health for Refugees Project/JSI Research & Training Institute, Inc., February 2009.

Data Demand and Information Use in the Health Sector: Strategies and Tools (PDF, 1.89 MB)

Data Demand and Information Use (DDIU) is a strategy to identify opportunities for and constraints to effective and strategic data collection, analysis, availability, and use. This strategy begins with an assessment that helps stakeholders, policy-makers, and monitoring and evaluation (M&E) practitioners determine points of entry for DDIU intervention. Once specific needs are identified, DDIU core tools can be utilized to stimulate data demand and capacity building and enhance evidence-based decision making. K. Foreit, S. Moreland, Anne LaFond, 2006.

Assessment Tool for Laboratory Services (ATLAS) (PDF, 1.38 MB)

This document is a data gathering tool developed by the DELIVER project to assess laboratory services and logistics. The ATLAS is a diagnostic and monitoring tool that can be used as a baseline survey to complete an annual assessment or as an integral part of the work planning process. The ATLAS is primarily a quantitative tool with a small sample qualitative facility survey of available commodities and equipment. The information collected using the ATLAS is analyzed to identify issues and opportunities, and to outline further assessment and/or appropriate interventions. 2006.

Download document in French (PDF, 1.7 MB)

Guidelines for Implementing Computerized Logistics Management Information Systems (CLMIS) (PDF, 4,169 KB)

This guide defines the computerized LMIS, recommends the components of a successful computerized LMIS, and explains development, implementation, and operations of such a system. It also provides lessons learned and sample reports and graphs. 2006.

Monitoring and Evaluation Indicators for Assessing Logistics Systems Performance (PDF, 1,56 MB)

Quality monitoring is an integral part of any health logistics system, as it can describe the performance of the logistics system, evaluate strengths and weaknesses, and allow for accountability and advocacy. This document describes several indicators that are useful tools in monitoring and evaluating a logistics system for reproductive health commodities. These indicators can be used to measure the availability of commodities in a health facility, the performance of the logistics system, and, ultimately, commodity security. 2006.

Download document in French (PDF, 2.32 MB)

Download document in Spanish (PDF, 1.09 MB)

Routine Immunization Checklist (PDF, 62 KB)

Developed by IMMUNIZATIONbasics, this document lists 12 indicators to help district- and national-level staff strengthen the performance of their routine immunization programs. IMMUNIZATIONbasics, 2005.

Download document in French (PDF, 63 KB)

Logistics Indicators Assessment Tool (LIAT) (DOC, 620 KB)

A quantitative data collection instrument, developed by DELIVER, that assesses health commodity logistics system performance and commodity availability at health facilities. The User's Guide is included and provides detailed instructions on how to use the tool. 2005.

Download document in French (DOC, 666 KB)

Download document in Spanish (DOC, 863 KB)

Logistics System Assessment Tool (LSAT) (DOC, 2,183 KB)

Newly revised, this qualitative data collection instrument provides a comprehensive system-level assessment of logistics system performance for any program that manages a health commodity. 2005.

Download document in French (DOC, 3,054 KB)

Download document in Spanish (DOC, 4,107 KB)

Tool to Assess Site Readiness for Initiating Antiretroviral Therapy (ART) or Capacity for Existing ART Sites Version 1.3. (PDF, 254 KB)

This 20-page guide helps programs select ART sites based not on site type but on capacity, vision, and activities needed for rational introduction and expansion of ART into HIV care. JSI/DELIVER Project, 2007.

Spanish Version. Version 1.1, 2003. (PDF, 182 KB)

A Guide to Monitoring and Evaluation of Capacity-Building Interventions in the Health Sector in Developing Countries (PDF, 757 KB)

This guide was developed to assist health planners and evaluators to gain a clear understanding of the concepts of capacity and capacity building. It looks at ways to critically evaluate the strengths and limitations of current approaches to capacity measurement, as well as design a capacity-building monitoring and evaluation plan that outlines a systematic approach to measuring capacity and assessing the results of capacity-building interventions in the health sector. Anne LaFond and Lisanne Brown, MEASURE Evaluation Project/JSI Research & Training Institute, Inc., 2003.

Rapid assessment of referral care systems: a guide for program managers (PDF, 2.56 MB)

BASICS II, 2003.

Mapping capacity in the health sector: a conceptual framework. International Journal of Health Planning Management. (PDF, 180 KB)

Building capacity has become central to strategies used to develop health systems in developing countries. However, despite the growing interest in capacity improvement, there has been little consensus among practitioners and academics on definitions of capacity building and how to evaluate it. This paper reviews current knowledge and experiences from ongoing efforts to define, monitor and evaluate capacity building interventions in the health sector in developing countries. It presents a conceptual framework for mapping capacity and measuring the effects of capacity building intervention. 17(1):3-22. Anne K. LaFond, Lyndon Brown, K. Macintyre, 2002.

Evaluation Reports

Evaluation of Vermont's Alzheimer's Disease Demonstration Grant to States Programming Reportt (PDF, 483 KB)

JSI Research & Training Institute, Inc., conducted an evaluation of Vermont programs that provide supportive, educational and direct service interventions to caregivers of people with Alzheimer's disease or a related disorder (ADRD) for the Vermont Department of Disabilities, Aging and Independent Living. JSI's evaluation presents important considerations for other states which may be planning similar programs. Craig Stevens, JSI Research & Training Institute, Inc., 2008.

Dutchess County Cancer Control Needs Assessment Report (PDF, 560 KB)

As part of the Dutchess County Department of Health efforts to develop a county-level cancer control plan, JSI Research & Training Institute completed a comprehensive assessment of cancer prevention, screening, treatment, and support services in the county. The report includes three substantive parts: an epidemiologic profile of the burden of cancer in Dutchess County; detailed findings from the three components of the needs assessment (key informant interviews, the resource inventory survey, and focus groups); and a set of themes and recommendations to Dutchess County based on these findings. Together, these themes and recommendations form a basis and provide direction for the next stage of comprehensive cancer control in Dutchess County. Stewart Landers, JSI Research & Training Institute, Inc., 2008.

Evaluation of the African Youth Alliance Program in Ghana, Tanzania, and Uganda (PDF, 538 KB)

In 2005, the Bill & Melinda Gates Foundation asked JSI Research & Training Institute, Inc., to evaluate the impact of The African Youth Alliance (AYA) program. The main objective of the evaluation was to determine whether exposure to AYA's comprehensive, integrated program resulted in improved adolescent sexual and reproductive health (ASRH) knowledge, attitudes, and sexual behaviors among male and female youth age 17-22 in areas where AYA worked in Ghana, Tanzania, and Uganda.

Results from the evaluation show that a significant number of young people in AYA implementation areas were reached by AYA programs and were able to recall ASRH messages, although the degree of exposure varied by country and by type of intervention. Results further demonstrated a significant positive impact of AYA on several variables, most notably condom use, contraceptive use, partner reduction, and several self-efficacy and knowledge antecedents. Overall, the impact of AYA on ASRH behaviors and their antecedents was greater for young women than for young men, especially in Ghana and Uganda. AYA was implemented from 2000 - 2006 by a unique partnership between the United Nations Population Fund, Pathfinder International, and the Program for Appropriate Technology in Health. Tim Williams, Stephanie Mullen, Ali Karim, Jessica Posner, JSI Research & Training Institute, Inc., 2007.

Download country reports:

Ghana (PDF, 1.25 MB)

Tanzania (PDF, 1.29 MB)

Uganda (PDF, 823 KB)

Cultural Diversity Focus Group Report (PDF, 834 KB)

This final report presents findings from a qualitative study of elders from diverse communities conducted on behalf of the Boston Partnership on Elder Adults and the Eldercare Alliance. The consultant utilized community-based participatory research methods to rapidly organize focus groups with elders who are African-American; Cape Verdean; Chinese; Haitian; Latino; Lesbian, Gay, Bisexual and Transgender; Russian; and Vietnamese. The key questions focused on whether elders from each community would use various community- and home-based services and the factors influencing their choices. The findings were presented at a summit entitled What Does It Take to Keep an Elder at Home? on May 6, 2005 in Roxbury, Massachusetts. JSI Research & Training Institute, Inc., June 2005.

An Evaluation of the WIN Project—Evidence of Effectiveness (PDF, 1.24 MB)

From 1999 to 2003, the Women and Infant Health (WIN) Project worked in close collaboration with the Ministry of Health of the Russian Federation to improve the quality of maternal and newborn services and to increase access to and demand for high quality reproductive health services in three Russian cities. This report compiles the findings from the project's varied baseline and endline facility and household surveys and routine facility monitoring system in a data-based description of project results. JSI/WIN Project, August 2003.

Ritual Use of Mercury: Assessment and Education Project (PDF, 227 KB)

Already under Environmental Health. In June 2003, the JSI Center For Environmental Health Studies conducted an educational program and assessed the use of elemental mercury for spiritual, ritual, and cultural practices among Latino residents of Lawrence, Massachusetts. The study was conducted by Latino residents who have been hired and trained as environmental health educators in the community. JSI Center For Environmental Health Studies, August 2003.

Health Care Purchasing Among Large Private and Public Employers in California (PDF, 1.66 MB)

This report by JSI Research & Training Institute, Inc. presents the findings from a comprehensive survey of health purchasing practices among 318 of California's largest private and public employers. Conducted by researchers at the JSI Research & Training Institute and the Massachusetts Institute of Technology, the survey attained a response rate of 85 percent, presenting an accurate view of health benefits among large employers in California.