Remote Data Collection During COVID-19

The COVID-19 pandemic and associated lockdowns have forced us to shift from traditional approaches of in-person data collection for research and evaluation to remote, or mostly remote, practices. For example, in many cases it is no longer allowable or ethical for data collectors to conduct fieldwork because of the risks associated with face-to-face interviews and observations. Although this has implications for evaluation design, the quality of the data collection, as well as our ability to work closely with our evaluation partners, we are committed to maintaining both the quality and integrity of the research.

Evaluation design

While the overall objectives and research questions have remained largely consistent, it’s been necessary to optimize existing data sources and streamline data collection approaches. Given the challenges of securing primary data, we will rely more heavily on existing reports and data sources, including routine data, and build on and validate them for some variables rather than duplicate data with our primary data collection. We have also decided to conduct more key informant interviews in lieu of focus group discussions, which would require in-person congregation or more complex remote solutions. We will revisit the original sampling criteria and sample size mid-assessment if the team encounters obstacles, such as difficulty in reaching participants, lower response rates, or staff absences from their posts.

Data quality

To ensure data quality, we are partnering with well-respected research firms that have the organizational capabilities and protocols in place for remote data collection. Since international travel and in-person support from our home office is not possible at this time, we are working closely with partners through frequent planning and oversight meetings. We are also emphasizing the need for thorough yet remote supervision of the fidelity and quality of the data collected through strategies such as: 

• Using automated systems for quantitative data collection to ensure that human error is kept to a minimum. 
• Using a smartphone application to record phone calls as a quality assurance method. 
• Duplicating spot-checking of interviews by making a three-way phone call with a supervisor.
• Avoiding questions with long lists of answer choices and instead using a modular questionnaire design, in which a long question is broken into parts and administered in different rounds to the same or different participants to lower the chance of high non-response rates.

Working with evaluation partners

We always work in collaboration with a local research team or firm that has demonstrated research strengths, and the pandemic has not changed this. But other considerations for the selection of research partners given the new context include:  

• Approach to collaboration: How do they propose to collaborate with us, given the remote nature of the evaluation management?
• Data collector training: Will training be done remotely with JSI’s involvement or with local partners and JSI’s assigned consul?
• Technical oversight: What will regular communication between ourselves and partners look like (what platform and how often)? How often will we have partners share data with us now that we cannot work alongside each other? In what capacity will we be involved in qualitative interviews? How will JSI provide regular feedback to partners after reviewing the data?