How to design field surveys that survive quality checks

Bad data collected cleanly is still bad data. The point of a quality-control system is not to catch errors after collection — it is to prevent them from happening in the first place. This distinction matters because field data that fails quality checks at the analysis stage is expensive: you have already paid for the collection, and now you either have to redo it or report with lower confidence than the decision requires.

The five areas below cover most of what separates a survey system that generates reliable evidence from one that generates plausible-looking but ultimately unreliable data.

1. Instrument design — before the field team sees it

Every question in a survey instrument is a potential source of error. Questions that are ambiguous will be interpreted differently by different enumerators. Questions with many answer options will produce response distributions that reflect the option design as much as actual respondent views. Skip logic that is complex will be misapplied in the field unless it is automated.

The discipline before finalising an instrument is to ask, for every question: what specific answer are we trying to capture, and does this question reliably produce it? Piloting the instrument with a small group of target respondents — not colleagues — before field deployment will surface problems that desk review misses.

2. Sampling logic — clear enough to audit

A sampling approach that cannot be explained in two paragraphs is usually a sampling approach with gaps. The enumerator needs to know specifically who to approach, what to do when a target respondent is unavailable, and how to document refusals. These decisions, made inconsistently in the field, are one of the most common sources of bias in survey data.

If probability sampling is used, the selection rules should be documented and applied consistently. If purposive or convenience sampling is used, the rationale and its limitations should be explicit — not assumed.

3. Enumerator training — not just orientation

Enumerator training is frequently too short and too focused on tool navigation rather than data quality. An enumerator who knows how to complete a form but does not understand what each question is trying to capture will fill it out correctly in format while producing unreliable content.

Effective training includes: reading every question out loud and discussing how to handle common responses; practicing the full instrument with a mock respondent; reviewing common errors from previous surveys in the same context; and clear guidance on what to do when something unexpected happens in the field. The last item is particularly important — enumerators who encounter situations not covered by the protocol will improvise, and improvisation introduces noise.

4. Real-time quality control during collection

QC applied only at the end of data collection is too late. At a minimum, a supervisor should review a sample of completed instruments each day and flag anomalies before the next day's collection begins. Common things to check in real time: implausible values, response patterns that suggest fabrication (identical responses across multiple items), skip logic errors, and unusually fast completion times.

If digital collection is used, built-in validation rules — range checks, required fields, branching logic — reduce a category of errors at source. This is not a substitute for supervisor review, but it eliminates the mechanical errors that would otherwise arrive in the dataset.

5. Back-checks — systematic, not ad hoc

A back-check is a follow-up contact with a respondent to verify that the interview was conducted and the answers recorded correctly. Back-checks are the primary tool for detecting fabricated interviews. A back-check rate of five to ten percent of completed surveys, selected randomly, is enough to deter fabrication while remaining operationally feasible.

Back-checks should ask a subset of questions from the original instrument — not the same questions in the same order, which would produce the same answers even if the original was fabricated. They should also ask the respondent whether the interview took place at all, and whether the enumerator behaved appropriately.

Documentation and cleaning protocols

Every decision made during data cleaning — how missing values are handled, how outliers are treated, what duplicate records mean — should be documented in a cleaning log. This is not bureaucratic overhead. It is the record that allows someone else to understand the dataset without asking the analyst to explain every decision in retrospect. It is also the evidence that quality was taken seriously rather than managed selectively.