Kosuke Imai, Gary King, and Clayton Nall. "The Essential Role of Pair Matching in Cluster-Randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation," copy at http://gking.harvard.edu/files/abs/cluster-abs.shtml. (Article: PDF)

Abstract

A basic feature of many field experiments is that investigators are only able to randomize clusters of individuals --- such as households, communities, firms, medical practices, schools, or classrooms --- even when the individual is the unit of interest. To recoup the resulting efficiency loss, some studies pair similar clusters and randomize treatment within pairs. However, many other studies avoid pairing, in part because of claims in prominent methodological articles, echoed by clinical trials standards organizations, that this matched-pair, cluster-randomization design has serious problems. We prove that all such claims are unfounded. We then derive the model underlying the estimator recommended in the literature for this design. It is shown that this existing estimator is unbiased only in situations when matching is unnecessary, and that its commonly used standard error is invalid. To overcome this problem without modeling assumptions, we develop a simple design-based estimator with much improved statistical properties. We also propose a model-based approach that encompasses our design-based estimator as well as the estimator in the literature. Our methods also address individual-level noncompliance, which is common in applications but ignored by most existing methods. We show that from the perspective of bias, efficiency, power, or robustness, and in large or small samples, pairing should be used in cluster-randomized experiments whenever feasible; failing to do so is equivalent to discarding a considerable fraction of one's data. We develop these techniques in the context of a randomized evaluation we are conducting of the Mexican Universal Health Insurance Program.