2.2 Sampling estimators
When analyzing survey data, it is also essential to define the parameter of interest: a fixed numerical value that describes a characteristic of the whole population, denoted as \(U\). The most common parameters include proportions, sizes, totals, means, and ratios, among others. Since, in practice, it is not possible to observe the entire population, sample surveys make it possible to infer these parameters from a sample, denoted as \(s\).
In probability sampling, each unit in the population has a known inclusion probability greater than zero of being selected in the sample. These probabilities are the basis for calculating the basic sampling weights, which are then used to estimate population parameters through weighted sums of the data collected in the survey. When the design and selection are implemented correctly, the resulting estimates are unbiased, meaning that on average they coincide with the true population value if the sampling process were repeated under the same conditions.
Nevertheless, basic sampling weights often require additional adjustments in order to improve the precision and robustness of the estimates. One of the most common adjustments is the treatment of nonresponse, through which the weights of the units that did respond are increased so that they also represent selected units that did not participate in the survey, thereby helping to reduce possible biases. Another widely used adjustment is calibration, which consists of modifying the weights to ensure that the weighted sums of certain auxiliary variables, such as age or sex, match known population totals from censuses or demographic projections. In addition to improving the consistency of estimates, calibration is a useful tool for detecting coverage problems or nonresponse patterns.
Weight adjustments, and calibration in particular, play a fundamental role in the analysis of surveys with complex designs. These procedures not only correct imperfections arising from nonresponse or incomplete coverage, but also strengthen the coherence between the sample and the target population, ensuring that inferences are statistically valid and comparable with other official sources (Kalton & Flores-Cervantes, 2003).
Calibration is especially relevant because it makes it possible to compare the survey’s initial estimates with external reference values, usually obtained from censuses, administrative records, or demographic projections. This comparison is a powerful tool for detecting inconsistencies in the composition of the sample, identifying potential biases associated with nonresponse, and assessing the quality of fieldwork.
In addition, calibrated weights tend to reduce the variance of estimates, improving their precision without altering known population totals (Särndal & Lundström, 2005). Consequently, calibration not only corrects but also optimizes the use of available information, making it possible to obtain more precise results.
The estimation of totals in surveys is a central step in statistical analysis applied to finite populations. Many indicators of interest for public policy formulation (such as, for example, the number of people living in poverty, the total number of employed persons, or aggregate household expenditure) are derived from a population total. For this reason, understanding how totals are defined and estimated is fundamental for ensuring the quality and relevance of the information produced.
In formal terms, if \(y_k\) denotes the value of a variable of interest for unit \(k \in U\), the population total is defined as
\[ t_y = \sum_{U} y_k, \]
Letting \(N\) be the size of the population, the population mean is defined as \(\bar{y} = \frac{t_y}{N}\). Since in practice only a sample \(s \subset U\) is observed, it is necessary to use estimators that incorporate the sampling design. For example, the Horvitz-Thompson (HT) estimator is expressed as
\[ \hat{t}_{y} = \sum_{s} d_k y_k \]
where \(d_k = 1/\pi_k\) are the basic weights from the sampling design and \(\pi_k = \Pr(k \in s)\) are the first-order inclusion probabilities.
In practice, design weights are often modified to reflect additional processes such as nonresponse adjustment or calibration to known population totals, thus obtaining new adjusted weights, denoted as \(w_k\) for all \(k \in s\). Replacing \(d_k\) with \(w_k\) makes it possible to improve precision and reduce bias in the estimates, especially when reliable auxiliary information sources are available.
Nevertheless, any estimate from a sample involves uncertainty. Even when the estimator is unbiased, results vary from one sample to another because of the randomness property of the design. This variability is quantified through the variance of the estimator, from which the standard error (\(se\)) or the coefficient of variation (\(cv\)) can be calculated. These indicators are indispensable tools for assessing the precision of estimated totals and, therefore, for interpreting statistical information properly. Under the design-based approach, the unbiased variance of the Horvitz-Thompson estimator can be expressed as:
\[ \widehat{Var}_p(\hat{t}_{y}) = \sum_{k \in s} \sum_{l \in s} \bigl( d_k d_l - d_{kl} \bigr) y_k y_l, \]
where \(d_{kl} = 1/\pi_{kl}\) and \(\pi_{kl} = \Pr(k,l \in s)\) represent the joint inclusion probabilities. This expression requires the sampling design to satisfy \(\pi_{kl} > 0\) for every pair of units \(k,l \in U\).
To understand more concretely the relevance of considering the sample design when estimating totals and their variances, let us analyze an example from United Nations Statistics Division (2026) that shows how the estimation method adjusts to the selection scheme adopted. Suppose there is a finite population of size \(N=6\), from which a simple random sample (MAS) without replacement of size \(n=3\) is selected. In that sample, the values \((y_1=10, y_2=14, y_3=18)\) are observed. Under this design, the Horvitz-Thompson estimator of the population total is defined as the sum of the observed values divided by their inclusion probabilities, and its estimated variance is obtained from the covariances induced by the selection process. Thus, the estimated variance of the Horvitz-Thompson estimator is calculated as:
\[ \widehat{Var}_{MAS}(\hat{t}_{y}) = \frac{N^2}{n}\left(1-\frac{n}{N}\right)S_{y_s}^2 \]
where \(S_{y_s}^2\) corresponds to the sample variance of the observed values. Substituting into the expression gives
\[ \widehat{Var}_p(\hat{t}_{y}) = \frac{36}{3}\left(1-\frac{3}{6}\right)16 = 96 \]
By contrast, if the sampling design is ignored, an inexperienced analyst might incorrectly calculate the variance using the simplified formula:
\[ \frac{N^2}{n}S_{y_s}^2 = 192, \]
which would lead to an overestimation of the variance because the characteristics of the sample design are not considered. Likewise, the estimate of the population total is \(\hat{t}_{y}=84\). The standard error calculated according to the sampling design is
\[ \sqrt{\widehat{Var}_p(\hat{t}_{y})} = \sqrt{96} \approx 9.80. \]
By contrast, if the variance is estimated using a naive method that ignores the sampling design, the confidence interval would be wider and clearly biased, which could lead to erroneous inferences.
Finally, according to Kish (1965, p. 258), the design effect (DEFF) is defined as the ratio between the variance of an estimator obtained under a complex sampling design and the variance of the same estimator under simple random sampling (MAS) with the same sample size. Its estimate is expressed as:
\[ \widehat{{DEFF}} = \frac{\widehat{Var}_{p}(\hat{\theta})}{\widehat{Var}_{MAS}(\hat{\theta})} \]
Here, \(\widehat{Var}_{p}(\hat{\theta})\) corresponds to the estimated variance of \(\hat{\theta}\) under the complex design \(p(\cdot)\), while \(\widehat{Var}_{MAS}(\hat{\theta})\) represents the estimated variance of the same estimator under a MAS design with the same sample size.
This indicator makes it possible to quantify how much the variance increases due to clustering and other characteristics of complex designs compared with simple sampling. According to Naciones Unidas (2009, p. 40), the DEFF can be understood in three ways: as the variance inflation factor relative to MAS, as a measure of the relative loss of precision, or as an indication of the increase in sample size that would be necessary in a complex design to achieve the same variance level as in a MAS. According to Park et al. (2003), the design effect of a survey may be due to the following three factors:
- Unequal weighting: the presence of nonuniform sample weights usually increases the variance slightly; therefore, the use of uniform weights is advantageous and explains why self-weighting designs are preferred in household surveys.
- Stratification: when applied correctly, it can reduce the variance, although in practice its variance-reducing effect is usually moderate.
- Multistage sampling: this generally increases the variance, since units within the same cluster tend to be more homogeneous among themselves than compared with units in other clusters.
In survey analysis, the design effect (DEFF) is an essential indicator for assessing the precision and efficiency of estimates, as well as for guiding the planning of future studies. A high value shows that the complex design introduces an increase in variance, reducing the precision of the results. By contrast, a value close to one indicates that the design has a minimal effect on the variance. This information allows researchers to identify whether it is necessary to adjust weighting, optimize stratification, or modify the subsampling size to increase efficiency in future survey operations.
The interpretation of a high DEFF should be done with caution, since it does not always imply that the sample design is inadequate. It is essential to consider the survey context: a value greater than three might seem alarming, but it is often due to practical limitations such as budget constraints, logistical difficulties, or the need to guarantee respondent participation. In certain household surveys, it may be essential to select only a fraction of eligible individuals in each household. In addition, coverage problems or nonresponse rates can increase the variability of sampling weights and, consequently, raise DEFF values. Gutiérrez & Babativa-Márquez (2023) provide a detailed analysis of design effects in household surveys in Latin America using BADEHOG data.