4.6 \(\chi^{2}\) test of independence
As introduced in previous chapters, a hypothesis test is a statistical procedure used to evaluate whether the evidence observed in a sample is compatible with a statement about the population. In the context of contingency tables, tests of independence make it possible to analyze whether there is an association between two categorical variables.
In this case, the null hypothesis (\(H_{0}\)) states that both variables are independent; that is, the distribution of one variable does not depend on the categories of the other. Under this assumption, the differences observed in the sample are attributed only to sampling variability and not to a real relationship between the variables in the population. Mathematically, this hypothesis can be expressed as:
\[ H_0: p_{rc}^0 = p_{r+} \times p_{+c}, \quad \text{para todo } r = 1,\ldots,R \text{ y } c = 1,\ldots,C \]
Where \(p_{rc}^0\) represents the expected proportion in cell \((r,c)\) under the assumption of independence, while \(P_{r+}\) and \(P_{+c}\) correspond to the row and column marginal proportions, respectively. Under this formulation, the independence hypothesis implies that the joint probability of each cell can be expressed as the product of its marginal probabilities.
Consequently, the test of independence consists of comparing the observed or estimated proportions \(\hat{p}_{rc}\) with the expected proportions \(p_{rc}^0\) under the null hypothesis. When the differences between them are small, the empirical evidence is consistent with the assumption of independence. Conversely, sufficiently large discrepancies suggest the existence of an association between the variables and lead to rejection of \(H_0\).
As noted previously, in household surveys, features of the complex sampling design, such as stratification, cluster selection, and the use of expansion factors, affect the variability of the estimators compared with what would be obtained under simple random sampling. As a result, Pearson’s classical \(\chi^2\) test of independence is not appropriate for analyzing contingency tables from this type of design, because it may underestimate or overestimate the real variability of the estimates.
To correct this problem, Fay (1979) and Fellegi (1980) proposed the first adjustments to Pearson’s chi-square statistic based on the generalized design effect (\(GDEFF\)). Later, Rao & Scott (1984) and Thomas & Rao (1987) expanded and formalized the theoretical framework for these corrections, giving rise to what is now known as the Rao-Scott test, considered the reference procedure for analyzing categorical data obtained from complex surveys.
The central idea of this approach is to adapt the classical test of independence by incorporating the generalized design effect, thereby obtaining a statistic that is robust to the complexities of the sampling design:
\[ \chi_{RS}^2 = \frac{n_{++}}{GDEFF} \sum_r \sum_c \frac{(\hat{p}_{rc} - p_{rc}^0)^2}{p_{rc}^0} \]
where \(n_{++}\) represents the total sample size, \(\hat{p}_{rc}\) corresponds to the estimated proportion in cell \((r,c)\) of the contingency table, and \(p_{rc}^0\) denotes the expected proportion under the null hypothesis of independence, calculated from the product of the row and column marginal proportions.
The term \(GDEFF\) (Heeringa et al., 2017) represents the generalized design effect and measures how much the variability of the estimates increases or decreases as a consequence of the complex survey design relative to the variability that would be observed under simple random sampling. In this way, the Rao–Scott adjustment allows the inferences derived from the test of independence to adequately reflect the structure of the sampling design.
Under the null hypothesis \(H_0\), the statistic \(\chi_{RS}^2\) approximately follows a \(\chi^2\) distribution with \((R-1)(C-1)\) degrees of freedom, where \(R\) and \(C\) correspond to the number of row and column categories, respectively. In situations with small samples or few degrees of freedom, adjustments based on the \(F\) distribution are often used because they improve the precision of statistical inference. Rao-Scott tests are currently the standard for analyzing association between categorical variables in complex surveys (Heeringa et al., 2017).
The survey package in R implements this type of test through the svychisq() function, which automatically incorporates the Rao-Scott adjustments to account for the characteristics of the complex sampling design. This function makes it possible to evaluate the existence of association between two categorical variables using weighted contingency tables and properly correcting the variability of the estimators. As an example, to evaluate whether poverty status is independent of sex, the following code is run:
##
## Pearson's X^2: Rao & Scott adjustment
##
## data: NextMethod()
## F = 0.06, ndf = 1, ddf = 119, p-value = 0.8In this case, the formula argument specifies the two categorical variables whose independence is to be evaluated; design corresponds to the sampling design defined previously; and statistic = "F" indicates that the test will be carried out using the adjusted version based on the \(F\) distribution. If the p-value is greater than 5%, the null hypothesis \(H_0\) is not rejected, leading to the conclusion that there is not enough statistical evidence to state that poverty and sex are associated. Conversely, if the p-value is less than 5%, \(H_0\) is rejected, suggesting the existence of dependence or association between the two categorical variables. Other relationships can be evaluated in the same way, such as unemployment and sex or poverty and region.
##
## Pearson's X^2: Rao & Scott adjustment
##
## data: NextMethod()
## F = 62, ndf = 2, ddf = 201, p-value <0.0000000000000002In this first case, for the test of independence between the variables Sex and Employment, the statistic obtained was \(F = 62.251\), with a p-value less than \(2.2\times10^{-16}\), providing statistically significant evidence to reject the null hypothesis of independence between the two variables. Consequently, the results suggest that there is a significant association between sex and employment status in the study population.
##
## Pearson's X^2: Rao & Scott adjustment
##
## data: NextMethod()
## F = 0.5, ndf = 3, ddf = 358, p-value = 0.7On the other hand, this test of independence between the variables Region and poor produces \(F = 0.48794\), with a p-value of \(0.6914\). Because this p-value is considerably greater than conventional significance levels, there is not enough evidence to reject the null hypothesis of independence. Therefore, the results suggest that, under the sampling design considered, no statistically significant association is observed between region and poverty status.