Metric Invariance in Exploratory Graph Analysis via Permutation Testing

Metric Invariance in Exploratory Graph Analysis via Permutation Testing

2024-06-28 | Laura Jamison, Alexander P. Christensen, Hudson F. Golino
The paper "Metric Invariance in Exploratory Graph Analysis via Permutation Testing" by Laura Jamison, Alexander P. Christensen, and Hudson F. Golino addresses the importance of establishing measurement invariance (MI) in psychological research to ensure the validity and comparability of measurements across different groups. Traditional methods for testing MI, such as Item Response Theory (IRT) and Structural Equation Modeling (SEM), often suffer from limitations, including the need for intensive model specifications and the potential for model misspecification. The authors propose a new method within the Exploratory Graph Analysis (EGA) framework that uses network loadings, analogous to factor loadings, to test metric invariance. This method is designed to be more conceptually consistent and powerful, especially in scenarios with smaller or unequal sample sizes and lower noninvariance effect sizes. The paper begins by discussing the limitations of traditional psychometric methods, such as the need for multiple hypothesis testing and the potential for false positives. It then introduces EGA, a network psychometric method that represents variables as nodes and their relationships as edges. The authors propose a permutation test to compare network loadings across groups, which is more flexible and robust to parametric deviations compared to traditional hypothesis tests. The method is evaluated through a simulation study, where it is compared to traditional SEM methods using the Free, Fixed, and Wald approaches. The simulation results show that the proposed method, when corrected for multiple comparisons using the Benjamini-Hochberg procedure, outperforms traditional methods in terms of hit rate, sensitivity, specificity, and F1 score, particularly in conditions with smaller or unequal sample sizes and lower noninvariance effect sizes. Overall, the study provides a novel and powerful method for testing metric invariance in the EGA framework, offering a more straightforward and robust alternative to traditional SEM methods.The paper "Metric Invariance in Exploratory Graph Analysis via Permutation Testing" by Laura Jamison, Alexander P. Christensen, and Hudson F. Golino addresses the importance of establishing measurement invariance (MI) in psychological research to ensure the validity and comparability of measurements across different groups. Traditional methods for testing MI, such as Item Response Theory (IRT) and Structural Equation Modeling (SEM), often suffer from limitations, including the need for intensive model specifications and the potential for model misspecification. The authors propose a new method within the Exploratory Graph Analysis (EGA) framework that uses network loadings, analogous to factor loadings, to test metric invariance. This method is designed to be more conceptually consistent and powerful, especially in scenarios with smaller or unequal sample sizes and lower noninvariance effect sizes. The paper begins by discussing the limitations of traditional psychometric methods, such as the need for multiple hypothesis testing and the potential for false positives. It then introduces EGA, a network psychometric method that represents variables as nodes and their relationships as edges. The authors propose a permutation test to compare network loadings across groups, which is more flexible and robust to parametric deviations compared to traditional hypothesis tests. The method is evaluated through a simulation study, where it is compared to traditional SEM methods using the Free, Fixed, and Wald approaches. The simulation results show that the proposed method, when corrected for multiple comparisons using the Benjamini-Hochberg procedure, outperforms traditional methods in terms of hit rate, sensitivity, specificity, and F1 score, particularly in conditions with smaller or unequal sample sizes and lower noninvariance effect sizes. Overall, the study provides a novel and powerful method for testing metric invariance in the EGA framework, offering a more straightforward and robust alternative to traditional SEM methods.
Reach us at info@study.space
[slides] Metric invariance in exploratory graph analysis via permutation testing | StudySpace