Evaluating significance in linear mixed-effects models in R

Evaluating significance in linear mixed-effects models in R

2017 | Steven G. Luke
This paper evaluates methods for determining significance in linear mixed-effects models using the lme4 package in R. The two most common methods are the likelihood ratio test (LRT) and the t-as-z approach, which uses the z distribution to approximate p-values from t-values. Simulations show that both methods are somewhat anti-conservative, especially for smaller sample sizes. Other methods, such as parametric bootstrapping and the Kenward-Roger and Satterthwaite approximations for degrees of freedom, were also evaluated. The results suggest that the Kenward-Roger and Satterthwaite approximations produce Type 1 error rates closest to 0.05, even for smaller samples. These methods are less sensitive to sample size and more accurate than the LRT and t-as-z approaches. The paper also discusses the limitations of the t-as-z approach and the potential issues with using LRTs, such as their sensitivity to sample size. It concludes that the Kenward-Roger and Satterthwaite approximations are preferable for evaluating significance in mixed-effects models, especially when sample sizes are small. The paper also notes that while these methods are more accurate, they do not provide the exact degrees of freedom for linear mixed-effects models, and that results should be interpreted with caution regardless of the method used to obtain p-values. The paper recommends that users be cautious when using the LRT or t-as-z approaches, especially for small sample sizes, and suggests that the Kenward-Roger and Satterthwaite approximations are preferable for more accurate results.This paper evaluates methods for determining significance in linear mixed-effects models using the lme4 package in R. The two most common methods are the likelihood ratio test (LRT) and the t-as-z approach, which uses the z distribution to approximate p-values from t-values. Simulations show that both methods are somewhat anti-conservative, especially for smaller sample sizes. Other methods, such as parametric bootstrapping and the Kenward-Roger and Satterthwaite approximations for degrees of freedom, were also evaluated. The results suggest that the Kenward-Roger and Satterthwaite approximations produce Type 1 error rates closest to 0.05, even for smaller samples. These methods are less sensitive to sample size and more accurate than the LRT and t-as-z approaches. The paper also discusses the limitations of the t-as-z approach and the potential issues with using LRTs, such as their sensitivity to sample size. It concludes that the Kenward-Roger and Satterthwaite approximations are preferable for evaluating significance in mixed-effects models, especially when sample sizes are small. The paper also notes that while these methods are more accurate, they do not provide the exact degrees of freedom for linear mixed-effects models, and that results should be interpreted with caution regardless of the method used to obtain p-values. The paper recommends that users be cautious when using the LRT or t-as-z approaches, especially for small sample sizes, and suggests that the Kenward-Roger and Satterthwaite approximations are preferable for more accurate results.
Reach us at info@futurestudyspace.com
[slides and audio] Evaluating significance in linear mixed-effects models in R