Determining Predictor Importance in Multilevel Models for Longitudinal Data: An Extension of Dominance Analysis

Abstract

Longitudinal models are used not only to analyze the change of an outcome over time but also to describe what person-level and time-varying factors might influence this trend. Whenever a researcher is interested in the factors or predictors impacting an outcome, a common follow-up question asked is that of the relative importance of such factors. Hence, this study aimed to extend and evaluate Dominance Analysis (DA), a method used to determine the relative importance of predictors in various linear models (Budescu, 1993; Azen & Budescu, 2003; Azen, 2013), for use with longitudinal multilevel models. A simulation study was conducted to investigate the effect of number of measurement occasions (level-1 units), number of subjects (level-2 units), different levels of model complexity (i.e., number of predictors at level-1 and level-2), size of predictor coefficients, predictor collinearity levels, misspecification of the covariance structure, and measures of model fit on DA results and provide recommendations to researchers who wish to determine the relative importance of predictors in longitudinal multilevel models. Results indicated that number of subjects was the most important factor influencing the accuracy of DA in rank-ordering the model predictors, and that more than 50 subjects are needed to obtain adequate power and confidence in the reproducibility of DA results. The McFadden pseudo R² is recommended as the standard measure of fit to use when performing DA in multilevel longitudinal models. Finally, asymptotic standard error and percentile confidence intervals constructed through bootstrapping can be used to determine if one predictor significantly dominates another but might not provide sufficient power unless there are at least 200 subjects in the sample or the magnitude of the general dominance difference measure is greater than 0.01 using McFadden’s R².