首页 » 文章 » 文章详细信息
eLife Volume 7 ,2018-07-01
Assessing reliability in neuroimaging research through intra-class effect decomposition (ICED)
Andreas M Brandmaier 1 , 2 Elisabeth Wenger 1 Nils C Bodammer 1 Simone Kühn 3 Naftali Raz 1 , 4 Ulman Lindenberger 1 , 2
Show affiliations
Received 2018-02-06, accepted for publication 2018-07-01, Published 2018-07-01

10.7554/eLife.35718.001Magnetic resonance imaging has become an indispensable tool for studying associations of structural and functional properties of the brain with behavior in humans. However, generally recognized standards for assessing and reporting the reliability of these techniques are still lacking. Here, we introduce a new approach for assessing and reporting reliability, termed intra-class effect decomposition (ICED). ICED uses structural equation modeling of data from a repeated-measures design to decompose reliability into orthogonal sources of measurement error that are associated with different characteristics of the measurements, for example, session, day, or scanning site. This allows researchers to describe the magnitude of different error components, make inferences about error sources, and inform them in planning future studies. We apply ICED to published measurements of myelin content and resting state functional connectivity. These examples illustrate how longitudinal data can be leveraged separately or conjointly with cross-sectional data to obtain more precise estimates of reliability.


Human;individual differences;coefficient of variation;G theory;structural equation modeling;intra-class correlation;reliability


© 2018, Brandmaier et al
http://creativecommons.org/licenses/by/4.0/This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.


Andreas M Brandmaier,Elisabeth Wenger,Nils C Bodammer,Simone Kühn,Naftali Raz,Ulman Lindenberger. Assessing reliability in neuroimaging research through intra-class effect decomposition (ICED). eLife ,Vol.7(2018)



[1] M Arshad, JA Stanley, N Raz. (2018). Reliability of myelin water fraction in ALIC. OSF . DOI: 10.1002/hbm.23481.
[2] LJ Cronbach, GC Gleser, H Nanda, N Rajaratnam. et al.(1972). The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. DOI: 10.1002/hbm.23481.
[3] M Arshad, JA Stanley, N Raz. (2017). Test-retest reliability and concurrent validity of in vivo myelin content indices: Myelin water fraction and calibrated T w/T w image ratio. Human Brain Mapping.38:1780-1790. DOI: 10.1002/hbm.23481.
[4] JJ McArdle. (1980). Causal modeling applied to psychonomic systems simulation. Behavior Research Methods & Instrumentation.12:193-209. DOI: 10.1002/hbm.23481.
[5] E Filevich, N Lisofsky, M Becker, O Butler. et al.(2017). Day2day: investigating daily variability of magnetic resonance imaging measures over half a year. BMC Neuroscience.18. DOI: 10.1002/hbm.23481.
[6] JP Lerch, AJ van der Kouwe, A Raznahan, T Paus. et al.(2017). Studying neuroanatomy using MRI. Nature Neuroscience.20:314-326. DOI: 10.1002/hbm.23481.
[7] RC Craddock, PE Holtzheimer, XP Hu, HS Mayberg. et al.(2009). Disease state prediction from resting state functional connectivity. Magnetic Resonance in Medicine.62:1619-1628. DOI: 10.1002/hbm.23481.
[8] JJ Bartko. (1966). The intraclass correlation coefficient as a measure of reliability. Psychological Reports.19:3-11. DOI: 10.1002/hbm.23481.
[9] DV Cicchetti, SA Sparrow. (1981). Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. American Journal of Mental Deficiency.86:127-137. DOI: 10.1002/hbm.23481.
[10] T Sørensen. (1948). A method of establishing groups of equal amplitude in plant sociology based on similarity of species and its application to analyses of the vegetation on danish commons. Biologiske Skrifter.5:1-34. DOI: 10.1002/hbm.23481.
[11] C Finkbeiner. (1979). Estimation for the multiple factor model when data are missing. Psychometrika.44:409-420. DOI: 10.1002/hbm.23481.
[12] RA Morey, ES Selgrade, HR Wagner, SA Huettel. et al.(2010). Scan-rescan reliability of subcortical brain volumes derived from automated segmentation. Human Brain Mapping.31:1751-1762. DOI: 10.1002/hbm.23481.
[13] DD Garrett, GR Samanez-Larkin, SW MacDonald, U Lindenberger. et al.(2013). Moment-to-moment brain signal variability: a next frontier in human brain mapping?. Neuroscience & Biobehavioral Reviews.37:610-624. DOI: 10.1002/hbm.23481.
[14] JJ McArdle. (1994). Structural factor analysis experiments with incomplete data. Multivariate Behavioral Research.29:409-454. DOI: 10.1002/hbm.23481.
[15] JB Willett. (1989). Some results on reliability for the longitudinal measurement of change: implications for the design of studies of individual growth. Educational and Psychological Measurement.49:587-602. DOI: 10.1002/hbm.23481.
[16] T Raykov, GA Marcoulides, T Patelis. (2015). The importance of the assumption of uncorrelated errors in psychometric theory. Educational and Psychological Measurement.75:634-647. DOI: 10.1002/hbm.23481.
[17] T von Oertzen. (2010). Power equivalence in structural equation modelling. British Journal of Mathematical and Statistical Psychology.63:257-272. DOI: 10.1002/hbm.23481.
[18] Y Rosseel. (2012). : an R package fo structural equation modeling. Journal of Statistical Software.48:1-36. DOI: 10.1002/hbm.23481.
[19] PD Bliese. (2000). Multilevel Theory, Research, Andmethods in Organizations: Foundations, Extensions, and New Directions:349-381. DOI: 10.1002/hbm.23481.
[20] SM Boker, JJ McArdle, M Neale. (2002). An algorithm for the hierarchical organization of path diagrams and calculation of components of expected covariance. Structural Equation Modeling: A Multidisciplinary Journal.9:174-194. DOI: 10.1002/hbm.23481.
[21] S Noble, MN Spann, F Tokoglu, X Shen. et al.(2017). Influences on the Test-Retest reliability of functional connectivity MRI and its relationship with behavioral utility. Cerebral Cortex.27:5415-5429. DOI: 10.1002/hbm.23481.
[22] NU Dosenbach, B Nardos, AL Cohen, DA Fair. et al.(2010). Prediction of individual brain maturity using fMRI. Science.329:1358-1361. DOI: 10.1002/hbm.23481.
[23] DG Gee, SC McEwen, JK Forsyth, KM Haut. et al.(2015). Reliability of an fMRI paradigm for emotional processing in a multisite longitudinal study. Human Brain Mapping.36:2558-2579. DOI: 10.1002/hbm.23481.
[24] RV de Schoot, H Hoijtink, R Jan-Willem. (2011). Moving beyond traditional null hypothesis testing: evaluating expectations directly. Frontiers in Psychology.2. DOI: 10.1002/hbm.23481.
[25] AM Brandmaier, T von Oertzen, P Ghisletta, C Hertzog. et al.(2015). LIFESPAN: a tool for the computer-aided design of longitudinal studies. Frontiers in Psychology.6. DOI: 10.1002/hbm.23481.
[26] LR Dice. (1945). Measures of the amount of ecologic association between species. Ecology.26:297-302. DOI: 10.1002/hbm.23481.
[27] AM Brandmaier, T von Oertzen, P Ghisletta, U Lindenberger. et al.(2018). Precision, reliability, and effect size of slope variance in latent growth curve models: implications for statistical power analysis. Frontiers in Psychology.9. DOI: 10.1002/hbm.23481.
[28] EM Gordon, TO Laumann, AW Gilmore, DJ Newbold. et al.(2017). Precision functional mapping of individual human brains. Neuron.95:791-807. DOI: 10.1002/hbm.23481.
[29] JR Nesselroade. (1991). The Warp and Woof of the Developmental Fabric Hillsdale:213-240. DOI: 10.1002/hbm.23481.
[30] T von Oertzen, C Hertzog, U Lindenberger, P Ghisletta. et al.(2010). The effect of multiple indicators on the power to detect inter-individual differences in change. British Journal of Mathematical and Statistical Psychology.63:627-646. DOI: 10.1002/hbm.23481.
[31] LJ Cronbach. (1957). The two disciplines of scientific psychology. American Psychologist.12:671-684. DOI: 10.1002/hbm.23481.
[32] C Hedge, G Powell, P Sumner. (2018). The reliability paradox: why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods.50:1166-1186. DOI: 10.1002/hbm.23481.
[33] T von Oertzen, AM Brandmaier. (2013). Optimal study design with identical power: an application of power equivalence to latent growth curve models. Psychology and Aging.28:414-428. DOI: 10.1002/hbm.23481.
[34] M Pannunzi, R Hindriks, RG Bettinardi, E Wenger. et al.(2018). Corrigendum to “Resting-state fMRI correlations: From link-wise unreliability to whole brain stability”. NeuroImage. 174:599-604. DOI: 10.1002/hbm.23481.
[35] WP Vispoel, CA Morris, M Kilinc. (2018). Applications of generalizability theory and their relations to classical test theory and structural equation modeling. Psychological Methods.23:1-26. DOI: 10.1002/hbm.23481.
[36] T von Oertzen, AM Brandmaier, S Tsang. (2015). Structural equation modeling with ωnyx. Structural Equation Modeling: A Multidisciplinary Journal.22:148-161. DOI: 10.1002/hbm.23481.