Intraclass correlation coefficient / reliability study

This function calculates the intraclass correlation coefficient (ICC) using the methods described by Shrout, P.E., and Fleiss, J.L "Intraclass correlations: uses in assessing rater reliability," Psychological Bulletin, 1979, 86, 420-428.

 

There are six intraclass correlations:

 

ICC(1,1):  used when each subject is rated by multiple raters, raters assumed to be randomly assigned to subjects, all subjects have the same number of raters.

ICC(2,1):  used when all subjects are rated by the same raters who are assumed to be a random subset of all possible raters.

ICC(3,1):  used when all subjects are rated by the same raters who are assumed to be the entire population of raters.

ICC(1,k):  Same assumptions as ICC(1,1) but reliability for the mean of k ratings.

ICC(2,k):  Same assumptions as ICC(2,1) but reliability for the mean of k ratings.

ICC(3,k):  Same assumptions as ICC(3,1) but reliability for the mean of k ratings. Assumes additionally no subject by judges interaction.

 

Method used for each correlation coefficient is as below:

ICC(1,1):

One-way random single measures

ICC(1,k):

One-way random average measures

ICC(2,1):

Two-way random single measures (Consistency/Absolute agreement)

ICC(2,k):

Two-way random average measures (Consistency/Absolute agreement)

ICC(3,1):

Two-way mixed single measures (Consistency/Absolute agreement)

ICC(3,k):

Two-way mixed average measures (Consistency/Absolute agreement)

 

A sample screen shot of input window:

ScreenHunter_01 Jan. 12 18.23.gif

Your data could be organized so that each class has one line (one record) or multiple lines. You can specify the data structure, and then select the method to calculate the desired ICC.

Sample output and explanation:

Intraclass correlation coefficient

        icc.lme         LCL        ICC1        UCL

BMI 0.217001593  0.13987535  0.20907559 0.28579992

SBP 0.061762465 -0.02357091  0.03137998 0.09617033

DBP 0.002549532 -0.07823121 -0.03037675 0.02726936

                                                                          

 icc.lme: intraclass correlation computed using one way random effect model

                                              

 ICC1: The intraclass correlation coefficient 1

                                                         

 ICC1: intraclass correlation computed using one way ANOVA

                                                                      

 LCL, UCL: 95%CI of intraclass correlation computed using one way ANOVA

                                                                                           

 ICC from random effect model should be equal an ICC from a one-way ANOVA only when the data are balanced (equal group sizes for all groups and no missing data).

                                                                                 

 The icc.lme should be a more accurate measure of ICC in all other instances.