PsychologyPsychometricsUniversity

AQAAPOntarioNSWCBSEGCE O-LevelMoECAPS

Intraclass Correlation (ICC)

Describes how strongly units in the same group resemble each other.

Understand the formulaSee the free derivationOpen the full walkthrough

I C C = \frac{M S _{B} - M S _{W}}{M S _{B} + ( k - 1 ) M S _{W}}

Open Full Walkthrough Try Calculator

This public page keeps the free explanation visible and leaves premium worked solving, advanced walkthroughs, and saved study tools inside the app.

Core idea

Overview

The Intraclass Correlation (ICC) is a statistical measure used in psychometrics to evaluate the reliability of ratings or measurements within grouped data. It quantifies the proportion of total variance that is explained by the differences between subjects, effectively distinguishing true score variance from measurement error.

When to use: Apply the ICC when assessing inter-rater reliability among multiple observers or when dealing with clustered data where observations are not independent. It is specifically used for continuous or interval-level data where you must account for both the correlation and the absolute level of agreement between scores.

Why it matters: In clinical psychology and research, high ICC values ensure that diagnostic tools provide consistent results regardless of who administers the test. This reliability is critical for the validity of psychological assessments, as it minimizes the risk of misdiagnosis due to observer bias or measurement noise.

Symbols

Variables

ICC = ICC, MS_B = MS Between, MS_W = MS Within, k = Num Raters

I C C

ICC

V a r iab l e

M S_{B}

MS Between

V a r iab l e

M S_{W}

MS Within

V a r iab l e

k

Num Raters

V a r iab l e

Walkthrough

Derivation

Formula: Intraclass Correlation (ICC)

Quantifies how strongly units in the same group resemble each other, used for inter-rater reliability.

Raters are a random or fixed effect depending on ICC model.
Data are continuous.

Partition variance:

Total variance is split into between-group and within-group components via ANOVA.

σ_{total}^{2} = σ_{between}^{2} + σ_{within}^{2}

Compute ICC:

ICC ranges from 0 (no agreement beyond chance) to 1 (perfect agreement).

I C C = \frac{σ _{between}^{2}}{σ _{between}^{2} + σ _{within}^{2}}

Note: There are several ICC forms (1,1), (2,1), (3,1) depending on the study design.

Result

I C C = \frac{σ _{between}^{2}}{σ _{between}^{2} + σ _{within}^{2}}

Source: University Psychology — Psychometrics

Free formulas

Rearrangements

Solve for $I C C$

Make icc the subject

I C C = \frac{M S _{B} - M S _{W}}{M S _{B} + ( k - 1 ) M S _{W}}

The Intraclass Correlation Coefficient (ICC) measures the resemblance of units within the same group.

Difficulty: 1/5

Solve for $M S_{B}$

Make msb the subject

M S_{B} = \frac{M S _{W} ( I C C ( k - 1 ) + 1 )}{1 - I C C}

Rearranging the Intraclass Correlation Coefficient formula to solve for the MS Between.

Difficulty: 3/5

Solve for $M S_{W}$

Make msw the subject

M S_{W} = \frac{M S _{B} ( 1 - I C C )}{I C C ( k - 1 ) + 1}

Rearranging the Intraclass Correlation Coefficient formula to solve for the MS Within.

Difficulty: 3/5

Solve for $k$

Make k the subject

k = \frac{M S _{B} - M S _{W} - I C C \cdot M S _{B} + I C C \cdot M S _{W}}{I C C \cdot M S _{W}}

Rearranging the Intraclass Correlation Coefficient formula to solve for the Number of Raters.

Difficulty: 3/5

The static page shows the finished rearrangements. The app keeps the full worked algebra walkthrough.

Visual intuition

Graph

Graph unavailable for this formula.

The graph of ICC against the independent variable is a hyperbolic curve that approaches a horizontal asymptote as the ratio of between-group variance to within-group variance changes. Because the formula is a ratio of linear terms, the ICC value is constrained between -1 and 1, showing a non-linear transition as the group size k influences the denominator.

Graph type: hyperbolic

Why it behaves this way

Intuition

Imagine a scatter plot where each subject's measurements form a cluster of points; the ICC assesses how tightly grouped the points are within each cluster (low $M S_{W}$ )

ICC

A statistical measure quantifying the proportion of total variance in a set of measurements that is attributable to the true differences between subjects or groups, rather than to

It tells you how much of the observed variation is due to actual differences between subjects ('signal') versus inconsistencies or errors within subjects ('noise').

M S_{B}

The Mean Square Between groups (or subjects), representing the variance component due to differences *between* the subjects or groups being measured.

This term reflects how much the subjects or groups differ from one another. If subjects are truly distinct,

M S_{B}

will be large.

M S_{W}

The Mean Square Within groups (or subjects), representing the variance component due to differences *within* the measurements for the same subject or group, often interpreted as

This term captures the inconsistency or random error in measurements taken for the same subject. If raters disagree or measurements fluctuate for the same subject,

M S_{W}

will be large.

The number of ratings or measurements obtained for each subject or group.

It scales the influence of within-group variability in the denominator. A larger 'k' means that within-group variability (

M S_{W}

) has a greater impact on the overall denominator, reflecting the increased opportunity for

Signs and relationships

MS_B - MS_W (numerator): This difference represents the 'signal' (variance between subjects) minus the 'noise' (variance within subjects). A positive numerator indicates that the variability between subjects is greater than the variability
(k-1)MS_W (denominator component): The (k-1) factor scales the within-group variance ( $M S_{W}$ ) in the denominator. It reflects that as the number of measurements per subject (k)

Free study cues

Insight

Canonical usage

The Intraclass Correlation (ICC) is a dimensionless statistical measure, typically reported as a value between 0 and 1, quantifying the reliability or agreement among measurements.

Common confusion

Students sometimes mistakenly try to assign a unit to the ICC itself, or worry about the specific units of the raw scores. However, only the consistency of units for the mean squares ( $M S_{B}$ and $M S_{W}$ )

Dimension note

The Intraclass Correlation (ICC) is inherently dimensionless. It is calculated as a ratio of mean squares (variances), where the units of the original measurements in the numerator and denominator cancel out, resulting

Unit systems

$M S_{B}$ dimensionless · Mean Square Between groups. If the original measurements are dimensionless (e.g., scores on a rating scale), then MS_B is also dimensionless. Its unit (or lack thereof) must be consistent with MS_W.

$M S_{W}$ dimensionless · Mean Square Within groups. If the original measurements are dimensionless, then MS_W is also dimensionless. Its unit (or lack thereof) must be consistent with MS_B.

$k$ dimensionless · Number of raters or measurements per subject. This is a count, and therefore inherently dimensionless.

Ballpark figures

Quantity:

One free problem

Practice Problem

A psychology study measures the anxiety levels of 3 siblings (k = 3) across several families. If the Mean Square Between-groups (msb) is 15.0 and the Mean Square Within-groups (msw) is 5.0, calculate the Intraclass Correlation (icc).

MS Between15

MS Within5

Num Raters3

Solve for: $i cc$

Hint: Subtract msw from msb for the numerator, then divide by the sum of msb and (k - 1) times msw.

The full worked solution stays in the interactive walkthrough.

Study smarter

Tips

Ensure the data follows a normal distribution for both between-group and within-group components.
Check whether your research design requires a one-way or two-way model before selecting an ICC variant.
Values closer to 1.0 indicate excellent reliability, whereas values near 0 suggest the ratings are no more consistent than random chance.

Common questions

Frequently Asked Questions

Quantifies how strongly units in the same group resemble each other, used for inter-rater reliability.

Apply the ICC when assessing inter-rater reliability among multiple observers or when dealing with clustered data where observations are not independent. It is specifically used for continuous or interval-level data where you must account for both the correlation and the absolute level of agreement between scores.

In clinical psychology and research, high ICC values ensure that diagnostic tools provide consistent results regardless of who administers the test. This reliability is critical for the validity of psychological assessments, as it minimizes the risk of misdiagnosis due to observer bias or measurement noise.

Ensure the data follows a normal distribution for both between-group and within-group components. Check whether your research design requires a one-way or two-way model before selecting an ICC variant. Values closer to 1.0 indicate excellent reliability, whereas values near 0 suggest the ratings are no more consistent than random chance.

References

Sources

Psychometric Theory by Jum C. Nunnally and Ira H. Bernstein
Applied Psychometrics: The Statistical Foundations of Psychological Measurement by Robert J. Gregory
Wikipedia: Intraclass correlation
Intraclass correlation (Wikipedia article)
Statistical Methods for Rates and Proportions by Joseph L. Fleiss, Bruce Levin, and Myunghee Cho Paik
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420-428.
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research.
Streiner, D. L., Norman, G. R., & Cairney, N. (2015). Health Measurement Scales: A Practical Guide to Their Development and Use (5th ed.).

Intraclass Correlation (ICC)

Overview

Variables

Derivation

Partition variance:

Compute ICC:

Rearrangements

Graph

Intuition

Insight

Practice Problem

Tips

Related Formulas

Cronbach's Alpha

ANOVA F-Ratio

Pearson's Correlation (r)

Frequently Asked Questions

Sources