Data & ComputingInformation TheoryUniversity
AQAAPOntarioNSWCBSEGCE O-LevelMoECAPS

Mutual Information (2×2) Calculator

Mutual information between two binary variables from joint probabilities.

Use the free calculatorCheck the variablesOpen the advanced solver
Open Advanced Calculator
This is the free calculator preview. Advanced walkthroughs stay in the app.
Result
Ready
Mutual Information

Formula first

Overview

Mutual Information quantifies the statistical dependence between two discrete random variables by measuring how much information is shared between them. In the 2×2 contingency case, it calculates the Kullback-Leibler divergence between the joint probability distribution and the product of the marginal distributions of two binary variables.

Symbols

Variables

I(X;Y) = Mutual Information, = P(X=0,Y=0), = P(X=0,Y=1), = P(X=1,Y=0), = P(X=1,Y=1)

I(X;Y)
Mutual Information
nats
P(X=0,Y=0)
Variable
P(X=0,Y=1)
Variable
P(X=1,Y=0)
Variable
P(X=1,Y=1)
Variable

Apply it well

When To Use

When to use: Apply this formula when analyzing the relationship between two binary variables, such as comparing a test result with the presence of a disease. It is preferred over linear correlation when you need to capture non-linear dependencies or general statistical association.

Why it matters: It is a foundational concept in communication theory for calculating channel capacity and in machine learning for feature selection. High mutual information indicates that knowing the state of one variable significantly reduces uncertainty about the other.

Avoid these traps

Common Mistakes

  • Forgetting to normalize probabilities to sum to 1.
  • Mixing logs (ln vs log2) and units (nats vs bits).

One free problem

Practice Problem

A researcher is studying the link between a specific gene mutation and a rare trait. In a perfectly balanced population, the joint probabilities are all equal (0.25 each). Calculate the Mutual Information.

P(X=0,Y=0)0.25
P(X=0,Y=1)0.25
P(X=1,Y=0)0.25
P(X=1,Y=1)0.25

Solve for:

Hint: If the joint probability of every cell is equal to the product of its marginal probabilities, the variables are independent.

The full worked solution stays in the interactive walkthrough.

References

Sources

  1. Cover, Thomas M., and Joy A. Thomas. Elements of Information Theory. 2nd ed. Wiley-Interscience, 2006.
  2. Wikipedia: Mutual Information
  3. Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory (2nd ed.). Wiley.
  4. Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory (2nd ed.). Wiley-Interscience.
  5. Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379-423.