MathLiveMutual information
Mutual information measures how much knowing one random variable reduces uncertainty about another, counting shared information in bits or nats. It is computed as the sum of the two individual entropies minus their joint entropy.
I(X;Y) = H(X) + H(Y) - H(X,Y)What the symbols mean
| mutual information between X and Y (bits or nats) | |
| entropy of X (information content of X) | |
| entropy of Y (information content of Y) | |
| joint entropy of X and Y |
About this formula
- FieldMathematics
- TopicInformation theory
- LevelUniversity
- Also calledtransinformation content, transferred information, transmitted information, information gain between variables, mutual information formula, entropy mutual information
- SourceWikidata Q252973
Write it in Obsidian
MathLive is a free visual math editor for Obsidian. Run Find a formula, type “mutual information”, and it goes into your note as editable LaTeX. You can also scan formulas from screenshots, PDFs and handwriting.