Eric J Ma's Website

Probability Simplex

written by Eric J. Ma on 2021-05-05 | tags: bayesian probability til

Today I learned about the definition of the term "probability simplex". Here's what it is.

Doing another TIL (as inspired by Simon Willison), this time being the definition of a probability simplex.

I gleaned a pretty good definition of the probability simplex from this article. From a computational perspective, it is represented as:

  1. a vector of $K$ numbers ($K$ being the dimension of the simplex) that lie between 0 and 1, such that
  2. their sum is equal to 1.0 (it is a probability vector, after all), and
  3. each vector slot represents a choice mutually exclusive with others.

Here are a few examples.

  • A probability simplex where $K = 2$ might be the vector $(0.15, 0.85)$.
  • A probability simplex where $K = 3$ might be the fector $(0.10, 0.50, 0.40)$.

Probability simplices are usually used as the probability parameter in the Multinomial distribution. (Reminder: the Binomial is a special case of the Multinomial.) In Bayesian inference, we also place priors over the probability parameter (or probability simplices) to express what we might believe about the relative tendency to pick one choice over another.
Cite this blog post: 
@article{
    ericmjl-2021-probability-simplex,
    author = {Eric J. Ma},
    title = {Probability Simplex},
    year = {2021},
    month = {05},
    day = {05},
    howpublished = {\url{https://ericmjl.github.io}},
    journal = {Eric J. Ma's Blog},
    url = {https://ericmjl.github.io/blog/2021/5/5/probability-simplex},
}

I send out a newsletter with tips and tools for data scientists. Come check it out at Substack.

If you would like to sponsor the coffee that goes into making my posts, please consider GitHub Sponsors!

Finally, I do free 30-minute GenAI strategy calls for teams that are looking to leverage GenAI for maximum impact. Consider booking a call on Calendly if you're interested!