Anatomy of a probabilistic programming framework

From George Foo: https://eigenfoo.xyz/prob-prog-frameworks/

Key ingredients of a probabilistic programming framework:

- Language for specifying a model.
- Library of probability distributions + facilities to specify arbitrary distributions.
- Inference algorithm belonging to at least one of MCMC or VI.
- An optimizer, to compute mode of posterior density.
- Autodiff library, to compute gradients for items 3 and 4 (inference algo + optimizer)
- Diagnostics suite to analyze quality of inference.

PyMC3 provides a whole lot of these, alongside ArviZ!

Reading Bazaar

Interesting reads from my random walk over the internet

- Enterprise Software Monetization is Fat-Tailed
- Microsoft is allowing employees to go remote permanently
- Notes from Work Rules on hiring
- Limitations of Graph Neural Networks
- Running Python on .NET 5
- Why are tech companies making custom typefaces
- Anatomy of a probabilistic programming framework
- Will the M1 macs run the PyData stack
- Electric bicycles that double up as cars
- Avoiding technical debt with ML pipelines

Notes on Statistics

Some learnings while training myself in statistics.

Distributions:

- Probability distribution
- Dirichlet Process
- Dirichlet Distribution
- Estimating a multivariate Gaussian's parameters by gradient descent
- Gaussians come from processes that are additive

Statistical Estimation:

- Fermi estimation and Bayesian priors
- When is your statistical model right vs. good enough
- Maximum score estimator is used to maximize classification true positives and negatives

Papers that I'm writing:

Dealing with data:

Probabilistic Programming: