Infinitely Wide Neural Networks

How useful are infinitely wide neural networks? I tried writing a Jupyter notebook to explore the idea.

https://ericmjl.github.io/essays-on-data-science/machine-learning/nngp/