The place for overparametrized models is tooling
I think that the place where overparametrized models make the most sense for application are not in models with "high explanatory power" (see The goal of scientific model building is high explanatory power). Rather, their best realms of application are a bit more pedantic. We don't always make these large, overparametrized models to explain the world, per se, but to automate some task that could have been done by a human being. In other words, we come right back to the notion of using computing to automate routine tasks. Basically one of the two classes of tasks:
Both of these are things that would have originally required human intervention.
Seems to me, then, that many so-called "AI" applications are merely (yes, merely) about automating tasks that would have otherwise taken much longer with humans.
Other references:
The end goals of business data science
What are the end goals of business data science?
I think one of them is automation, being able to capture value from existing business processes by automating out manual, repetitive tasks, so that humans can be free to do other things.
Data science, as applied directly in most business contexts, usually revolves around business goals that are easily-defined in monetary terms, and where well-established processes are available to optimize. Put bluntly, to be able to save on costs in a business process, or to increase the amount of profit extracted from the business process.
Representation Learning
Research vs Business Data Science
One of my colleagues (well, strictly speaking my boss' boss) recently crystallized a very important and key idea for my colleagues: the difference between biomedical research data science and tech business data science. I gave his ideas some thought, and decided to pen down what I saw as the biggest similarities and differences.
The goals between the two "forms" of data science are different:
There are issues that I'm seeing in the data science field. Some of the problems I have seen thus far.
And what I think is needed:
The key difference, I think is that The end goals of business data science is about capturing value from existing processes, while The end goals of research data science is about expanding new avenues of value from unknown, un-developed, and un-captured business processes. The latter is and has always been an investment to make; in a well-oiled system, the former likely generates profit that can and should be invested in the latter.
The goal of scientific model building is high explanatory power
Why does mechanistic thinking matter? In The end goals of research data science, we are in pursuit of the invariants, i.e. knowledge that stands the test of time. (How our business contexts exploit that knowledge for win-win benefit of society and the business is a matter to discuss another day).
When we build models, particularly of natural systems, predictive power matters only in the context of explanatory power, where we can map phenomena of interest to key parameters in a model. For example, in an , the autoregressive coefficient may correspond to a meaningful properly in our research context.
Being able to look at a natural system and find the most appropriate model for the system is a key skill for winning the trust of the non-quantitative researchers that we serve. (ref: Finding the appropriate model to apply is key)