The Limitations of Machine Learning in Replacing Physical Laws: Expanding the Critique

Before diving into my analysis, I want to acknowledge the insightful blog by Mehmet Süzen that inspired this post, which eloquently discusses the fallacy of replacing physical laws with machine-learned inference systems. Having read it, I felt compelled to share my own perspective and expand on these critical arguments with additional examples from recent literature and research.

The Fundamental Problem of Circular Reasoning

The original blog brilliantly identifies the circular reasoning inherent in claiming that machine learning systems can discover or replace physical laws. This point deserves further emphasis: when a neural network is trained on data generated by known physical principles, it cannot be said to “discover” those same principles through inference.

Consider recent work in fluid dynamics, where physics-informed neural networks (PINNs) have gained popularity. The paper “Discovery of Physics From Data: Universal Laws and Discrepancies” highlights that “the naive application of ML/AI will generally be insufficient to infer universal physical laws without further modification”. The authors demonstrate this by examining falling objects, showing that measurement noise and secondary mechanisms (like fluid drag) obscure the underlying law of gravitation, leading to erroneous models that might suggest an Aristotelian theory where objects fall at speeds related to their mass, rather than identifying the true universal gravitational constant.

This illustrates perfectly how ML systems trained on physical data will incorporate all the complexities and noise present in that data, rather than abstracting to the elegant, universal laws that human scientists have carefully identified through theoretical reasoning and controlled experimentation.

Beyond Narrow Applications: The Generalization Problem

The original blog correctly identifies the problem of faulty generalization. Machine learning algorithms excel at computational acceleration within narrowly defined parameter spaces, but struggle with broader generalization.

A fascinating discussion on Reddit highlights this limitation: “In addition, the ‘marginally-better SOTA’-esque papers with no novel methods or aspects besides some parameter tuning or adding extra layers to the DNN are also tiring to read. The wall of math then exists only to provide a sense of rigor and novelty, obscuring the iterative nature lacking novelty”. This reflects how ML approaches in physics often claim breakthroughs that are actually just incremental improvements in limited domains.

Another illustrative example comes from the field of symbolic regression. While the AbdusSalam et al. paper in Physical Review D demonstrates how symbolic regression can help derive analytical expressions for physics beyond the Standard Model, the authors position it as a tool to assist numerical studies, not as a replacement for physical theory. The expressions derived still rely on the underlying physics-based model (the constrained minimal supersymmetric Standard Model) and serve primarily to accelerate computation, not to discover new physical laws.

The Irreplaceable Role of Scientists in Establishing Causality

Perhaps the most important point from the original blog is that causality still requires scientists. Machine learning excels at finding correlations but struggles with identifying true causal relationships.

The Amazon Science blog on physics-constrained machine learning notes that “the predictions of deep-learning models trained on physical data typically ignore fundamental physical principles. Such models might, for instance, violate system conservation laws” (see here). This highlights why human scientists remain essential - they understand that physical laws must adhere to conservation principles, symmetries, and other fundamental constraints that ML systems don’t inherently respect.

A conversation on Reddit about Physics Informed Neural Networks (PINNs) further illuminates this issue. One commenter precisely notes: “The point of including a physical loss function, in addition to a data-driven loss, is to impose inductive bias into the training process”. This human-guided approach to incorporating physics into ML demonstrates that we’re not replacing physics with ML, but rather using our understanding of physics to guide ML - the exact opposite of what some overenthusiastic claims suggest.

The Scientific Machine Learning Fallacy: A Deeper Look

The term “Scientific Machine Learning Fallacy” coined in the original blog deserves broader recognition. Claims of “machine scientists” or “automated scientific discovery” fundamentally misunderstand the nature of scientific inquiry.

A recent paper on “Scientific machine learning for closure models in multiscale problems” acknowledges that “the generalizability and interpretability of learned models is a major issue that needs to be addressed further”. This admission from researchers in the field underscores the gap between current ML capabilities and true scientific discovery.

The Conversation article about an “AI scientist” further reveals the limits of these approaches. While Sakana AI Labs claims their system can “make scientific discoveries in the area of machine learning in a fully automated way,” the article questions whether such a system can produce truly “interesting” scientific papers, noting that “good science requires novelty”. The ability to generate papers that look like scientific literature doesn’t equate to generating novel scientific insights or laws.

The AutoML Misnomer and Meta-Scientific Work

I strongly agree with the original blog that “AutoML” is a misnomer in scientific contexts. These systems don’t replace scientists but rather change the nature of scientific work.

The paper on “Combining physical modeling and machine learning for micro-scale modeling of a fuel cell electrode” demonstrates this well. It describes a “comprehensive transition from white-box models, characterized by their reliance on physical laws, to black-box models exemplified by neural networks”. Yet the core contribution isn’t replacing physics but creating a “synergistic integration” where neural networks complement physical modeling.

This represents what the original blog aptly calls “MetaML” - a transformation of scientific workflows rather than a replacement of scientific thinking.

The Proper Role: Augmentation, Not Replacement

To conclude, I believe the most productive path forward is viewing machine learning as an augmentation to physical sciences, not a replacement. The paper on “Learning physical laws: the case of micron size particles in dielectric fluid” demonstrates this approach well, noting that “representation structure is key in learning generalizable models”. The authors use “the port-Hamiltonian formalism as a high level model structure” that is “continuously refined based on our understanding of the physical process.” This integration of physics understanding with machine learning represents the right approach.

Similarly, the work on “Discovering Artificial Viscosity Models for Discontinuous Galerkin Approximation of Conservation Laws” shows how physics-informed machine learning can automate the discovery of models - but within a physics-informed framework, not replacing it

In summary, while machine learning offers powerful tools for scientific research, the fallacy of replacing physical laws with learned models deserves continued critical attention. True scientific progress will come from the thoughtful integration of machine learning with physical understanding, not from claims that ML can autonomously discover or replace the fundamental laws of nature. The original blog’s warning about circular reasoning, faulty generalization, and the continued need for human scientists remains prescient and worthy of expansion as these technologies continue to develop.

Human Thinking, Not Machine Imitation

This philosophical depth reminds us that true scientific thinking involves not just pattern recognition and prediction, but deep conceptual understanding that may not be reducible to computational processes. When we forget this, we risk confusing the map (our mathematical models and computational simulations) with the territory (physical reality itself).