Wittgenstein and Boltzmann—
Language, Probability,
and the Uncertainties of Communication
Elizabeth Rohwer - erohwer@san.rr.com
Boltzmann's breakthrough in statistical thermodynamics provides valuable insights for elucidating the Tractatus’ main point, which Wittgenstein defines as “the theory of what can be expressed by propositions.” The connection between these two fields, mediated by probability theory, reflects Boltzmann’s influence on Wittgenstein. However, this connection has not been fully explored, as it took the scientific community over a century to formalize and explain the generality of Boltzmann's entropy law. The statistical nature of this law created a paradigm shift in physics, and the delay in its acceptance hindered understanding of Wittgenstein’s treatment of probability, leaving his Propositional Theory, where probability assumes a pivotal position, elusive. Consequently, a coherent, comprehensive and universally agreed-upon interpretation of the Tractatus remains unattained. The paper offers a new interpretational framework that addresses this shortcoming.
1. Boltzmann’s Scientific Enigma and the Problem with Interpreting the Tractatus
Boltzmann developed a mathematical framework that quantifies the Second Law of Thermodynamics. However, despite its practical effectiveness, his Maximum Entropy law (ME) could not be explained within the existing framework of classical mechanics, as it relied on statistical calculations. In contrast to Newton's deterministic and time-reversible laws, Boltzmann's ME is probabilistic and non-reversible, operating differently when applied to the forward or backward directions of time. This lack of theoretical explanation was so perplexing that, in 1906—the year Wittgenstein intended to study under him—Boltzmann committed suicide, unable to withstand the criticism from his colleagues. The scientific community struggled to adopt a probabilistic worldview and acknowledge its inherent uncertainty, which is the modern perspective.
The formalism of probability theory underpinning Boltzmann’s statistical mechanics was also at play in quantum mechanics; yet, a fundamental mystery persisted. This scientific enigma is encapsulated in Einstein's 1921 remark:“In so far as the statements of mathematics speak about reality, they are not certain, and in so far as they are certain, they do not speak about reality.” Likewise, the failure to consider the role of uncertainty in the realm of language communication has made it challenging to track down, conceptualize, and examine Wittgenstein’s Propositional Theory. However, this oversight is understandable given that the concepts of uncertainty and entropy were not clearly defined at the time. “We can reason clearly only in areas where we have an established symbolism for the concepts,” noted E.T. Jaynes (1922-1998), another theoretical physicist and one of the fathers of modern AI.
2. Wittgenstein Re-applies Boltzmann’s Approach to the Calculus of Inductive Reasoning
It took a long time to equate Boltzmann’s entropy formula, famously engraved on his tombstone, with the measure of uncertainty—its modern interpretation. Physicists and mathematicians such as Frank Ramsey (1903-1930), Bruno de Finetti (1906-1985), R.T. Cox (1898-1991), and Claude Shannon (1916-2001), among others, contributed to the development of modern probability theory, which links the two concepts. Their work established a sound mathematical formalism behind the Second Law of Thermodynamics, thus elevating Boltzmann’s ME to a foundational principle in physics. Boltzmann’s “line of thinking” was adopted not only across various scientific fields but also in philosophy. Wittgenstein himself admitted to having “simply straight away seized on it [as well as on the work of others] with enthusiasm for [his own] work of clarification.”
E. T. Jaynes’ research, in particular, had a significant impact on the development of Boltzmann’s probabilistic approach, aligning philosophically with Wittgenstein's “work of clarification”. Jaynes traced the progression of the technical ideas that led to Boltzmann's breakthrough, “showing that the Second Law of Thermodynamics became, over a Century ago, a general principle of reasoning that can be applied to scientific inference across various fields beyond thermodynamics.” [my emphasis] The remarks on probability in the Tractatus suggest that Wittgenstein had intuitively grasped the universality of Boltzmann's statistical method, as he tackled the problem of logical inference, formulated his Propositional Theory, and built his philosophical model influenced by it.
Wittgenstein re-applied Boltzmann’s innovative use of probabilities in thermodynamics to the challenges of a different domain: that of logic and language. This move, in itself, is a major philosophical (and technical) achievement that was difficult to comprehend at the time; however, it has been corroborated by Jaynes' remark: "What’s new in this field [thermodynamics] is not the method, but the recognition of its generality.” Jaynes notably proved that the formalism of probability theory, an early version of which can be found in the Tractatus, is, in fact, the calculus of inductive reasoning.
3. Jaynes’ Robot—A Pragmatic Answer to the Uncertainty Puzzle
E.T. Jaynes broadened the scope of statistical inference to include both communication theory and thermodynamics as special cases, showed that probability is a form of extended logic, and in the last years of his life, laid down the normative principles of logic. To formalize these principles, he devised a mathematical abstraction in the form of an idealized linguistic agent endowed with common sense, known as Jaynes’ Robot. This artificially intelligent construct is designed to reason about propositions as humans do and to be able to communicate with us, impersonating a human.
It is noteworthy that Wittgenstein devotes the longest (technical) section of his book, section five, to probability theory and the development of the notation of truth functions and tables. The subsequent (philosophical) section presents the formula for “the general form of proposition” (TLP 6) and the graphical expressions of the propositions of logic (the tautology graphs in 6.1203). This notation in the form of tables, functions and graphs, introduces the concepts and symbols that constitute the formalism of his Propositional Theory. Understanding what makes Jaynes’ Robot operational is key to comprehending Wittgenstein’s rationale for developing this theory and its formal expression.
Jaynes’ Robot serves as a pragmatic answer to the uncertainty puzzle. It is a model of human reasoning, which in most cases—although crucially not always—effectively performs the tasks it has been trained to do. However, unlike the traditional design of a classical, deterministic Newtonian machine, Jaynes Robot’s operational mechanisms remain concealed. Thus, the uncertainty mystery is circumvented, not solved. Jaynes' innovation hinges on a crucial shift in perspective: “[I]nstead of asking, ‘How can we build a mathematical model of human common sense?’ let us ask, ‘How could we build a machine that would carry out useful plausible reasoning following clearly defined principles expressing an idealized common sense?’” This shift in perspective opens new avenues in interpreting the Tractatus.
4. Wittgenstein’s Probabilistic Model of the System of Language Communication
The challenges in comprehending Wittgenstein’s book stem from the perspective from which we engage with the text. Previous interpretations have remained incomplete due to their alignment with the direction suggested by Jaynes’ first question, which he ultimately dismisses, rather than with the engineering perspective suggested by his second, more pragmatic question. I propose a new approach to the book, contending that Wittgenstein, who had a mechanical engineering background and an aeronautical patent for an advanced turbo-propeller design, employed his model-building skills and familiarity with thermodynamics to construct a philosophical model that bears conceptual similarities to Jaynes' pragmatic model of Common Sense probabilistic reasoning.
Wittgenstein’s counterpart to Jaynes’ Robot is a philosophical abstraction of the highest order—one that is conceptually all-encompassing, mathematically accurate, and ahead of its time. The Tractatus presents a dynamic, probabilistic model of the real-life, human phenomenon of language communication. This model, labeled The World, is set in motion by a physical theory—Wittgenstein’s Propositional Theory— whose formalism aligns with the calculus of inductive reasoning; that is, the axioms and equations of probability theory.
Furthermore, the symmetrical structure of the Tractatus suggests that Wittgenstein's model shares functional similarities with Jaynes' Robot. To communicate with us, Jaynes’ Robot generates and interprets propositions. Correspondingly, sections 1, 2, and 3 of the Tractatus describe the generation of a propositional sign (which is an articulated thought in the form of a physical speech signal or written sequence of words), while sections 4, 5, and 6 engage with its formal interpretation by developing notation for truth-functional analysis. Thus, Wittgenstein’s picture theory of language outlined in section 2 and his probability theory developed in its symmetrical counterpart—section 5—should be considered to work in tandem. Like Jaynes’ Robot’s generation and interpretation processes they are intrinsically linked. Together they constitute two sides of the same coin: Wittgenstein’s Propositional Theory which sets into motion a dynamic distributed probabilistic model of the system of same-language communication between humans.
Wittgenstein wrote to his prospective publisher:
About a year before becoming a prisoner of war I finished a philosophical work on which I had worked for the preceding seven years. It is essentially the presentation of a system and this presentation is extremely compact since I have only recorded in it what—and how it has—really occurred to me… The work is strictly philosophical and simultaneously literary, and yet there is no blathering in it. [his emphasis]
The common denominator between Jaynes' Robot and Wittgenstein's model is the modern understanding and application of probabilities, which enables the normative principles of logic to be formalized as generalization across language users and interpolation over time. Probability Theory was qualitatively defined by Laplace as "Common Sense reduced to calculations”, quantified by Boltzmann's ME Principle, and validated by the success of AI applications. Modern AI relies on two computational techniques that were known at the time Wittgenstein wrote the Tractatus. I will outline them below as understanding how they work in practice sheds new light on the difficulties of comprehending the book.
5. Probabilistic and Philosophical Models Plagued by Uncertainty
Discussing model-building in philosophy, Williamson argues that, much like in science, progress is being achieved through the development of increasingly better models of the phenomena under study, especially in areas such as the philosophy of language that deal with "the human world in all its complexity and mess”. Accordingly, Wittgenstein's legacy could be construed as organised around two philosophical models: one centered on language communication, presented in the Tractatus, and a better one, associated with his later philosophy, which expands the linguistic mode of communication to encompass our actions in physical reality. Thus, the perceived obscurity of Wittgenstein’s oeuvre can be attributed to the probabilistic nature of human interactions, which reflects the uncertainty inherent in our reality.
This probabilistic aspect of model-building in philosophy is further explored by Titelbaum, who differentiates between descriptive and normative models. He defines normative models as abstract structures that may not have any “interesting ontological commonalities” with the phenomena being modeled and conducts their analysis within a statistical framework. In this context, Titelbaum supports the view that normative models are not intended to model or explain actual behavior, but rather show how agents ought to act. The deviation of the actual from the expected is a sign of uncertainty being at play.
Wittgenstein depicts this misalignment with a sketch of Maxwell’s method of mechanical models in his Notebooks entry of 15.11.14. His drawing reveals an attempt to apply a probabilistic method in the analysis of the “Projection” of the “Model (Picture)” onto “Reality”. The excruciating difficulty of the task is seen in the entry he wrote on the next day in his secret diary:
Again no clarity of vision, although I am evidently so close to finding the answer to the most profound questions that the solution is practically under my nose!!! […..] I feel that I'm standing in front of the gate itself but cannot see clearly enough to open it. It is an extremely strange state of mind that I have never before experienced as clearly as now.
6. The Inaccessible Side of Wittgenstein’s Philosophical Model
Wittgenstein did open the gate; in the preface of the Tractatus, he famously states that "the problems have in essentials been finally solved." However, he does not provide an answer to the question that Jaynes casts aside; this answer is impossible to articulate within the confines of a mere description. Although it is not the answer we expect or can easily comprehend, Wittgenstein’s solution is, in fact, the correct one. Because probability theory is part of it, his approach better reflects the reality we inhabit. However, it is not possible to describe "the human world in all its complexity and mess” using traditional Newtonian methods, such as drawing a blueprint and building the model it depicts.
Hence, Wittgenstein’s puzzling recommendation on how to engage with his book: "My work consists of two parts: the one presented here plus all that I have not written, and it is precisely this second part that is the important one.” His statement pulls us away from the constraints of deterministic reasoning and into the realm of Jaynes’ Common Sense logic. Jaynes is recognized as one of the first to realize that probability theory, as developed by Laplace, serves as a generalization of Aristotelian logic. Wittgenstein’s insight into the role of probability in enabling us to communicate meaning through propositions—by generating and interpreting them—is a precursor to Jaynes’ plausible logic, which reduces to deductive logic in cases where our individual subjective hypotheses are either true or false.
However, Wittgenstein’s vision was ahead of its time. "It is VERY hard not to be understood by a single soul," he wrote, while also acknowledging his debt for "the stimulation of his thoughts” to those who could not comprehend him—Frege and Russell. Furthermore, in a remark recognizing Boltzmann’s influence, he somewhat introspectively continues: "It is typical for a Jewish mind to understand someone else's work better than he understands it himself." While rooted in the works of his predecessors, Wittgenstein's theory of what can be expressed by propositions extends beyond their scope. This aligns with the progress in AI; early AI was deterministic and rule-based, whereas modern AI relies entirely on probability theory and statistics, providing tools to deepen Wittgenstein’s studies.
7. Large Language Models Demystify the Tractatus’ Infamous Final Remark
Modern AI provides practical solutions to the problem of inductive reasoning. Its computational toolbox employs algorithms for the optimal processing of incomplete information that are used to build mathematical models of different parts of the world. For instance, generative AI applications, powered by Large Language Models (LLMs), function as agents able to communicate with us in natural language and to perform complex reasoning tasks. However, LLMs’ design is plagued by two intractable problems. They do not always act as they ought to act. Their behavior is uncertain. Furthermore, while we know how to train them—teach them how we expect them to act—we still do not fully comprehend the LLMs inner workings. They remain opaque.
Nevertheless, modern AI applications have proven to be exceedingly useful. Their commercial success results from a technique known as machine learning, which employs a 250-year-old computational rule formulated by Thomas Bayes (1702 - 1761) that enables learning from statistical data. In brief, a computer program digests information repeatedly presented to it in the form of statistical samples reflecting reality, and updates itself by transforming its internal structure. Over time, this intelligent agent becomes better at carrying out the task it is assigned to do. Its self-improvement occurs gradually, through trial and error—one step at a time—with these steps being repeated over and over, guided by another century-old innovation: Boltzmann’s formula for calculating entropy as a measure of uncertainty.
Discussing how much of the world is possible to model, Rockmore likens a LLM to “a vast data structure akin to a tangle of Christmas lights whose on-off patterns attempt to capture a chunk of historical word usage. [my emphasis]” In this respect, it resembles the mind. LLM’s lack of explicability aligns with Jaynes’ Robot’s design and reflects “the deepest commitment of [Wittgenstein’s] first masterwork”, as pointed out by Hacker: namely, “the distinction between what can be said, and what can only be shown.”
Moreover, a physics-informed interpretation of the Tractatus’ famous final remark —“Whereof one cannot speak, thereof one must be silent”—is substantiated by Jaynes, whose statement on silence stresses the importance of Boltzmann’s discovery: “At first glance it seems idle and trivial that we should have to do all this in order to learn HOW TO SAY NOTHING [my emphasis]”. And he continues:
The important point, however, is that we have here found a consistent way of saying nothing in a new language: the language of probability theory. The triviality fades away entirely when we notice that the problem of inferring the macroscopic properties of matter from the laws of atomic physics is exactly of the type we are considering [Common Sense plausible reasoning]. All of thermodynamics, including the prediction of every experimentally reproducible feature of irreversible processes, is contained in the above solution [probability theory]. (emphasis in the original)
Notably, the communication of meaning by means of a proposition is an irreversible process. What is said cannot be unsaid. Jaynes’s new ‘language: the language of probability theory’, which was anticipated in the Tractatus, is mainstream today.
8. Conclusion
The paper introduces a new interpretational framework for the Tractatus focused on Wittgenstein's Propositional Theory. It argues that the book builds a philosophical model of the system of human language communication that is dynamic, distributed and probabilistic with an intractable normative core evidenced by the achievements of modern AI. The proposed approach is informed by physics, originates with Boltzmann's ME Principle, and culminates with Jaynes' concept of probability as an extended form of logic.
However, this framework is just the beginning; further research is needed to fully develop a comprehensive, cohesive, and agreed-upon interpretation of Wittgenstein's seminal work. AI’s
toolbox of techniques can assist us in comprehending the book. Thus, Wittgenstein prescient simile, “[A] key can lie forever in the place where the locksmith left it, and never be used to open the lock the master forged for it”, need no longer apply to the Tractatus. Moreover, this endeavor could potentially contribute to a unified and consistent analysis of Wittgenstein’s philosophical investigations, resolving long-standing interpretative debates.
References
Cox, R. 1946. “Probability, Frequency and Reasonable Expectation.” American Journal of Physics 14,1. https://doi.org/10.1119/1.1990764
de Finetti, B. 1937. “Foresight: Its Logical Laws, Its Subjective Sources.” https://www.stat.unm.edu/~ronald/courses/Int_Bayes/definetti_exchangeability.pdf
Hacker, P. M. S. 2017. “Metaphysics: From Ineffability to Normativity.” In A Companion to Wittgenstein. Edited by H.-J. Glock and J. Hyman. Oxford: Wiley-Blackwell.
Jaynes, E. T. 1957. “How Does the Brain do Plausible Reasoning?” In Maximum-Entropy and Bayesian Methods in Science and Engineering. Edited by G. J. Erickson and C. R. Smith, 1–24. Fundamental Theories of Physics, vol. 31–32. Dordrecht: Springer. https://bayes.wustl.edu/etj/articles/brain.pdf
——— 1984. “The Evolution of Carnot’s Principle.” In EMBO Workshop on Maximum-Entropy Methods, Orsay, France, April 24-28. https://bayes.wustl.edu/etj/articles/ccarnot.pdf
——— 1994. “Probability Theory as Logic.” https://bayes.wustl.edu/etj/articles/prob.as.logic.pdf
——— 2003. Probability Theory: The Logic of science. Cambridge: Cambridge University Press.
Perloff, M. 2022. Private Notebooks 1914-1916 Ludwig Wittgenstein. New York: Liveright Publishing Corporation.
Popper, K. 1935. The Logic of Scientific Discovery. New York: Routledge Classics, 2002.
Ramsey, F. 1926. “Truth and Probability”. In Ramsey Philosophical Papers. Edited by D.H. Mellor. Cambridge: Cambridge University Press, 1994.
Rockmore, D. 2024. “How Much of the World is it possible to model?” The New Yorker. January 15. https://www.newyorker.com/culture/annals-of-inquiry/how-much-of-the-world-is-it-possible-to-model
Shanon, C. 1948. “A Mathematical Theory of Communication.” The Bell System Technical Journal, Vol. 27, pp. 379-423, July, October 1948. https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf
Titelbaum, M. 2004. "Normative Modeling." Forthcoming in Methods in Analytic Philosophy: A Contemporary Reader. Edited by J. Horvath, S. Koch, and M. Titelbaum. Forthcoming. https://drive.google.com/file/d/1x5Nxyyrii2w290Yea_-wse4DKmwDVqMj/view
Williamson, T. 2017. “Model-building in philosophy.” In Philosophy’s Future: The Problem of Philosophical Progress, pp. 106–22. Edited by R. Blackford and D. Broderick. Oxford: Wiley Blackwell. https://media.philosophy.ox.ac.uk/assets/pdf_file/0008/38393/Blackford.pdf
Wittgenstein, Ludwig. Culture and Value. (C&V).
Wittgenstein, Ludwig. Tractatus Logico-Philosophicus. Translated by C.K. Ogden. Santiago, Chile: Seven Treasures Publications, 2014, (TLP).