The Missing Link:
How Boltzmann Shaped Wittgenstein’s Tractatus
- Wittgenstein’s World as a Simile Inspired by Boltzmann
Ludwig Wittgenstein is one of the most influential philosophers of the 20th century, known for his work on language, logic, and the nature of reality. However, the Tractatus-Logico-Philosophicus (TLP), the only book he published during his lifetime, is notoriously difficult to comprehend.
In a remark made in 1931, Wittgenstein acknowledged Ludwig Boltzmann, a 19th-century German physics professor, as having the greatest influence on his thinking. Boltzmann’s research in thermodynamics led to an empirical discovery that was highly effective in practice but lacked theoretical explanation. The mystery of ‘Boltzmann’s entropy law’, as Wittgenstein referred to it, took a century to be resolved.
Today, this law is recognized as a fundamental principle in theoretical physics, with a wider area of application than that of Newton’s laws. It is integral to statistical and quantum mechanics, information theory, and machine learning. Without understanding the impact of Boltzmann’s breakthrough ideas on Wittgenstein’s thinking, the Tractatus remains an insurmountable challenge.
It is intriguing that in the same 1931 remark Wittgenstein admitted, ‘I don’t believe I have ever invented a line of thinking. I have always taken one over from someone else. I have simply straightaway seized on it with enthusiasm for my work of clarification. [...] What I invent are new similes.’ Wittgenstein’s unique contribution to philosophy lies in his ability to create striking new similes and metaphors. Through his creative analogies, he is able to shed light on complex concepts.
My objective is to enhance the comprehension of the Tractatus by illustrating how Boltzmann’s statistical thermodynamics principles are deeply embedded within it. I shall examine the relationship between Wittgenstein’s notion of the world as a complex philosophical abstraction, and the simile-model he devised, influenced by his engineering background and Boltzmann’s groundbreaking ideas. It is intriguing that Wittgenstein’s fictional model, designed to elucidate the logic of our language, has exhibited remarkable technological ingenuity and has garnered validation from modern artificial intelligence (AI).
How logic and language reflect reality is evident in our uniquely human activity: language communication. Every language consists of vocabulary and grammar, which refers to words and the rules for combining them into sentences. In addition, every language user has an inherent ability to interpret and generate propositions that others can understand. This ability is subjective, varies from person to person, and remains concealed. Vocabulary and grammar, on the other hand, are established, publicly available resources that we effortlessly employ to convey meaning. Interestingly, we use them spontaneously, as if by habit, much like we breathe in and out the air that we share.
As we shall see, Wittgenstein adopted Boltzmann’s groundbreaking notion that the connection between the hidden and observable sides of the world is probabilistic. In the Tractatus, this is the link between a concept introduced by Wittgenstein: the logical space, which is concealed and distributed, part of it in each one of us, and our shared empirical reality, also called spacetime in physics. What is unusual about the inner–outer connection is that it is not always reliable. Sometimes it fails, and it is impossible to predict when this will happen. The crossing of Boltzmann’s statistical bridge is irreversible and non-deterministic. Its most awkward aspect is that moving in one direction is different from moving in the opposite. On it, we cannot retrace our steps, metaphorically speaking.
Newton’s deterministic laws that form the foundation of classical mechanics work in both directions of time. However, whereas Newton’s bridge crosses a span of time, Boltzmann’s statistical bridge presents an unfamiliar scenario whereby movement is not forward and back, but outward and inward. In language communication, the acts of ‘speaking’ and ‘listening’ replace the familiar notions of ‘walking forth’ and ‘returning back’ that we associate with the crossing of a bridge in everyday life. That is because language communication crosses the barrier between the mental and the physical.
Apprehending and explaining Boltzmann’s discovery can be challenging. However, Wittgenstein effectively addresses this challenge by creating an immaculately constructed text that captures the essence of both speaking and listening. To clarify his approach, I shall delve into the enigma of Boltzmann’s entropy law with a focus on its application in the field of language communication.
- An Early Definition of Entropy Reverberates in the Motto
Boltzmann’s legacy is encapsulated in a formula so elegant and concise that it is engraved on his tombstone. However, its underlying idea was so much at odds with the established way of thinking that Boltzmann, disheartened by his detractors’ attacks, committed suicide in 1903, coincidentally the same year in which Wittgenstein intended to begin studying under him.
In keeping with his family’s business tradition, Wittgenstein pursued a technical education to become a steam engineer. His father, a prominent steel magnate, had amassed his fortune during the Industrial Revolution, a period marked by the widespread use of steam engines. Wittgenstein acquired a comprehensive understanding of the principles of thermodynamics (the theory of heat) and even patented the design of an advanced turbo-propeller. However, although Boltzmann’s entropy formula was applied in the field and demonstrated to be of great practical value, its underlying rationale remained elusive.
Under the spell of entropy’s unresolved enigma and lured by a recently discovered, seemingly insurmountable contradiction between logic and language known as Russell’s paradox, Wittgenstein abandoned his career as an aeronautical engineer and embarked on a new path as a philosopher. In the summer of 1911, he drafted a plan for a philosophy book that eventually became the Tractatus. It seeks a formal solution to Russell’s paradox, the infamous ‘barber’s mystery’: in a small village, the barber shaves every man who does not shave himself. Who then shaves the barber? Wittgenstein was determined to uncover the answer to a perplexing question that appears valid but defies common sense. Philosophical inquiries often exhibit this intriguing nature.
In the preface, he states that ‘The book deals with the problems of philosophy and shows that the method of formulating these problems’ is incorrect because ‘it rests on a misunderstanding of the logic of our language’. Wittgenstein’s objective is to find a better way to comprehend logic. To achieve this, he aims ‘to draw a limit’ ‘to the expression of thoughts’ in language. Boltzmann’s entropy law enables him to explore both sides of this limit.
To understand Wittgenstein’s solution, we must first address an even more perplexing question: ‘What is entropy?’ The first attempt to define entropy was made by Sadi Carnot, the forefather of steam engineers, who created a model of an ideal steam engine in 1824. Although Carnot’s model was purely theoretical and could not be constructed, it revolutionised the industry by providing engineers with a practical thinking tool to create efficient machines that could convert hidden heat energy into visible and useful work.
‘In this case useful work might mean accelerating a locomotive, which would require converting heat energy into kinetic energy of a large mass of iron’, writes theoretical physicist Leonard Susskind in his book The Black Hole Wars (p. 167):
Heat energy means the disorganized chaotic energy of random molecular motion. By contrast, the kinetic energy of a locomotive is organized into the simultaneous synchronized motion of a huge number of molecules, all moving together. So the problem was how to turn a given amount of disorganized energy into organized energy. The problem was that no one really understood exactly what organized and disorganized energy meant. Carnot was the first to define entropy qualitatively as a measure of disorganization.
Half a century later, Boltzmann made a fundamental breakthrough in theoretical physics by discovering a way to quantify this measure using probability. In the field of thermodynamics, his formula establishes a link between the hidden microstate of a gas in equilibrium with its surroundings and its observable macrostate, whose characteristics (volume, temperature, pressure) can be measured. Essentially, Boltzmann’s formula bridges the gap between the intangible inner and the tangible outer by aligning their statistical properties. Recognising the vast potential of Boltzmann’s entropy law, Wittgenstein expanded its application to the domain of logic and language.
The transformation of disorder into order is conveyed in the motto he chose for his book: ‘...and everything you know, not just heard rustling and roaring, can be expressed in three words.’. These three words are no longer disordered thoughts that rustle and roar in someone’s head, confined to that individual’s logical space. A thought that is spoken out loud is expressed in language and thus becomes organised into a sequence of words ordered by the rules of grammar. Although it may seem counterintuitive, Boltzmann’s entropy law is reflected in the restrictions imposed by grammar, which limit the number of possible combinations of words that we use to convey meaning.
The nature of language communication is inherently statistical because human interaction is not always precise. The communication channel originates within the logical space of the speaker or writer (the source), traverses empirical reality, and ends in the logical space of the listener or reader (the receiver). Furthermore, the exchange of meaning through language between speakers and listeners occurs countless times in our everyday lives. Over time, these repetitions give rise to statistical patterns that shape our communication rules and establish the foundation of common-sense logic. Today, this logic is referred to as the logic of uncertainty or statistical logic. Like Carnot’s heat energy, it is the hidden organising force that sets Wittgenstein’s model in motion to produce a sequence of grammatically ordered words.
Intriguingly, the three words of the motto are not disclosed, symbolising an empty structure of something that is said. What follows is Wittgenstein’s explanation of the process that fills that structure. This is the process that leads to the articulation of thoughts. The Tractatus presents a captivating account of how a thought comes to be expressed in language. It is a fictional tale inspired by Boltzmann’s mathematical truth. It is the proposition’s origin story narrated by a brilliant engineer. Surprisingly, it has been validated by the success of modern AI.
- The Tractatus’ Protagonist and Backstory
It has been suggested that Wittgenstein used the German word ‘Satz’ as a working title for his manuscript, as if he were naming his book after its principal character: the proposition. ‘Satz’ can be translated in various ways, including ‘proposition,’ ‘sentence’, ‘statement’, ‘assertion’, ‘utterance’, ‘remark’, and ‘what was said’. Its ambiguity reflects the elusive nature of Wittgenstein’s hero, who serves as a vehicle (and an unreliable one at that) for the transportation of meaning between speakers and listeners. When children play live-telephone, the message of the first child often deteriorates to the extent that the last one may not understand what was said.
It is important to note that Satz undergoes two marked transformations when crossing Boltzmann’s statistical bridge. Its journey begins as a thought in the speaker’s logical space. The articulation of that thought creates an event in spacetime. This event is the generation of an ordered sequence of words in the form of a speech audio signal or text, which Wittgenstein refers to as a ‘perceptional sign’ as it is perceived by the senses. Speaking is the act of generating the proposition’s perceptual sign. It corresponds to the outward crossing from the mental to the physical.
The interpretation of the proposition by one or multiple listeners within their respective logical spaces corresponds to the inward crossing from the physical back to the mental. Wittgenstein refers to this act as understanding, as it transforms the proposition from something said to something that has been understood. However, the regained meaning of the proposition might be slightly different, as its interpretation is performed by a different person – the listener. In today’s context, the listener could be a non-human linguistic agent, such as a generative AI chatbot. In modern terms, we would say that information is lost, and although we cannot measure sense or meaning, we can quantify the loss of information.
The twofold transformation of Satz is a result of the generation and interpretation performed on it by the speaker and the listener. However, Satz is not passive; it has an effect on those who use it. It teaches the speaker how to imbue it with meaning, or sense, and the listener how to extract the meaning it carries (i.e. to understand its sense). The all-encompassing Boltzmann entropy law reins over Satz’s metamorphoses by aligning the statistical properties of the inner and outer twice, on both ends of the statistical bridge. ‘The silent adjustments to understand colloquial language are enormously complicated’, Wittgenstein wrote (TLP 4.002). However, today we know how to technically replicate such adjustments to train the large language models that animate generative AI chatbots.
Typically, a human can both speak and listen, but notably not at the same time. A speaker–listener agent learns how to generate and interpret propositions one step at a time, through countless, imperceptible alignments. These alignments occur externally in the communication channel between the speaker (source) and the listener (receiver), as well as internally in the logical space of the linguistic agent. An AI agent in training builds its neural network, or logical scaffolding in Tractarian simile speak, by repeated internal alignments of its inputs with the outputs it generates. It learns to understand others and to make itself understood. Technically speaking, it learns how to maintain coherence and demonstrate common sense from the statistical patterns in the data presented to it. Wittgenstein’s definition of form as a possibility of structure refers to these patterns prior to their detection.
Today, we understand how these two alignments function as a result of the groundbreaking work of two American scholars: mathematician Claude Shannon and theoretical physicist E.T. Jaynes. Shannon’s research focused on the external alignment between the source and receiver, while Jaynes demonstrates how an internal alignment allows an agent to cultivate and exhibit intelligence. It is remarkable that Wittgenstein was able to herald the discoveries made by Shannon and Jaynes by having a deep understanding of the principles of Boltzmann’s statistical thermodynamics.
Speaking can be compared to encoding and transmitting a message, while listening is akin to receiving and decoding that message. In 1948, Shannon provided a precise mathematical explanation of the communication process between source and receiver. He devised a method to quantify the amount of information transmitted and lost in this process. Originally, Shannon intended to call his formula a measure of uncertainty or surprise. However, owing to its mathematical resemblance to Boltzmann’s formula, it came to be known as ‘information entropy’. Today, the concept is a cornerstone of information theory and computer science.
Another decade passed before Jaynes established the theoretical connection between Boltzmann’s and Shannon’s concepts of entropy. He demonstrated that language communication is an irreversible and non-deterministic process governed by Boltzmann’s entropy law. Communication follows the arrow of time. Once something is communicated, it cannot be un-communicated. Jaynes’ work made him a pioneer in the field of modern AI, which emerged in the 1980s and overtook the rule-based, deterministic AI systems with the statistical approach that underpins neural networks.
Jaynes’ contribution is a mathematical construct known as Jaynes’ robot. It is an ideal agent designed to reason and behave sensibly in conditions of incomplete information, similarly to how we operate in our everyday lives. It learns how to cope with uncertainty. Jaynes was instrumental in the modern interpretation of probability as common-sense reasoning learned from statistical data. His ideal agent, like Carnot’s ideal steam engine, played the role of a practical thinking tool abetting a revolutionary industry. Wittgenstein’s simile-model is their close kin.
- The Tractatus’ Symmetrical Structure and Summary in Six Statements
In an information transmission system, the algorithms of the encoder and decoder modules are intrinsically connected. Similarly, the two acts of generation and interpretation of speech rely on each other. Their mutual dependence can be detected in the meticulous composition of the Tractatus, whose form is integral to its content. The text exhibits a notable symmetrical structure, an outward and inward direction, and a striking conceptual correspondence between key notions such as atomic facts and elementary propositions, facts and propositions, possibility and probability, and others.
The organisation of the Tractatus is an impressive engineering feat in itself. The text follows a hierarchical system of numbered statements and subordinate branches, with the most general and logically significant remarks at the top. Wittgenstein acknowledged that without this numbering scheme, the Tractatus would be an ‘incomprehensible jumble’. This arrangement allows the top layer consisting of six remarks and a coda to outline the book’s main idea. Moreover, each key remark serves as a summary heading for the section beneath it, where the topic is further elaborated.
Like the famous duck/rabbit picture, the scheme, shown below (my emphasis), can be seen in two ways: as Shannon’s information transmission channel connecting externally through spacetime the speaker’s and listener’s sides of two different linguistic agents and, conversely, as Jaynes’ ideal agent whose speaker’s and listener’s sides are connected internally through its logical space. Boltzmann’s entropy formula measures both the loss of information in the communication channel and the ignorance of the agent.
1. The world is everything that is the case.
2. What is the case – a fact – is the existence of atomic facts.
3. A logical picture of facts is a thought.
………………………………………….
3.5. The applied, thought out, propositional sign is a thought.
———————————— out ————————————
Boltzmann Statistical Bridge
———————————— in ————————————
4. A thought is a proposition with a sense.
5. A proposition is a truth-function of elementary propositions.
(An elementary proposition is a truth-function of itself.)
5.1 The truth-functions can be ordered in series.
That is the foundation of the theory of probability.
.………………………………………….
6. The general form of a truth-function is: [ , , N( )].
This is the general form of a proposition.
7. What we cannot speak about, we must pass over in silence.
Wittgenstein’s masterwork consists of two conceptually aligned parts that straddle the mental–physical divide. The first three sections, headed by the key remarks TLP 1, 2, and 3, explore how propositions are formed in the agent’s logical space. The remaining three sections, headed by remarks TLP 4, 5, and 6, focus on the formal analysis of these propositions. What makes Wittgenstein’s analysis so prescient is the introduction of probability theory in section 5. His approach is a significant advancement beyond the methods of formal logic employed by his contemporaries, who had encountered a roadblock with Russell’s paradox.
Section 5 is conceptually connected to section 2, in which Wittgenstein develops his famous picture theory of language. His probability theory and picture theory are like the two sides of the same coin. They are on the opposite sides of Boltzmann’s statistical bridge working in tandem, but in different directions, just as the encoder and decoder modules of a communication system for conveying messages perform two opposing operations. Understanding Wittgenstein’s masterpiece depends on grasping how the two theories are, in fact, one.
- The Elusive Entropy Principle — Echoed in the Coda
The journey of Satz begins with a puzzling definition (TLP 1) and ends with an intriguing formula, followed by an assertion: this is how a proposition is formed (TLP 6). TLP 1 sets the stage by sketching the background assumptions of probability theory, and TLP 6 delivers the result that the motto leads us to expect.
The remarks in between, TLP 2, 3, 4, and 5, outline the stages in the thought-proposition transition. The twofold definition of thought is a key hinge point in the crossing of Boltzmann’s statistical bridge. TLP 3 provides the definition from within the logical space of the speaker: ‘A logical picture of facts is a thought.’ That thought is articulated in (TLP 3.5). Encoded as a string of words, it crosses the empirical reality and is communicated to the listener. ‘A thought is a proposition with a sense’, (TLP 4) starts its formal analysis and interpretation on the other side.
In addition, the succinct formula in TLP 6 brilliantly sums everything up. It expands on Shannon’s concept of information entropy, encapsulating a universal approach to constructing a proposition based on the information it carries. Wittgenstein’s formula provides a framework for finding sensible answers to ‘who shaves the barber’ types of questions. Interestingly, the question has multiple common-sense answers – ‘The barber is a woman’, or ‘The barber has a beard’, or ‘The barber lives in the village nearby’ – which highlights the enigmatic nature of this new type of common-sense logic that is not precise and formal but ‘take[s] care of itself’ (TLP 5.437): ‘Our fundamental principle is that every question which can be decided at all by logic can be decided without more ado.’ (TLP 5.551)
The last concluding remark, TLP 7, appears to be trivial, almost a joke; however, without it, the proposition’s journey is incomplete. It is precisely this seemingly insignificant remark that reveals the profound impact of Boltzmann’s pioneering use of probability on Wittgenstein’s thinking.
Intriguingly, the TLP 7 insight is endorsed in Jaynes’ 1957 seminal paper ‘How does the Brain do Plausible Reasoning’:
At first glance it seems idle and trivial that we should have to do all this in order to learn how to say nothing. The important point, however, is that we have here found a consistent way of saying nothing in a new language: the language of probability theory. The triviality fades away entirely when we notice that the problem of inferring the macroscopic properties of matter [measured in empirical reality] from the laws of atomic physics [the matter’s hidden states] is exactly of the type we are considering [reasoning with propositions]. All of thermodynamics, including the prediction of every experimentally reproducible feature of irreversible processes, is contained in the above solution [i.e., the probability theory, notably the communication of meaning by a proposition, is an irreversible process].
Jaynes’s new ‘language: the language of probability theory’, which was anticipated in the Tractatus, is mainstream today.
- Conclusion
‘If someone is merely ahead of their time, it will catch up with them someday’, wrote Wittgenstein.
This article substantiates Wittgenstein’s premonition by offering a new framework for interpreting the Tractatus, with probability theory at its centre. Wittgenstein’s introduction of probability into the analysis of propositions was ahead of its time, stemming from Boltzmann’s breakthrough: the measure of entropy. Wittgenstein applied Boltzmann’s fundamental law of physics to a new domain – that of logic and language – which, in itself, is a trailblazing achievement.
The difficulty in understanding the Tractatus arises from the fact that the theoretical implications of Boltzmann’s entropy law were not fully understood until the early 21st century. These implications are evident in the achievements of modern AI, whose algorithmic toolbox, in turn, can assist us in setting Wittgenstein’s ‘carriage straight on the rails’. Thus, his prescient simile, ‘[A] key can lie forever in the place where the locksmith left it, and never be used to open the lock the master forged for it’, need no longer apply to the Tractatus.
Elizabeth Rohwer, PhD.
San Diego
erohwer@san.rr.com