Mathematics,  Philosophy,  Physics

Quantum Theory and Sāńkhya for Beginners

Many people have expressed interest in understanding quantum theory—both in terms of the underlying scientific problems, as well as what solutions could look like. While I have provided technical explanations in the past, they often prove inadequate for those who may not follow the technicalities. This post, rather ambitiously hopes to cover that shortfall. It covers the basic mathematical ideas underlying quantum theory, the nature of the quantum problem, what its semantic solution looks like, and the type of mathematics needed to formulate a formal scientific solution. For the sake of simplicity, I will cover the non-relativistic version of the theory, and then describe the unification problem. Wherever necessary, I have used examples to demystify the problem or explain the solution. These are helpful tools, not replacements for potential scientific rigor.

A Simplified Mathematical Background

Atomic theory describes light using the equation E = ħν where ν is the frequency and ħ is Planck’s constant. The problem is that the photons are often treated as independent particles, quite like classical particles, when they should be treated as particles in an ensemble. But, what is this thing called ensemble?

This is where a little bit of mathematics—called a Fourier Transform—is helpful to grasp the basic idea regarding what we mean by a photon and how it exists as part of an ensemble or a macroscopic object. Don’t worry, I will not use any equations—just a few pictures to explain the basic ideas.

The essential idea underlying a Fourier Transform is that we can express any arbitrary function as the sum of periodic functions. The periodic functions interfere constructively and destructively, and this interference constitutes their sum. As shown in the figure below, the red line expresses an arbitrary function, and the blue lines express its component periodic functions.

Now, in classical physics, we treated the red line (i.e. the macroscopic phenomenon) as the only reality and the blue lines (i.e. the constituents of this phenomenon) as only convenient mathematical fictions. But it is also possible to invert the thinking and consider the blue lines as the real physical entities and the red line as the “emergent” phenomena created as the consequence of adding the blue lines. The addition of blue lines to create the red line is called the superposition of the blue lines.

A Gentle Introduction to Quantum Theory

Quantum theory is the idea that the blue lines are real, and the red line is an emergent phenomenon because we are able to individually detect the blue lines in a measurement. Quantum theory is also based on the idea that the blue lines are not infinitely extended like the waves shown above. Rather, these lines are truncated, thereby creating small “packets” of energy.

Why do we need the packet? The reason is that we are able to discretely receive the full blue line in a very small time period rather than the continuous wave that is received continuously over an infinite time period. Thus, the quantum packet is called a particle. And owing to the fact that this packet is derived by componentizing the red line into many small waves, the quantum is also called a wave. This is then the origin of the wave-particle anachronism—it is a convenient way to describe two distinct ideas: one that the quantum is received discretely and two that it can be mathematically described as a wave component.

We can put this even more succinctly: observationally, the quantum is a particle, but theoretically it is a wave. If you notice the difference between observation and theory, you will quickly realize what I’m saying here—namely, that the quantum is a particle for the senses, but it is a wave for the mind. During a measurement we observe a particle, but we describe this observation conceptually as a wave. The wave-particle duality is a problem of how an object is both in the senses and in the mind, although as two different kinds of entities. If you have seen my earlier posts, I address this issue by calling the quantum a symbol of meaning: the symbol is sensual but the meaning is mental.

The Conceptual Difficulty in Quantum Theory

It is notable that the individual waves are obtained by dividing the macroscopic object into components. The macroscopic object, therefore, constitutes an ensemble. And yet, because we are able to detect the components individually, we are tempted to falsely think that these components are independent.

This false thinking is reinforced by the fact that these components are mathematically orthogonal—i.e. if we multiply any two of them together we will obtain a zero. Effectively, these components are like the dimensions of a multidimensional space in which the axes are mutually orthogonal (you won’t find any component of one axis in another axis). Since they are orthogonal, it is tempting to think that they are also mutually independent. But a variety of problems arise when we do so because if you happen to change only one component, then you must change all the components at once.

The components are therefore mutually orthogonal but not mutually independent. Their orthogonality represents the idea that they are individual entities. And yet, because each component is defined mutually and collectively with all the other components, they are somehow entangled with each other. This entanglement has proven very hard for physicists to visualize materially because we are used to thinking from classical physics that all material particles are independent of each other.

And yet, this kind of idea is not very hard to understand if you think of these waves as representing ordinary concepts such as color, which has to be decoded into orthogonal colors such as red, green, and blue. If you change one of these colors—e.g. red to magenta—then the other two colors will become cyan and yellow. The components of the color space are orthogonal axes, but these axes can only be rotated as a whole, not individually. If you change one of them, then you change all of them.

The Nature of Quantum Reality

The central problem of quantum theory is that we are used to thinking of material particles as objects rather than dimensions of space. If we were dealing with space, rather than matter, then this problem would not seem so hard. But due to classical physics, we think that there are only three dimensions in space, and there are infinite objects in this space. If we try to think of quantum objects as dimensions of space, then we end up with infinite dimensions in space, which seems to be physically meaningless.

Can we convert what we currently consider physically meaningless into physically meaningful? The first step in the resolution of this problem is to start thinking of matter as space, and this is where the idea of hierarchical space, which is structured like a tree becomes very important. Each branch of the tree is a dimension, which means that space (if structured like a tree) can have infinite dimensions. Furthermore, if these branches are now described as particles, then each particle is both a dimension and an object. The tree as a whole can exist in three dimensions, and yet it can contain infinite dimensions, each of which is now identified as a physical particle. This resolves the basic conundrum where the quantum entanglement is demystified by treating each particle as a dimension. 

The crux of the solution is that there isn’t one flat space, and all the points in space don’t exist a priori. Rather, space grows a like a tree, branching out subspaces, which then create further subspaces, etc. Each such subspace has its own dimensions, and if we put together all these spaces, then there are as many dimensions as there are objects.

Effectively, each dimension is an object, and all objects are dimensions. Traditionally, we have always used dimensions to denote properties and concepts in science. For example, we denote the property called “momentum” as a kind of dimension in the phase space. We can use this idea to say that the objects that are dimensions are properties or concepts. The world that has so far appeared physical to us, suddenly becomes conceptual because the space has infinite dimensions. There is no difference between matter and space; matter does not exist inside space. Rather, matter itself is space. Therefore, we don’t need two theories—one for matter and another for space; we need one theory that describes space, and this space is structured like a tree with infinite branches, rather than a box.

This idea is not new; material phenomena have been described using fractals and holograms which replicate the superstructure into each microstructure. However, in these descriptions, we distinguish between matter and space; the material phenomena have a fractal structure, but the space is still linear and flat. The difference is that we are now talking about all of space itself as being a tree. In this space, you cannot walk between any two points directly. You have to traverse up the tree branches until you find the branch that leads to the destination. This kind of space has a completely different definition of distance—not as a straight line, but as the hierarchical tree structure. You think two physically nearby nodes are close? You could be very wrong in hierarchical space.

Describing the Quantum Measurements

The oft-stated problem of quantum theory is that we are unable to predict the order of quantum detection. The quanta don’t arrive in a linear order, which is not difficult to understand if we try to project the tree onto a flat surface, and count the nodes in the order in which the leaves grew on the tree—i.e. how the original tree was constructed through state preparation. Clearly, the branch must exist before the leaves, and the trunks must exist before the branches. So, there is a natural order in the tree based on how it is constructed, but we miss this order if we think the space is flat. If space were hierarchical, then order in detection would simply a projection of a tree into a flat surface.

Measurements invert state preparation; during a measurement, the process of construction is unraveled, or the steps are retraced in the reverse order. If you have built a house, you lay the foundation first, then build the walls, lay the roof, and finally paint everything. But if you want to unravel the house, then you start by chipping the paint, and reach the foundation at the end. In other words, while building the house, you start from the root of the tree and reach the leaves, and during measurement you start from the leaves and reach the root. The construction is not random if we see the entire tree.

The solution to the problem of quantum measurements is that we have to describe the components of the macroscopic world not as existing in a flat space, but as leaves, twigs, branches, and trunks of a tree. In other words, all the quanta that we see are not of the same type. Some of these quanta are “higher” nodes in the tree, while others are “lower”. Some of the quanta are branches, while others are leaves. Their order corresponds to this hierarchy. To enable this idea mathematically, some changes are needed.

Is Quantum Reality Truly Linear?

The blue lines can be added in any order to create the red line, and it doesn’t matter if we do A + B, or B + A. This idea is called linearity—i.e. the whole is the sum of its parts. There is a sense in which this idea is true, and another sense in which it is false.

The linearity is true for a given level of the tree, but false across levels. As a crude example, all the people in a team at the same level add up linearly. But the manager of the team doesn’t add up linearly with the team, because the manager is always “higher” than the team. Linearity in this case means that the hierarchy in the organizational structure would be flattened, and there would be no managers and no employees. Clearly, that idea would violate the tree structure, and make the world flat rather than hierarchical. Present quantum theory is linear and this linearity is an indication of flatness of space.

The linearity of quantum theory is embedded in its mathematics, because the mathematics allows any two functions to be added like numbers and the order of addition doesn’t matter. The arbitrary order of adding components then appears as the randomness in the order in which quanta are detected, because the theory supports all orders—A + B and B + A—equally well. Whether A precedes B or vice versa cannot be predicted and that results in randomness because we cannot predict the order after we have assumed that all orders are equally good. (In fact, the ability to make such predictions would contradict the foundational premise and make the theory self-contradictory.)

Quantum randomness directly results from the quantum linearity formulation. To fix this problem, we have to discard the linearity and add the quantum objects in a specific order. In short, quantum phenomena cannot be described using a linear theory.  

The Semantic Solution to the Quantum Problem

The notion of hierarchical space entails that the quantum objects are not classical particles, which existed independently of each other. We have to rather treat these objects as symbols of “higher” and “lower” meanings. This hierarchy includes a description of reality as well as observer activities in relation to this reality. An example of the former is the conceptual hierarchy in which “mammal” is higher and “dog” is lower. An example of the latter is the sentence structure such as “I saw a man on the hill with a telescope” in which words are attached to the observer at different levels in a hierarchical structure.

Quantum linearity entails that “dog” and “mammal” are just two objects, and since we don’t care about their meaning, we also remain unaware of the hierarchy between the symbols that signify them. Therefore, we should be able to add “dog” to “mammal” or vice versa and the inversion of order should not matter. With such a view, we lose the ability to understand the order, which results in probabilities. The hierarchical proposal changes the mathematical formulation of the quantum phenomena and suggests that “dog” can be added to “mammal” but not vice versa. This hierarchical order employed during state preparation then accurately predicts the order during observation.

Trying to decipher the meanings from the words is an unsolvable problem because the same words can have many meanings. But meanings can be deterministically converted into words. These meanings exist in the words and the grammatical structure.

What are Matter and Force?

In current science, we describe the words as “material particles” and the grammar structure as “force particles”. For instance, the material “objects” are held in their location by “forces” of nature. In the semantic approach, the “objects” are symbols, and they are connected by a grammar to create sentences. The “forces” of nature are therefore the means by which we take “atomic” meanings, and construct “complex” meanings. We can view this analogously to sentences created from words. Forces are now the analogues of grammar in language that creates semantic complexity from simplicity.

What is grammar? By grammar I don’t mean the sense in which grammar is taught in schools—e.g. as the parts of speech, rules of style, punctuation, etc. Rather, I mean grammar in a very technical sense as used in computational linguistics where it is called a parse tree or a syntax tree. The idea is pretty simple: each sentence has a tree-like structure, and there are rules for forming such trees which determine whether the sentence is well-formed in order to decide if they are meaningful. This grammar—as we have seen in an earlier post—changes the forms of words to create various kinds of cases (e.g. Locative Case, Possessive Case, Instrumental Case, Nominative Case, etc.).

The forces of nature in atomic theory, similarly, define the form of a quantum—which is called the wavefunction—and it constitutes everything that needs to be known about the quantum. For example, the electrons in an atom have various kinds of forms which are known as orbitals—called s, p, d, and f. Quantum theory—owing to classical physical history also postulates a variety of physical properties and forces in order to compute these forms, but nobody can say if those properties really exist or if they are real, because what truly matters in quantum theory is to obtain the above said forms.

From these forms, the periodic table of chemical elements is constructed by employing the combinations of different orbitals.

2s            2p2p2p
3s            3p3p3p
4s       3d3d3d3d3d4p4p4p
5s       4d4d4d4d4d5p5p5p

In other words, all the electrons in an atom don’t have the same form. Rather, each electron is a different form, owing to which it is appropriate to think of them as different objects because of their form. It is also interesting that the “force particles” or bosons have no form, because they constitute the “field” in which these forms exist. The field is everywhere, but is not called “space” due to historical reasons where the field and space were separated (these reasons were discarded in Einstein’s relativity).

The physical and the semantic views about the quantum phenomena thus differ in two respects: (1) these shapes are treated as symbols with meanings, and (2) they are arranged hierarchically. The cause underlying this form creation is not a physical “force” driven by material properties but the grammar which combines words. This grammar is the “generative force” (see generative grammars) that produces meaningful sentences, and the world around us is these sentences—which can both be sensed and understood.

Modern atomic theory describes various kinds of force particles or “bosons” and matter particles called “fermions”. Ultimately, the names by which we call them (e.g. quarks, leptons, baryons, gravitons, photons, etc.) are not very important. Only the shapes or wavefunctions are important because these represent all the properties of the quantum. Current quantum theory computes these forms using the Schrodinger’s Equation in which the Hamiltonian incorporates classical physical forces. But a new quantum theory can explain these shapes using grammar that constructs sentences, in which the atomic objects are the elementary symbols of meaning. The important lesson of quantum theory is that any theory that treats the wavefunctions physically will always remain incomplete, but the theory that treats them as symbols can be made complete.

The Use of Complex Numbers

One of the mysterious aspects of quantum physics is that it uses complex numbers (which combine real numbers and imaginary numbers). The imaginary numbers cannot be physically computed. That is, you cannot put an imaginary number into a calculator and expect an answer. And yet, such numbers are employed in quantum theory, which makes the theory very mysterious. How can reality involve complex numbers when we cannot physically conceive an imaginary number (i..e the square root of -1)?

A real number in ordinary three-dimensional space employs three coordinates—e.g. x, y, and z. An imaginary number in the imaginary three-dimensional space also employs three coordinates—these are called i, j, and k. A complex number is one that employs both real and imaginary numbers, which means that it uses six dimensions in a three-dimensional space! This is achieved by calling each of the dimensions by two names: {x, y, z} and {i, j, k}. The real component of a complex number uses {x, y, z} while the imaginary component employs {i, j, k}. Now, you have six logical dimensions and three physical dimensions.

This is rather weird because we treat the same dimension in two ways—real and imaginary—and the reasons for this kind of use have to be understood in the context of atomic theory. I will provide three explanations of this problem here—one simple, a second one more technical, and a third one even more technical. Depending on how much physics and mathematics you know, you can grasp the same idea in different ways.

The simple explanation is that a symbol has two properties—a physical existence, and a meaning—which are not identical, and both of these require three dimensions. Ideally, we need two spaces to describe the two properties, but that is both counterintuitive (how can an object exist in two spaces?) and observationally false (we don’t see two spaces). Thus, we have one three-dimensional space, but we use six dimensions to describe the physical existence, and a meaning—so we are compelled to use complex numbers.

This idea can be easily understood as follows. Imagine a company in which the boss sits on the 3rd floor and the employee on the 4th floor. Physically, the boss is lower than the employee, but semantically the boss is higher. We can’t fit both descriptions in the same space, if the space is three-dimensional. To solve this problem we must either draw two pictures—one in which the boss is lower than the employee, and the other in which the boss is higher than the employee. Alternately, we use complex numbers in which the real part denotes the physical height and the imaginary part denotes the semantic height.

A more technical explanation is that in physical space the object’s location doesn’t represent all its properties. Thus, we think that objects are inside space, and space is a container, while objects are separate from the container. To make this distinction, we need two kinds of numbers—to describe the object’s location and to describe the object’s properties (e.g. mass and charge). We need both of these properties to completely describe the object, and hence three-dimensions don’t suffice.

An even more technical explanation is that classical physics requires two sets of three-dimensions: one for space and another for momentum, and a six-dimensional phase space is needed to describe all the properties of the objects. The momentum of the classical particle now constitutes the additional three dimensions, and this momentum includes the ideas of mass and velocity. Since the momentum is independent of the position (after all, these are separate dimensions), inherent in the phase space description is the separation of space and matter. If, for whatever reason, this distinction between matter and space is dissolved, then we have to fit six dimensions in three dimensions, and we would be compelled to use complex numbers. Quantum theory forces that unification, but why it does is not very apparent unless we adopt a semantic approach.

Why do we need complex numbers? The reason is that there are three dimensions in space and both matter and meaning are parts of the same space, and both matter and meaning require three dimensions, and we cannot equate the semantic to the physical. This problem doesn’t need complex numbers, if we employ a hierarchical space. But if we use flat space, then we are forced to use complex numbers. The craziness of complex numbers—which is having six-dimensions in three-dimensional space—is resolved when space is hierarchical, so we don’t need to worry about complex numbers. Such a description is an artifact of current science, but has no significance in a semantic theory.

The Unification Problem

Modern physics can be broadly classified into three theories: (1) atomic theory which deals with atomic particles, (2) thermodynamics which deals with everyday macroscopic objects, and (3) relativity theory which describes the universe at cosmic scales. These three theories are mutually incompatible. Quantum theory and relativity are incompatible because of the non-locality problem: relativity says that all forces must travel at a finite speed, while quantum theory violates that premise. Quantum theory and thermodynamics are incompatible because quantum theory is linear and thermodynamics is non-linear. The unification problem in physics is how to bring the theories of atomic, macroscopic, and cosmic scales into a single unified theory that applies at all scales of nature.

I have earlier discussed why relativity theory becomes problematic when we view the world semantically: if we turn our laptops upside down, the 1s don’t become 0s because we give a fixed interpretation to quantum spin (up and down) as 1 and 0. If we allow relativistic inversions then we would have to eliminate all meanings. I have also discussed how the constant speed of light can be viewed as the time taken in light absorption rather than the time taken in light travel. Put together, these two premises invalidate relativity as a theory of the world that has meanings too. Since quantum theory cannot be treated as a theory of the physical world alone, relativity too must be discarded, and that is achieved if the speed of light is due to time spent in absorption rather than in travel.

We saw above how the linearity of quantum theory in a flat space is a problem, which means that the theory must be made non-linear. When space is described hierarchically, then we discover a new kind of linearity in which details can be added to or removed from abstractions without impacting the abstractions. In that sense, the abstractions are independent of the details, but the details depend on the abstractions.

In essence, unification requires us to discard relativity, and embed the non-linearity of thermodynamics in quantum theory, making the new theory a unified view of the material universe. The new picture of nature is not “quantum theory”. Rather, it is a theory of atomic, macroscopic, and cosmic phenomena because it brings together the ideas of entanglement, non-linearity, and space. Since we discard relativity, space and time are not “merged” into “space-time”. Matter and space are identical; time, however, is distinct.

The Relationship to Sāńkhya

The material “objects” in Sāńkhya are called “gross” and “subtle” matter: the “higher” objects are “subtle” and the lower objects are “gross”. All these higher and lower objects are connected by a grammatical structure called prāna—which is treated as a “force” as it is involved in changes such as the creation and destruction of the material sentences.

In a sense, we are all suspended in a force field, which is identical to space, and that force field creates material objects, causing objects to “emerge” and “dissolve” in space. All this is very similar to quantum theory. The difference is that matter and force are not physical but rather semantic. The things that we call objects are words and meanings, while the things that we call force constitute the grammatical structure in which some words are “attached” to other words, making them higher and lower. All this arranges itself into an inverted tree-like structure, and therefore the tree is the most succinct description of nature. This tree is not a physical entity; rather, the tree is a different kind of geometrical and mathematical structure in which objects spawn space, and that space then again spawns objects, producing a hierarchy of space (grammar) and objects (words).

Choice and Consciousness

The prāna carries the choices of consciousness, and choice has a very specific technical implication: it is used to order, prioritize or sequence material entities. The hierarchy of the tree is the result of this choice or ordering, but the choice is not free.

The choices are constrained by four factors: (1) kāla or time which defines which of the possibilities can exist at any moment, (2) the material nature or guna which define the past habits that force us to choose certain alternatives, (3) karma or consequences of past actions which constrain which of the choices by limiting the opportunities, and (4) a logic of nature which defines the hierarchy of material categories called Ishvara or God (this is an advanced topic and needs a more detailed discussion, which I will not undertake here). Thus, while prāna carries the choices of consciousness, prāna is constrained by kāla, guna, karma and Ishvara. All of nature therefore is created from these five categories: (1) consciousness represented in matter by prāna, (2) kāla or time, (3) karma or consequences of past actions, (4) guna or past habits, and (5) the logic of nature or God.

A small subset of this description—the tree structure—is sufficient to understand the quantum phenomena, and why it appears as a problem in modern science. The solution of this problem, however, requires the invocation of the five categories mentioned above.