The $H$ theorem

Consider a system of weakly interacting particles. In quantum mechanics we can write the Hamiltonian for such a system as

$$H = H_0 + H_1,$$ (88)

where $H_0$ is the Hamiltonian for completely non-interacting particles, and $H_1$ is a small correction due to the particle interactions. We can define approximate stationary eigenstates of the system using $H_0$. Thus,

$$H_0 \,{\mit\Psi}_r = E_r \,{\mit\Psi}_r,$$ (89)

where the index $r$ labels a state of energy $E_r$ and eigenstate ${\mit\Psi}_r$. In general, there are many different eigenstates with the same energy: these are called degenerate states.

For example, consider $N$ non-interacting spinless particles of mass $m$ confined in a cubic box of dimension $L$. According to standard wave-mechanics, the energy levels of the $i$th particle are given by

$$e_i = \frac{\hbar^2 \pi^2}{2\,m \,L^2}\left( n_{i1}^{~2}+n_{i2}^{~2} +n_{i3}^{~2}\right),$$ (90)

where $n_{i1}$, $n_{i2}$, and $n_{i3}$ are three (positive integer) quantum numbers. The overall energy of the system is the sum of the energies of the individual particles, so that for a general state $r$

$$E_r = \sum_{i=1}^N e_i.$$ (91)

The overall state of the system is thus specified by $3N$ quantum numbers (i.e., three quantum numbers per particle). There are clearly very many different arrangements of these quantum numbers which give the same overall energy.

Consider, now, a statistical ensemble of systems made up of weakly interacting particles. Suppose that this ensemble is initially very far from equilibrium. For instance, the systems in the ensemble might only be distributed over a very small subset of their accessible states. If each system starts off in a particular stationary state (i.e., with a particular set of quantum numbers) then, in the absence of particle interactions, it will remain in that state for ever. Hence, the ensemble will always stay far from equilibrium, and the principle of equal a priori probabilities will never be applicable. In reality, particle interactions cause each system in the ensemble to make transitions between its accessible ``stationary'' states. This allows the overall state of the ensemble to change in time.

Let us label the accessible states of our system by the index $r$. We can ascribe a time dependent probability $P_r(t)$ of finding the system in a particular approximate stationary state $r$ at time $t$. Of course, $P_r(t)$ is proportional to the number of systems in the ensemble in state $r$ at time $t$. In general, $P_r$ is time dependent because the ensemble is evolving towards an equilibrium state. We assume that the probabilities are properly normalized, so that the sum over all accessible states always yields

$$\sum_r P_r(t) =1.$$ (92)

Small interactions between particles cause transitions between the approximate stationary states of the system. There then exists some transition probability per unit time $W_{rs}$ that a system originally in state $r$ ends up in state $s$ as a result of these interactions. Likewise, there exists a probability per unit time $W_{sr}$ that a system in state $s$ makes a transition to state $r$. These transition probabilities are meaningful in quantum mechanics provided that the particle interaction strength is sufficiently small, there is a nearly continuous distribution of accessible energy levels, and we consider time intervals which are not too small. These conditions are easily satisfied for the types of systems usually analyzed via statistical mechanics (e.g., nearly ideal gases). One important conclusion of quantum mechanics is that the forward and inverse transition probabilities between two states are the same, so that

$$W_{rs} = W_{sr}$$ (93)

for any two states $r$ and $s$. This result follows from the time reversal symmetry of quantum mechanics. On the microscopic scale of individual particles, all fundamental laws of physics (in particular, classical and quantum mechanics) possess this symmetry. So, if a certain motion of particles satisfies the classical equations of motion (or Schrödinger's equation) then the reversed motion, with all particles starting off from their final positions and then retracing their paths exactly until they reach their initial positions, satisfies these equations just as well.

Suppose that we were to ``film'' a microscopic process, such as two classical particles approaching one another, colliding, and moving apart. We could then gather an audience together and show them the film. To make things slightly more interesting we could play it either forwards or backwards. Because of the time reversal symmetry of classical mechanics, the audience would not be able to tell which way the film was running (unless we told them!). In both cases, the film would show completely plausible physical events.

We can play the same game for a quantum process. For instance, we could ``film'' a group of photons impinging on some atoms. Occasionally, one of the atoms will absorb a photon and make a transition to an ``excited'' state (i.e., a state with higher than normal energy). We could easily estimate the rate constant for this process by watching the film carefully. If we play the film backwards then it will appear to show excited atoms occasionally emitting a photon and decaying back to their unexcited state. If quantum mechanics possesses time reversal symmetry (which it certainly does!) then both films should appear equally plausible. This means that the rate constant for the absorption of a photon to produce an excited state must be the same as the rate constant for the excited state to decay by the emission of a photon. Otherwise, in the backwards film the excited atoms would appear to emit photons at the wrong rate, and we could then tell that the film was being played backwards. It follows, therefore, that as a consequence of time reversal symmetry, the rate constant for any process in quantum mechanics must equal the rate constant for the inverse process.

The probability $P_r$ of finding the systems in the ensemble in a particular state $r$ changes with time for two reasons. Firstly, systems in another state $s$ can make transitions to the state $r$. The rate at which this occurs is $P_s$, the probability that the systems are in the state $s$ to begin with, times the rate constant of the transition $W_{sr}$. Secondly, systems in the state $r$ can make transitions to other states such as $s$. The rate at which this occurs is clearly $P_r$ times $W_{rs}$. We can write a simple differential equation for the time evolution of $P_r$:

$$\frac{dP_r}{dt} = \sum_{s\neq r} P_s W_{sr} - \sum_{s\neq r} P_r W_{rs},$$ (94)

or

$$\frac{dP_r}{dt} = \sum_{s} W_{rs}(P_s -P_r),$$ (95)

where use has been made of the symmetry condition (93). The summation is over all accessible states.

Consider now the quantity $H$ (from which the $H$ theorem derives its name), which is the mean value of $\ln P_r$ over all accessible states:

$$H\equiv \overline{\ln P_r} \equiv \sum_r P_r \ln P_r.$$ (96)

This quantity changes as the individual probabilities $P_r$ vary in time. Straightforward differentiation of the above equation yields

$$\frac{dH}{dt} = \sum_r \left(\frac{d P_r}{dt} \ln P_r +\frac{d P_r} {dt}\right) = \sum_r \frac{d P_r}{dt}\,(\ln P_r + 1).$$ (97)

According to Eq. (95), this can be written

$$\frac{d H}{dt} = \sum_r\sum_s W_{rs}\, (P_s-P_r)\,(\ln P_r +1).$$ (98)

We can now interchange the dummy summations indices $r$ and $s$ to give

$$\frac{d H}{dt} = \sum_r\sum_s W_{sr}\, (P_r-P_s)\,(\ln P_s +1).$$ (99)

We can write $d H/dt$ in a more symmetric form by adding the previous two equations and making use of Eq. (93):

$$\frac{d H}{dt}= -\frac{1}{2}\sum_r \sum_s W_{rs}\,(P_r-P_s)\,(\ln P_r -\ln P_s).$$ (100)

Note, however, that $\ln P_r$ is a monotonically increasing function of $P_r$. It follows that $\ln P_r > \ln P_s$ whenever $P_r > P_s$, and vice versa. Thus, in general, the right-hand side of the above equation is the sum of many negative contributions. Hence, we conclude that

$$\frac{d H}{dt} \leq 0.$$ (101)

The equality sign only holds in the special case where all accessible states are equally probable, so that $P_r=P_s$ for all $r$ and $s$. This result is called the $H$ theorem, and was first proved by the unfortunate Professor Boltzmann.

The $H$ theorem tells us that if an isolated system is initially not in equilibrium then it will evolve under the influence of particle interactions in such a manner that the quantity $H$ always decreases. This process will continue until $H$ reaches its minimum possible value, at which point $d H/dt=0$, and there is no further evolution of the system. According to Eq. (100), in this final equilibrium state the system is equally likely to be found in any one of its accessible states. This is, of course, the situation predicted by the principle of equal a priori probabilities.

You may be wondering why the above argument does not constitute a mathematically rigorous proof that the principle of equal a priori probabilities applies to many particle systems. The answer is that we tacitly made an unwarranted assumption: i.e., we assumed that the probability of the system making a transition from some state $r$ to another state $s$ is independent of the past history of the system. In general, this is not the case in physical systems, although there are many situations in which it is a pretty good approximation. Thus, the epistemological status of the principle of equal a priori probabilities is that it is plausible, but remains unproven. As we have already mentioned, the ultimate justification for this principle is empirical: i.e., it leads to theoretical predictions which are in accordance with experimental observations.