The two-state system

The simplest non-trivial system which we can investigate using probability theory is one for which there are only two possible outcomes. There would obviously be little point in investigating a one outcome system. Let us suppose that there are two possible outcomes to an observation made on some system $S$. Let us denote these outcomes 1 and 2, and let their probabilities of occurrence be

$$P(1) = p,$$ (9)

$$P(2) = q.$$ (10)

It follows immediately from the normalization condition (5) that

$$p+q=1,$$ (11)

so $q=1-p$. The best known example of a two-state system is a tossed coin. The two outcomes are ``heads'' and ``tails,'' each with equal probabilities $1/2$. So, $p=q=1/2$ for this system.

Suppose that we make $N$ statistically independent observations of $S$. Let us determine the probability of $n_1$ occurrences of the outcome $1$ and $N-n_1$ occurrences of the outcome 2, with no regard to the order of these occurrences. Denote this probability $P_N(n_1)$. This type of calculation crops up again and again in probability theory. For instance, we might want to know the probability of getting nine ``heads'' and only one ``tails'' in an experiment where a coin is tossed ten times, or where ten coins are tossed simultaneously.

Consider a simple case in which there are only three observations. Let us try to evaluate the probability of two occurrences of the outcome 1 and one occurrence of the outcome 2. There are three different ways of getting this result. We could get the outcome 1 on the first two observations and the outcome 2 on the third. Or, we could get the outcome 2 on the first observation and the outcome 1 on the latter two observations. Or, we could get the outcome 1 on the first and last observations and the outcome 2 on the middle observation. Writing this symbolically

$$P_3(2)= P(1\otimes 1\otimes 2 \mid 2\otimes 1\otimes 1 \mid 1\otimes 2 \otimes 1).$$ (12)

This formula looks a bit scary, but all we have done here is to write out symbolically what was just said in words. Where we said ``and'' we have written the symbolic operator $\otimes$, and where we said ``or'' we have written the symbolic operator $\mid$. This symbolic representation is helpful because of the two basic rules for combining probabilities which we derived earlier

$$P(X\mid Y) = P(X) + P(Y),$$ (13)

$$P(X \otimes Y) = P(X)\,P(Y).$$ (14)

The straightforward application of these rules gives

$$P_3(2) = p\,p\,q + q\,p\,p + p\,q\,p = 3 \,p^2\,q$$ (15)

in the case under consideration.

The probability of obtaining $n_1$ occurrences of the outcome $1$ in $N$ observations is given by

$$P_N(n_1) = C^{ N}_{n_1,\,N-n_1}\,p^{n_1}\, q^{N-n_1},$$ (16)

where $C^{ N}_{ n_1,\, N-n_1}$ is the number of ways of arranging two distinct sets of $n_1$ and $N-n_1$ indistinguishable objects. Hopefully, that this is, at least, plausible from the example we just discussed. There, the probability of getting two occurrences of the outcome 1 and one occurrence of the outcome 2 was obtained by writing out all of the possible arrangements of two $p$s (the probability of outcome 1) and one $q$ (the probability of outcome 2), and then added them all together.