Let us briefly review how we
obtained the Gaussian distribution function in the
first place. We started from a
very simple system with only two possible outcomes. Of course, the
probability distribution function (for 
) for this system did not look
anything like a Gaussian. However, when we combined very many
of these simple systems together,
to produce a complicated system with a great number of possible
outcomes, we found that the resultant probability distribution function
(for )
reduced to a
Gaussian in the limit as the number of simple systems tended
to infinity.
We started from a two outcome
system because it was easy to calculate the final probability
distribution function when a finite number of such systems were
combined together. Clearly, if we had started from a more complicated
system then this calculation would have been
far more difficult.
Let me now tell you something which
is quite astonishing! Suppose that we start from any system,
with any distribution function (for some measurable quantity 
). If
we combine a sufficiently large number of
such systems together, the resultant distribution function
(for ) is always Gaussian.
This proposition is known as the central limit theorem. As far as
Physics is concerned, it is one of
the most important theorems in the whole of mathematics.
Unfortunately, the central limit theorem is notoriously difficult to prove. A somewhat restricted proof is presented in Sections 1.10 and 1.11 of Reif.
The central limit theorem guarantees that the probability distribution of any measurable quantity is Gaussian, provided that a sufficiently large number of statistically independent observations are made. We can, therefore, confidently predict that Gaussian distributions are going to crop up all over the place in statistical thermodynamics.