We end this lecture sequence by stepping back to discuss
what probability theory really is and what exactly is the
meaning of the word probability.
In the most narrow view, probability theory is just a
branch of mathematics.
We start with some axioms.
We consider models that satisfy these axioms, and we
establish some consequences, which are the
theorems of this theory.
You could do all that without ever asking the question of
what the word "probability" really means.
Yet, one of the theorems of probability theory, that we
will see later in this class, is that probabilities can be
interpreted as frequencies, very loosely speaking.
If I have a fair coin, and I toss it infinitely many times,
then the fraction of heads that I will
observe will be one half.
In this sense, the probability of an event, A, can be
interpreted as the frequency with which event A will occur
in an infinite number of repetitions of the experiment.
But is this all there is?
If we're dealing with coin tosses, it makes sense to
think of probabilities as frequencies.
But consider a statement such as the "current president of
my country will be reelected in the next election with
probability 0.7".
It's hard to think of this number, 0.7, as a frequency.
It does not make sense to think of infinitely many
repetitions of the next election.
In cases like this, and in many others, it is better to
think of probabilities as just some way of
describing our beliefs.
And if you're a betting person, probabilities can be
thought of as some numerical guidance into what kinds of
bets you might be willing to make.
But now if we think of probabilities as beliefs, you
can run into the argument that, well, beliefs are
subjective.
Isn't probability theory supposed to be an objective
part of math and science?
Is probability theory just an exercise in subjectivity?
Well, not quite.
There's more to it.
Probability, at the minimum, gives us some rules for
thinking systematically about uncertain situations.
And if it happens that our probability model, our
subjective beliefs, have some relation with the real world,
then probability theory can be a very useful tool for making
predictions and decisions that apply to the real world.
Now, whether your predictions and decisions will be any good
will depend on whether you have chosen a good model.
Have you chosen a model that's provides a good enough
representation of the real world?
How do you make sure that this is the case?
There's a whole field, the field of statistics, whose
purpose is to complement probability theory by using
data to come up with good models.
And so we have the following diagram that summarizes the
relation between the real world, statistics, and
probability.
The real world generates data.
The field of statistics and inference uses these data to
come up with probabilistic models.
Once we have a probabilistic model, we use probability
theory and the analysis tools that it provides to us.
And the results that we get from this analysis lead to
predictions and decisions about the real world.