Let us now consider a simple example.
Let random variables X and Y be described by a joint PMF
which is the one shown in this table.
Question--
are X and Y independent?
We can try to answer this question by using the
definition of independence.
But it is actually more instructive to proceed in a
somewhat more intuitive way.
We look at this table, and we observe that the value of one
is possible for X. In particular, the probability
that X takes the value of one, this is the marginal
probability, this can be found by adding the probabilities of
all of the outcomes in this column, which, in this case,
is 3 over 20.
Suppose now that somebody tells you the value of Y. For
example, I tell you that Y happens to be equal to one, in
which case you are transported into this universe.
In this universe, the conditional probability that X
takes a value of one, given that Y takes a value of one,
what is it?
In this universe, there's zero probability
associated to this outcome.
So this probability is zero, which is
different than 3 over 20.
And since these two numbers are different, this means that
information from Y changes our beliefs about what's going to
happen to X. And so, we do not have independence.
So again, intuitively, in the beginning, we thought that X
equal to one was possible.
But information given by Y, namely that Y is equal to one,
tells us that, actually, X equals to one is impossible.
Information about Y changed our beliefs about X, so X and
Y are dependent.
Now, when we first introduced the notion of independence
some time ago, we also introduced the notion of
conditional independence.
And we said that conditional independence is the same as
ordinary independence, except that it would be applied to a
conditional universe.
Something similar can be done for the case of random
variables as well.
So suppose, for example, that someone tells us that the
outcome of the experiment was such that it belongs
to this blue set.
This is the set where X is less than or equal to 2, and Y
is larger than or equal to three.
So we're given this information, and this is now
our new conditional model.
The question is, within this new conditional model are
random variables X and Y independent?
Let's just right down the conditional model, where I'm
only showing the four possible outcomes that are allowed in
the conditional model.
All the others, of course, will have zero probability in
the conditional model.
So in the conditional model, probabilities will keep the
same proportions as in the unconditional model--
and the proportions are 1, 2, 2, 4--
but then they need to be scaled, or normalized, so that
they add to 1.
And to make them add to 1, we need to divide them all by 9.
In this conditional model, this is the joint PMF of the
two random variables X and Y. Let us find the marginal PMFs.
To find the marginal PMF of X, we add the
entries in this column.
And we get here 1/3, and here 2/3.
And to find the marginal PMF of y, we add the
entries in this [row]
to find 2/3.
And we adds the entries in that [row]
to find 1/3.
So this is the marginal PMF of x.
That's the marginal PMF of Y. And now we notice that this
entry of the joint PMF is 1/3 times 1/3, the
product of the marginals.
This entry is the product of 1/3 times 2/3, the product of
the marginals, and so on for the remaining entries.
So each entry of the joint PMF is equal to the product of the
corresponding entries of the marginal PMFs.
And this is the definition of independence.
So in this conditional blue universe, we do have
independence.
And the way that this was established was to check that
the joint PMF factors as a product of marginal PMFs.