By this point, we have discussed pretty much
everything that is to be said about individual discrete
random variables.
Now let us move to the case where we're dealing with
multiple discrete random variables simultaneously, and
talk about their distribution.
As we will see, their distribution is characterized
by a so-called joint PMF.
So suppose that we have a probabilistic model, and on
that model we have defined two random variables--
X and Y. And that we have available
their individual PMFs.
These PMFs tell us about one random variable at the time.
This tells us about X, this tells us about Y. But they do
not give us any information about how the two random
variables are related to each other.
For example, if you wish to answer this question, whether
the numerical values that the two random variables happen to
be equal, and what is the probability of that event, you
will not be able to answer this question if you only know
the two individual PMFs.
In order to be able to answer a question of this type, we
will need information that tells us what values of X tend
to occur together with what values of Y. And this
information is captured in the so-called joint PMF.
So the joint PMF is nothing but a piece of notation for an
object that's familiar.
This is the probability that when we carry out the
experiment we happen to see random variable X take on a
value, little x.
And simultaneously see that random variable Y takes on a
value, little y.
This quantity we indicate it with this notation.
The letter little p stands for a PMF.
The subscripts tell us which random variables
we're talking about.
And finally, this is a function of two arguments.
Depending on what pair (x,y) we're interested in, we're
going to get a different numerical value for this
probability.
As an example of a joint PMF in which the two random
variables take values in a finite set, we might be given
a table of this form.
Using this table, we can answer questions such as the
following--
what is the probability that the random variables X and Y
simultaneously take the values, let us say, 1 and 3?
Then we look up in this table, and we identify that it's this
probability, X takes the value of 1, and Y takes
the value of 3.
And according to this table, the answer would be 2/20.
Now, something to notice about joint PMFs.
When you add over all possible pairs, x and y, this exhausts
all the possibilities.
And therefore, these probabilities should add to 1.
In terms of this table, all of the entries that we have here
should add to 1.
Now, once we have in our hands the joint PMF, we can use it
to find the individual PMFs of the random variables X and Y.
And these individual PMFs are called the marginal PMFs.
How do we find them?
Well, the joint PMF tells us everything there is to be
known about the two random variables, so it should
contain enough information for us to
answer any kind of question.
So for example, if we wish to find the probability that the
random variable X takes the value of 4, we look at all
possible outcomes in which X is equal to 4, and add the
probabilities of these outcomes.
So in this case, it would be 1/20 plus 2/20.
So what we're doing is that if we're interested in a specific
value of X, the probability that X takes on a specific
value, we consider all possible pairs associated with
that fixed x.
That is, we're considering one column of the PMF, and we're
adding the corresponding probabilities.
So to find this entry here, let's say px(3), what we need
is to add these terms on that column.
Similarly, we can find the PMF of the random variable Y.
So for example, the probability that the random
variable Y takes on a value of, let's say, 2, can be found
as follows.
You look at the probabilities of all pairs associated with
this specific y, and you add over the x's.
So we fix Y to have a value of 2, and we add over all pairs
in this row.
So in this example, it would be 1/20 plus 3/20, plus 1/20.
Finally, notice that we are able to answer the question
that got us motivated in the first place.
To find the probability that the two random variables take
equal values, we look at all the outcomes for which the two
random variables indeed take the same numerical values.
And we see that it is this event in this diagram, and the
probability of that event is going to be 2/20.
So in general, once we have available the joint PMF of two
random variables, we will be able to answer any questions
regarding probabilities of events that have to do with
these two random variables.
How about more than two random variables?
It's just a matter of notation.
For example, we can define the joint PMF of three random
variables, and you can use the same idea for the joint PMF,
let's say, of five, or 10, or n random variables.
Let's just look at the notation for three.
There is a well-defined probability that when we carry
out the experiment X, Y and Z as random variables take on
certain specific values.
So we look at the probability of that particular triple, and
we indicate that probability with this notation.
Once more, the sub-scripts tell us which random variables
we're talking about.
And the PMF, of course, is going to be a function of this
triple, little x, little y, little z, because each triple
in general should have a different probability.
Of course, probabilities must always add to 1.
So when we consider all triples and we add their
corresponding probabilities, we should get 1.
And finally, once we have the joint PMF, we can again
recover the marginal PMF.
For example, to find the probability that the random
variable takes on a specific value, little x, we consider
all possible triples in which the random variable indeed
takes that value, little x.
And then we sum over all the possible y's and z's that
could go together with this particular x.
In the same spirit, to find the probability that these two
random variables take on two specific values, we consider
all the possible z's that could go together with this
(x,y) pair.
So this way we're ranging over all outcomes in which X and Y
take on these specific values.
But Z is free to take any value, and so we consider all
those possible values of Z and sum the corresponding
probabilities.
Finally, we can talk about functions of
multiple random variables.
Suppose that we have two random variables, x and y, and
that we define a function of them.
So this function is, of course, a random variable.
And we can find the PMF of this random variable if we
know the joint PMF of X and Y.
So the PMF, which is the probability that the random
variable takes on a specific numerical value, that's the
probability that the function of X and Y takes on a specific
numerical value.
And we can find this probability by adding the
probabilities of all (x,y) pairs.
Which (x,y) pairs?
Those (x,y) pairs for which the value of Z would be equal
to this particular number, little z, that we care about.
So we collect essentially all possible outcomes that make
this event to happen, and we add the probabilities of all
those outcomes.
Finally, similarly to the case where we have a single random
variable and function of it, we now can talk about expected
values of functions of two random variables, and there is
an expected value rule that parallels the expected value
rule that we had developed for the case of a
function of this form.
The form that the expected value rule takes is similar,
and it's quite natural.
The interpretation is as follows.
With this probability, a specific
(x,y) pair will occur.
And when that occurs, the value of our random variable
is a certain number.
And the combination of these two terms gives us a
contribution to the expected value.
Now, we consider all possible (x,y) pairs that may occur,
and we sum over all these (x,y) pairs.