The subject of this segment is the calculation of the PMF of
the sum of two independent, discrete random variables.
This is the simplest example of a function of two random
variables, a function of the form of g of X and Y, where
the function g happens to be just the
sum of the two arguments.
This is a very important example, because there are
many situations where random variables get
added to each other.
We work with discrete random variables as a warm up.
And later, we will consider the case of
continuous random variables.
So suppose that we know the PMFs of X and Y and that we
want to compute the probability that the sum is
equal to 3.
It always helps to have a picture.
The sum of X and Y will be equal to 3.
This is an event that can happen in many ways.
For example, x could be 3 and Y could be 0, or X could be 1
and Y equal to 2.
The probability of the event of interest, that the sum is
equal to 3, is going to be the sum of the probabilities of
all the different ways that this event can happen.
So it is going to be a sum of various terms.
And the typical term would be the probability, let's say of
this outcome, which is that X is equal to 0 and
Y is equal to 3.
Another typical term in the sum will be the probability of
this outcome here, the probability that X is equal to
1, Y is equal to 2, and so on.
Now, here comes an important step.
Because we have assumed that X and Y are independent, the
probability of these two events happening is the
product of the probabilities of each one of these events.
So it is the product of the probability that X is equal to
0, where now I'm using PMF notation, times the
probability that Y is equal to 3.
Similarly, the next term is the probability that X is
equal to 1 times the probability that
Y is equal to 2.
Again, we can do this because we are assuming that our two
random variables are independent of each other.
Now, let us generalize.
In the general case, the probability that the sum takes
on a particular value little z can be calculated as follows.
We look at all the different ways that the sum of little z
can be obtained.
One way is that the random variable X takes on a specific
value little X. And at the same time, the random variable
Y takes the value that's needed so that the sum of the
two is equal to little Z. For a given value of little X, we
have a particular way that the sum is equal to Z. And this
particular way has a certain probability.
But little X could be anything.
And different choices of little x give us different
ways that the event of interest can happen.
So we add those probabilities over all possible X's.
And then we proceed as follows.
We invoke independence of X and Y to derive this
probability as a product of two probabilities.
And then we use PMF notation instead of probability
notation to obtain this expression here.
This formula is called the convolution formula.
It is the convolution of two PMFs.
What convolution means is that somebody gives us the PMF of
one random variable, gives us also the PMF of
another random variable.
And when we say we're given the PMF, it means we're given
the values of the PMFs for all the possible choices of little
X and little y, the arguments of the two PMFs.
Then the convolution formula does a certain calculation and
spits out now a new PMF, which is the PMF of the random
variable Z. Let's now take a closer look at what it takes
to carry out of the calculations involved in this
convolution formula.
Let's proceed by a simple example that will illustrate
the methodology.
We're given two PMFs of two random variables.
And assuming that they are independent, the PMF of their
sum is determined by this formula here.
And we want to see what those terms in this
summation would be.
Suppose that we're interested in the probability that the
sum is equal to 3.
Now, the sum is going to be equal to 3.
This can happen in several ways.
We could have X equal to 1 and and Y equal to 2.
This combination is one way that the
sum of 3 can be obtained.
And that combination has a probability of 1/3 times 3/6.
And that would be one of the terms in this summation.
Another way that the sum of 3 can be obtained is by having X
equal to 4 and y equal to minus 1.
And by multiplying this probability 2/3 with 2/6, we
obtain another contribution to this summation.
However, keeping track of these correspondences here can
become a little complicated if we have richer our PMFs.
So an alternative way of arranging the calculation is
the following.
Let us take the PMF of Y, flip it along this vertical axis.
So these two terms would go to the left side, and this term
will go to the right hand side.
And then draw it underneath the PMF of X.
This is what we obtain.
Then let us take this drawing here and shift it to
the right by 3.
So the entry of minus 2 goes to 1, minus 1 goes to 2,
and 1 goes to 4.
So what have we accomplished by these two transformations?
Well, the term that had probability 3/6 and which were
to be multiplied with the probability 1/3 on that side,
now this 3/6 sits here.
So we have this correspondence.
And we need to multiply 1/3 by 3/6.
Similarly, the multiplication of 2/3 with 2/6 corresponds to
the multiplication of this probability here times the
probability of this term here.
So when the diagrams are arranged this way, then we
have a simpler job to do.
We look at corresponding terms, those that sit on top
of each other, multiply them, do that for all the possible
choices, and then add those products together.
And this is what we do if we're shifting by 3.
Now, if we wanted to find the probability that Z equal to 4,
we would be doing the same thing, except that this
diagram would need to be shifted by one more unit to
the right so that we have a total shift of 4.
So we just repeat this procedure for all possible
values of Z which corresponds to taking this diagram here
and shifting it progressively by different amounts.
This turns out to be a fairly simple and systematic way of
arranging the calculations, at least if you're
doing them by hand.
Of course, an alternative is to carry out the calculations
on a computer.
This is a pretty simple formula that is not hard to
implement on a computer.