In order to build circuits to manipulate,
transmit or store information, we are going to need some
engineering tools to see if we're choosing a good
representation for the information --
that's the subject of this chapter.
We'll study different ways of encoding information as bits
and learn the mathematics that help us determine
if our encoding is a good one.
We'll also look into what we can do if our representation gets
corrupted by errors.
It would be nice to detect that something bad has happened
and possibly even correct the problem.
Let's start by asking "what is information?"
From our engineering perspective,
we'll define information as data communicated or received
that resolves uncertainty about a particular fact
or circumstance.
In other words, after receiving the data
we'll know more about that particular fact
or circumstance.
The greater the uncertainty resolved by the data,
the more information the data has conveyed.
LET'S LOOK AT AN EXAMPLE: a card has
been chosen at random from a normal deck of 52
playing cards.
Without any data about the chosen card,
there are 52 possibilities for the type of the card.
Now suppose you receive one of the following pieces of data
about the choice.
A. You learn the suit of the card is "heart".
This narrows the choice to down one of 13 cards.
B. You learn instead the card is NOT the Ace of Spades.
This still leaves 51 cards that it might be.
C. You learn instead that the card is a face card, that
is, a Jack, Queen or King.
So the choice is one of 12 cards.
D. You learn instead that the card is the "suicide king."
Our little blue friend is showing us that this is
actually a particular card – the King of Hearts where the king
is sticking the sword through his head!
No uncertainty here, we know exactly what the choice was.
Which of the possible pieces of data
conveys the most information?
In other words, which data resolves the most uncertainty
about the chosen card?
Similarly, which data conveys the least
amount of information?
Before we talk about the mathematics
behind the correct answers to these questions,
vote for your answers in the following poll.