Flash and JavaScript are required for this feature.
Download the video from Internet Archive.
Description: Quickly reviewed last lecture. Showed that \(\mathbb{N}\) and \(\mathbb{R}\) are not the same size to introduce the diagonalization method and used it to prove \(A_\bf{TM}\) is undecidable. Introduced the reducibility method to show that \(HALT_\bf{TM}\) is undecidable.
Instructor: Prof. Michael Sipser

Lecture 8: Undecidability
[SQUEAKING] [RUSTLING] [CLICKING]
PROFESSOR: OK, why don't I get started? So OK, what have we been doing? So last time, we considered a bunch of procedures for testing properties of various automata and grammars, the acceptance problem for DFAs, for NFAs, the acceptance problem, which is really degeneration problem for context-free grammars, and emptiness problems for DFAs and context-free grammars. And also, we showed that A,TM is Turing-recognizable.
There's a question here in the chat already about that. Yes, we did show that. That was the universal Turing machine that we presented at the end. That's what shows that A,TM is Turing-recognizable. We mentioned that A,TM is not decidable, which we promised we will prove today. And we will do so. OK, so that is the plan-- proving A,TM is undecidable. And we'll introduce the method for doing that called the diagonalization method.
I will also show the complement of A,TM is Turing-unrecognizable. Even though A,TM itself is recognizable, the complement is not. And then we will introduce another method called the reducibility method for showing other problems are not decidable and give one example of another problem, which is not decidable, assuming we have time to get to it. If not, we will delay that part until next Tuesday's lecture.
All right, acceptance problem for Turing Machines. So just as I mentioned, we showed that A,TM, which is the language of Turing machines and inputs where the machine accepts the input. We showed that was recognizable. We claimed it was decidable. Today, we're going to prove it's not decidable.
All right, now the method that we're going to use, which is really the only method out there for proving a problem as undecidable is called the diagonalization method. I mean, ultimately, we're going to show the reducibility method as well. But it really depends on having already shown some other problem undecidable via the diagonalization method. So ultimately, everything hinges on the diagonalization method, which is really what we have.
And so to introduce the diagonalization method, I'm going to make a bit of a digression into a branch of mathematics called set theory or it's a part of mathematical logic, where the method of diagonalization was first conceived of back in the late 19th century by a mathematician called Georg Cantor. And Cantor was considering the problem of, how do you compare the relative sizes of infinite sets? For finite sets, the problem of comparing-- somebody said that Cantor went crazy. That is true. And maybe I don't know why he went crazy. But he did go-- he had some mental problems, unfortunately.
And so how do we compare the sizes of sets in general? If they're finite sets, we can just count them up. We can say, whoa, this set has 11 elements, and the other set has 10 elements. So the one with 11 is bigger. Or if they both have 11, they're the same size. Well, that's not going to work for infinite sets because you can't count them up.
And so he had-- Cantor had the following idea for comparing the sizes of infinite sets. And that was, basically, to see whether you can have a function that would map from one set to the other set with certain properties. And those properties are called, traditionally, well, I mean, in the past, have been called the one-to-one and onto properties for the function. I'll tell you what that means.
But the concept is very simple. So a one-to-one function is a function that's mapping from A to B. Those are the two sets whose sizes we're trying to compare. And the function being one-to-one just means that there are no collisions. If you have two different elements of A, they're never going to map onto the same element of B. So two different elements of A always map onto two different elements of B. So that's the one-to-one property. It's also called injective.
And the other property is called onto or surjective, which is that the range of f has to be all of B. So you're not allowed to miss any elements, but you have to hit everything. And when you have both of those properties, the function is called a one-to-one correspondence or a bijection also, OK?
Now another way of looking at it-- I don't want to make this more complicated than it needs to be. It just simply means that two sets are considered to be the same size if we can match up the elements with one set with elements of the other set. You just pair them up. For example, well, if you have finite sets, that idea, that informal idea just works exactly as you would expect.
For example, if we have two sets-- here's a set of puppies. Here's a set of kittens. Now we want to show that those two sets have the same size. We could count them up, as I mentioned, and see that there are six elements in both. But counting up does not work for infinite sets.
So we can just match up the elements of the puppies with the kittens, and then we know we have the same number of puppies as kittens. OK, now that has the advantage of it making sense when you have infinite sets. So we're just going to extend that idea and apply it to infinite sets too. And then we'll have a notion of what it means for two infinite sets to have the same size.
And you might wonder, what do you get? Are all infinite sets of the same size when you use this notion or not? What happens? Well, some strange things do happen. But there actually are quite some interesting structure there that emerges. So anyway, I don't want to rush on. Questions on any of this? If you want to-- quick question. This, hopefully, was not too hard, but I want to make sure everybody's together with me on it, so we can pop in-- I'll give a few seconds for a chat if you have any questions.
The range of the set is all of the elements that you hit as you look at the different possible elements of A. So all of the things that f hits, the standard notion of a range of a function. So the range of f has to be equal to B. You have to hit everything.
All right, can you think of a one-to-one correspondence as a relabeling? Yeah, I'm not sure that's helpful. But, yes, you can think of as a relabeling of the elements in a sense, but yeah. I just think of it as a matching up.
All right, so let's move on. So coming out of this notion of sets being of the same size with this notion of countable sets, as we'll see in a minute, so let's do an example. Let's take the set of natural numbers-- 1, 2, 3, 4, dot, dot, dot, dot, and the set of integers, which includes the natural numbers but also the negative numbers and zero.
So the natural numbers are typically referred to as N. The integers are typically referred to as Z. And what do we think of the relative sizes of N and Z? Well, N is a subset of Z. It's a proper subset of Z. So you might, at first glance, think that Z is larger, and they're not really going to be of the same size.
But actually, it turns out that there's a very simple way-- the arithmetic and the properties of infinite sets can be a little bit surprising. And there is a way of matching up all of the elements of Z with their own elements of N. And so you can show that following that definition, that these two sets are, in fact, of the same size. So let's just quickly go make sure you see that.
So here is N. Here is Z I'm going to write down a table which shows how they match up. Here is the function f of n that I'm going to be describing. That's the one-to-one correspondence. And so here are the natural numbers-- dot, dot, dot, 1 through 7, dot, dot, dot. And here are the elements of Z that I'm going to be matching up. This is how the function is working.
So one, I'm going to-- f of 1 is going to be 0. F of 2 is going to be minus 1. F of 3 is going to be 1. And I'm just giving you a way to match up each of the natural numbers with integers so that every single integer has its own natural number and vice versa. So 4 goes to minus 2. 5 goes to 2. 6 goes to minus 3. 7 goes to 3, and then minus 4, and then 4, and minus 5, and then 5, and so on.
You're clearly going to cover all of the integers in this way. And each of the natural numbers is going to have its own integer. And there's never going to be any collisions. There's never going to be two natural numbers assigned to the same integer. So this meets the conditions of a one-to-one correspondence and shows that the natural numbers and the integers have the same size following this definition.
OK, let's do one that's slightly more complicated and, perhaps, slightly more surprising, which is that if you consider all of the rational numbers, and then they have the same-- that is a collection. Even though the rational numbers seem to be much richer and more numerous than the integers, from this perspective, they have the same size.
And for the simplicity of my presentation, I'm going to consider only the positive rational numbers, which I'm going to write as Q+. So those are all of the positive rational numbers that can be expressed as a ratio of two natural numbers. And I'm going to show that there is a one-to-one correspondence between these positive rational numbers and the natural numbers.
OK, so here, I'm going to, again, make a table, just as I did up above-- so comparing the natural numbers and the positive rational numbers. And to do that, I'm going to write down over here on the side, just to help you see how I'm getting this table, a separate table that has all of the positive rational numbers appearing in a nicely organized way.
Here are all of the rational numbers here that have 1 as a numerator, that have 2 as a numerator, and so on, and going through across the columns, the different denominators. So the numbers inside here represent all of the different rational numbers. And so whatever rational number you have, m over n, that's going to appear down the nth row in the nth column. That number is going to appear. So they're all going to show up.
And I'm going to use this table here to fill out this function. So here are all the natural numbers. And the way I'm going to assign the rational numbers to appear over here is I'm just going to work my way in from the corner. And I'm going to do that by doing layers. So first, I'll take the 1 in this number here in this corner. Then, I'll do these three that surrounded it, and then these guys that surround that, and these guys sort of in shells going around the previous ones that I've already covered. OK, so let me illustrate that.
So here's 1 over 1, my first rational number that I'm going to enter into the table, appearing right over there. So next, I'm going to go 2 over 1. That's going to appear in the table over there. Now, we have 2 over 2. Now that's actually a little bit problematic because 2 over 2 has already been done. And if we put that in-- because 2 over 2 equals 1 over 1. They're both the equal to the rational number 1.
So if we put 2 over 2 in this table over here, then we would no longer have the one-to-one property because both 1 and 3 would be mapping to the same point. So we're just going to simply skip over 2 over 2. I'll show that as kind of graying it out. So we're not going to add that one on to the table onto my function.
But so we'll skip over that. We'll go to 1 over 2, which has not been seen before. And then we're just going to continue doing the same thing, now going to this next shell out here. 3 over 1, 3 over 2, 3 over 3-- same thing, we're going to skip over that one-- 2 over 3, 1 over 3. I hope you're getting the idea. So I'm not going to fill this table. I ran out of room to fill out this table some more.
But just to look at how the process is going to continue here, we get 4 over 1. Now 4 over 2, again, is a number we previously seen because that's the rational number 2. We saw that up here when we had 2 over 1, so we're going to have to skip that one. 4 over 3, 4 over 4-- that was going to skip. 3 over 4, and so on. OK? So by following this procedure, I'm going to be able to define this function. Now this function is not particularly nice in terms of having an elegant, closed form.
But it is a totally legitimate function in the sense of being a mapping from natural numbers to the positive rational numbers. And it has the one-to-one correspondence property. So it pairs up each of the natural numbers with each of the positive rational numbers. They each get their own mate, in a sense. And so that shows that the rational numbers, despite the fact that they're dense, and they have all sorts of very more much bigger seeming, they really, from this perspective of the sizes of the sets, they have the same size as the natural numbers do.
And so with that, it suggests that the following definition that a set is countable if it has the same size as the natural numbers or it's finite. From this perspective, we're going to be focusing on infinite sets. But yeah, countable or countably infinite, sometimes people say, if it has the same size as the natural numbers. Or otherwise, you have to include the finite sets as well.
And countable means you can just go through the elements of the set as a list. So you can count them-- the first one, the second one, the third one, the fourth one, dot, dot, dot. And once you can do that, and that counting is going to hit everything, then you know can match them-- you can pair them up with the natural numbers. And so therefore, you have a countable set. OK.
So as we've shown, both Z, the integers, and the positive rational numbers are countable sets. OK? Now you might think that, well, since we're allowing ourselves to do something as arbitrary, in a way, as this kind of a function to match up the natural numbers with whatever set you're trying to measure, that every set is going to be countable, if you think about it that way because it seems like you're allowing too many possibilities.
And then all the infinite sets are going to end up being the same size as the natural numbers. Well, that is, in fact, is not true. And I don't know. Cantor, is the one who discovered that. I don't know if that was surprising or not. But it is interesting, I think, that there are different sizes of infinities.
And so we're going to now take a look and see that. But I want to, again, I want to stop and make sure we're all together. Can we always find a closed formula for f, somebody's asking me. I don't know. For this particular f, you probably could, but it would probably be very messy. But in general, that's not a requirement, having a closed form for the mapping function. Any function is allowed as long as a one-to-one correspondence.
Are we all together on this? Are we comfortable with the notion of some infinite sets having the same size as the natural numbers, and therefore, we're going to call them countable sets? And we're going to show some other sets are going to have more elements. They're going to be uncountable. They're going to be beyond-- strictly speaking, strictly larger sets in the sense that we won't be able to put them into a one-to-one correspondence with the integers. So is N the smallest infinity? Yes, N is the smallest one.
So any infinity is going to have-- you're going to always-- I don't think you even need it-- you need a special-- whenever you have an infinite collection, you can always find a subset which is going to be a countable subset and is going to be the smallest of the infinities, but there are bigger ones. All right, why don't we move on?
OK, so the example of a set that we will show is not countable-- an uncountable set, as we call it-- is a set of real numbers, which we all know what these are, I hope. These are all of the numbers that you can express by possibly infinite decimal representations like pi, or E squared of 2, or any of the other familiar ones. Rational numbers are also members of or also count as real numbers and integers too. And so but there are these additional numbers that you can get by the decimal expansions which may not be rational.
And that collection, even though, in some ways, it looks somewhat similar to the rational numbers, the real numbers-- I hope I said that right. The real numbers, which is the set I'm considering here, the ones with the infinite decimal expansions, they actually are much larger. So the theorem is-- and this was discovered by Cantor-- that R is an uncountable set. And the reason why I'm introducing this is, besides that I think it's interesting and has some relation to the kinds of things we're doing, but it's really for the purposes right now, is to introduce this method called diagonalization.
OK, that's what we're going to use later on again. But this is the first time it appeared. So we're going to use a proof by contradiction in order to show that R, the collection of real numbers, is uncountable. And we'll assume, for contradiction, that R countable. OK?
So if we assume that R is countable, it means we have, by definition, a one-to-one correspondence from the natural numbers to the real numbers. OK, so we can match up all of the natural numbers with the real numbers in a one-to-one and onto fashion. So we can pair up the natural numbers with the real numbers. And we will cover all of the real numbers by doing that. And we can make a table, just like I did before. Here it is.
And you can fill this out with all of the real numbers. And that was what the assumption means. And I'm going to show you that that's impossible. Something is going to go terribly wrong if you do that. Now you might disagree. You might be surprised. You might think, well, Professor Sipser is wrong, that I'm going to work on this overnight and forget the rest of my classes, and I'm going to come up with a list of all the real numbers. I'm going to fill them out here and show that that's impossible.
So let's say, hypothetically, just for my example's sake, here is the list that you came up with. So you have to match up something with the 1. Let's say, E. That was a special case, so you're going to stick that with 1. And then pi came out to be the number that you connected up with 2. You understand what I'm doing here? I'm making a list. I'm trying to make my correspondence-- my matching up between the natural numbers and my real numbers that I'm hypothesizing to exist for a contradiction.
So 3, I don't know, 3 is gets connected to 0. I'm making this up, obviously-- not on the spot. I wrote the slide, you know, yesterday. But here is the square root of 2 showing up for whatever reason. Here is 1/7. Here is some other number, which, I'll be interested if you recognize that one. That's a subtle one. This is 1.234-- yeah, some familiar looking sequence here. And whatever it is-- whatever it is, this is what you came up with.
You claim that this is going to work as-- very good. Somebody got it. It's i to the ith power. But let's not get hung up on that, please. I to the ith power, oddly enough, can be seen as a real number under-- if you define the imaginary exponentiation, which is weird.
But anyway, let's not get too distracted. So here's my list of numbers, my real numbers that I've matched up in my table here with the natural numbers. Now I claim something goes wrong. What goes wrong? I'm going to show you that you actually did not do as you claimed, that there's actually at least one number that's missing from this list. And I'll exhibit that number. I'm going to show you what that number is. I'm going to explicitly come up with a number and show you that there's that number it's missing from the list.
So here, it's going to be a number x. X is the one that's missing. So how am I going to get x? So x, well, I'm going to start it off with 0 point, and then I'm going to fill out the rest of the places. And how am I going to get those places? I'm going to look at the table here.
So I want to look at-- to get my first digit after the decimal point of x, I'm going to look at the first digit after the decimal point of the very first number. So I take the first number, look at the first digit after the decimal point, which happens to be a 7. And the number I'm going to put in for x It's not going to be 7. It's going to be anything except 7. Let say, 8.
For my next digit of x, I'm going to look at-- so my second digit of x, I'm going to-- everything's going to be up to the decimal point now, so I'm not going to keep saying that. I'm going to look at the second digit of the second number, which is a 4 after the decimal point. There it is. And for the second digit of x, I'm going to use something which is anything besides 4-- 5. I have some flexibility here.
So I could have used 6. I could have used 7. For those of you who have seen this before, who are going to nitpick on me, I'm going to stay away from 9 and 0 just because there is some little edge cases that arise there, which I don't want to get into because I don't think it's interesting for this argument. But since I have some flexibility, I'm going to avoid zeros and nines-- probably just nines is all I really need. And then the argument is just going to work fine, OK?
So the next digit is a 0. The third digit, or the third number is a 0. The third digit of x is going to be anything except for 0. It could be a 1. Then I have a 2 here. Anything except a 2, 6. Then a 5-- anything except a 5, a 1. A 9-- anything except a 9, an 8, and so on. There's an 8. There's a 2, and dot, dot, dot. I'm just going to keep doing that.
And I'm going to say you missed that number, whatever it is. But that number, that's a number. After I'm all done, it's going to be the decimal representation of something. And that number, I claim, is absent from the table. And you might say, well, it's really there. It's just on the 23rd row. You just didn't get to it and your slide. But it's there. Well, I'm going to say I know it's not in the 23rd row because it differs from the number in the 23rd row at the 23rd digit because I constructed it that way. This number is designed explicitly to be different from each of the numbers that's on the list.
So it can't be on the list because it's been constructed to be different from each of them in at least one place. So I think this is a beautiful argument, and it shows that no matter how you try to come up with a one-to-one correspondence, you're going to fail. You might say, oh, like, just one number? I did so much hard work. I'm missing just one number. Can't I get partial credit and put this one on the list now I fixed it? No.
Obviously, there are many, many numbers that are missing. If you put this one on the list, I was going to construct another number that was missing. So there's just not no possibility of fixing this. And in fact, the real numbers are uncountable-- cannot be matched up with the natural numbers. And there's nothing. It doesn't even come close.
So here, a summary of what I just said. F is not a one-to-one correspondence no matter what you try to do. You can't come up with a one-to-one correspondence. That's the contradiction. So that proves that R is uncountable. And that's why, by the way, we call this a diagonalization because we're going down this diagonal here that's where the term-- that's where the name is come from.
OK, so I'm happy to-- [CHUCKLES] OK, that's a-- somebody's asking me-- [CHUCKLES] I'm actually-- I have a-- is it here? I can't remember if it's here. I have a check-in coming about this. And somebody is anticipating that by asking the actually rather famous question about there being a possible infinity in between the natural numbers and the real numbers. We know the real numbers now are bigger than the natural numbers. But is there something that's an in between? I mean, this is a very, very famous problem, which I'll talk about when we get to our check-in, which is coming up.
Somebody is objecting just the way I've defined x using this procedure that really can't, in a sense, state, you know-- but x is a number. X is the result of what this procedure is. Following this procedure, we're converging on some particular value. And so that is a value. If you want, we can make a more precise way of determining. We don't have the flexibility of choosing the way I did in my example. We can make a precise procedure for coming up with these digits.
But I don't think there's anybody thinks there's anything that's-- there's any shortcoming in this argument in terms of the way we're defining x. I think it's worth understanding this because it's really going to set the groundwork for our proof that A,TM is undecidable, which is a little, I think, perhaps, slightly more abstract in a way in the way it sort of comes across. I'm going to try to connect the two. But I think it's helpful to understand at least this argument because this argument is diagonalization in its most raw form.
All right, I think we're good. So why don't we continue, then? So there are a number of corollaries that follow from the statement that the real numbers is uncountable. First of all, if you let script L be the collection of all languages, if you want to consider it over some particular alphabet, that's fine. But that's not going to be really important for this point that I'm going to make. So script L is the collection of all languages. So every subset of sigma star-- every subset of sigma star is a language. So look at all those possible subsets. So that includes 0 to the k, 1 to the k, plus every other language you can ever think of and more-- all possible languages.
So the collection of all languages is uncountable. There's uncountably many different languages out there. I don't want to belabor this point. You can just take this if you don't quite follow the quick argument I'm going to make here. But you can make a one-to-one correspondence from all languages to the reals so that each language gets its own real number. And the way I'm going to think about that-- let's put the real numbers into binary form.
And if you imagine here being sigma star, all of the possible strings of sigma star written out in their standard order. And now if you have a language here, A, it's just some arbitrary language. So that's going to have some of the strings of sigma star appearing. Like 0 is appearing, but 1 is not appearing. 00 is appearing and 01, but not these three. And I'm going to associate to A, my language, some real number in binary by putting in a 0 in the decimal representation-- well, the binary representation, I should say-- for that string if it's not there and a 1 if it is there.
And so a real number-- because there's infinitely many yes/no choices in the binary representation can represent a language because of each of the yes/no choices of a string being present or not. I'm going to put a 1 for when the string is present, a 0 when it's not present. So each language has its own real number, and each real number is going to be associated to its own language. Here, I'm restricting myself to real numbers between 0 and 1. That's not going to be an essential point. So let's not get hung up on that.
So the fact that the languages can be putting it into a one-to-one correspondence with the real numbers shows that the collection of all languages is also uncountable. Now another observation here-- another point worth noting is that if you look at sigma star itself, the strings, all possible strings, that's a countable set. The collection of all possible languages is uncountable. But the collection of all possible finite strings is countable because I can just list them. Here's my list of all possible strings, which you can put into a table if you like to think of it matching up with the natural numbers in that way.
Now I'm trying to make a point here, which is that if you take M, which is all possible Turing machines-- script M is all possible Turing machines-- the collection of all possible Turing machines is countable. There are only countably many different Turing machines. And you can argue that in all sorts of messy different ways. But I think the most simple way to see that is to think about each Turing machine as having a description, which is a string. So the collection of all descriptions of Turing machines is just a subset of sigma star, which we already know is countable. And the subset of any countable set is going to be countable.
So anyway, I think it's worth remembering that the collection of old Turing machines is countable. Whereas the collection of all languages is uncountable. And that tells you right there that some language is not decided because there are more languages than Turing machines. We've unaccountably many language, only countably many Turing machines, so that's fewer Turing machines than languages. There's no way to map all the languages onto Turing machines. So there's going to always be some languages that got unmapped. And so, in particular, there are going to be some languages which are undecidable. There are going to be some languages which are not Turing-recognizable. And anything based on some automata kind of a definition process is going to be some languages that they're not going to be defined.
OK, now what this corollary shows you that there is some language out there, which is not decidable. What we're going to show next is that there is a specific language-- A,TM-- which is not decidable. And but first, I think we have a check-in coming up. And let me give you a little bit of background here because this is relevant to this question that I got about intermediate size sets. So the question of whether there is a set of size between the natural numbers and the real numbers strictly in between-- so bigger than the natural numbers, not the same size as the natural numbers, but not the same size as the real numbers either, but in between in size. That was Hilbert's question number 1 out of his list of 23 that I talked about a few lectures back.
It's interesting that he put it as number 1 in that very privileged special place because I know Hilbert was very-- he felt that the understanding infinity was a really central issue in mathematics. And that if we can't answer a question like this, we don't really understand infinity. You want to understand what kind of sizes of infinities are there. We know there's the real number is bigger than the natural numbers. Is there something in between? So fundamental, really.
But it was shown during the course of the 20th century, really, in two separate steps-- one in the 1930s by Godel, one in the early 1970s by Cohen-- that we can't answer this question by using the standard axioms of mathematics. The answer can go either way. And both of them are consistent with the axes mathematics. So you're never going to be able to prove that there is a set whose size is in between or that there is no such set. It's just impossible to prove either way using the standard axiom of mathematics, which, actually, is kind of remarkable.
And so my question for you is-- and this is really a philosophical question, not one that is directly you need to know about material in the course. I think it's just a matter of your own interest. I hope you find it interesting. If you don't, you can just answer it randomly. But what's going on here that we can't answer that question about whether there is a set of intermediate size? Is it because our axioms for mathematics are inadequate?
Or maybe there is no such thing as a mathematical reality. You can talk about what's real here? What's the reality? Either there is a set in between or not. If you can imagine, all of these things have their own reality to them, well, then, there's going to be an answer. And then you would expect, well, maybe we can find better axioms, which will actually give us that answer. Or you can say, well, there is no reality.
And infinite sets are kind of human constructs anyway, so we can make them kind of play out any way we like. Mathematicians argue about that to this day. And it is, as I say, really, it's a philosophical question. But just out of curiosity, let's see how you guys end up deciding on that one.
So here is the poll. 5 seconds to go. Please vote. And we're going to end the polling, 1, 2, 3, now. All right, here we go. So there's no right answer. I think if most mathematicians were to, I think, the instinct of most logicians, especially, I'm not sure if general mathematicians really even care about this question, but logicians would probably have an instinct that, probably, there are sets in between. There's no reason that there shouldn't be. It seems kind of strange that there should be this sort of jump from the natural numbers to the real numbers and why nothing in between? But I don't think that question is really settled.
All right, let's continue on. I think we are-- OK, so we're going to-- our coffee break is coming, in case you're wondering. So this is my last slide before then. But this is an important slide, so please hold out. So here is our promised theorem of the day. I'm going to show that A,TM is not decidable, the acceptance problem for Turing machines.
And it's all going to be contained on this one slide. We're going to give a proof by contradiction using diagonalization. And we're going to assume some Turing machine, H, decides A,TM. And we're going to get a contradiction. So let's, first of all, make sure we understand what H is doing.
So H gets an input-- a Turing machine and an input. And H is going to be a decider, so it always halts with an accept or a reject. It's supposed to accept if M accepts w and reject if it doesn't. So in other words, it's going to reject if M rejects w either by halting or by looping. That's the job of H. And we're assuming we can do that.
But we will see a contradiction occur. So the way we're going to do that is really kind of just one step here in a way. And we're going to use H to construct another Turing machine D. H is going to be a subroutine to D. We've already seen us doing that in the past.
D is going to do something a little strange, just to warn you. D's input is just a Turing machine-- no w. And what D is going to do using its subroutine H is going to simulate H on input M, comma, the description of M. Now what is that?
Well, the description of M is just some string. So what H is trying-- what it's asking H to tell, to answer is, does M accept the string representing M's own description? It's as if we're feeding M into itself, which seems like a totally twisted thing to do, you might say. Why would you ever feed a program into itself? Somebody has written cannibalism here-- yeah, kind of. I'd say it's worse [LAUGHS] because it's not eating somebody else. It's eating yourself.
OK, but I claim that there are actual cases in practice where we do this. We feed programs into themselves. And the example that I know of where this is done is when you're making a compiler. You might want to make a compiler and then written in the same language that it's compiling. And then you feed the compiler into itself.
You may say, why even bother because it's already, obviously, if it, once it's running, you don't need to compile it again. But actually, an example where this was really used was when there was an optimizing compiler, I think, for C written on early Unix machines. And the optimizing compiler for C was written in C. So you would feed the optimizing compiler into the regular compiler, first of all. Now, you had the compiler running, but it was an optimized.
So but now that it's the optimizing compiler is running, you can feed the optimizing compiler into that, which is itself. Now you have an optimized optimizing compiler. So it really makes some-- there is at least one case where this has actually been done in practice. Not that we really care. This is a theory class-- but just for fun to observe that.
So here, H is trying to say, well, does in a M end up accepting when it's fed the description of itself? You know, at least, mathematically speaking, that's a reasonable thing to ask. And then what D is going to do when it gets the answer back from H-- H is a decider, don't forget-- is D's going to do the opposite of whatever H does. It's going to accept if H rejects and reject if H accepts.
So let's, in summary, let's pull this together, so it's easy to understand, in the end, what is D doing? D is going to accept. D is also going to be decider, by the way. So D is always going to either accept or reject-- Just. The opposite of what H tells it to do. So D is going to accept M exactly when M doesn't accept M because when M doesn't accept M, H is going to reject, and so then D is going to accept.
So D accepts M if and only if M doesn't accept M. That's exactly the condition in which D accepts M. I think it's important to just step back and make sure you understand this line because we have only one line left to get our contradiction. Right? Are we together? D accepts M if and only if M doesn't accept M. That's just by the way we've defined setup D.
Now what we're going to do is feed in, instead of M to D, and not some arbitrary feed, we're going to use feed in D into D. And that is going to be our contradiction because D is now going to accept D if and only if D doesn't accept D, and that's certainly impossible. That's our contradiction which proves that H cannot exist. And therefore, A,TM is undecidable.
OK, so we're done, except for the one point, which is that why is this a diagonalization? And I think you can get that from the following picture. If you imagine here writing down all possible times-- I'm going to make a table here. Here is the list of all Turing machines, including D, which is a machine which I built under the assumption that H exists. So D appears here somewhere. But here are all the other Turing machines. And here are all of these descriptions of the Turing machines along the labeling all of the columns.
OK, so these are the rows labeled. These are the column labels. And inside, I'm going to tell you the answer for whether a given machine accepts a given input. So for example, M1 accepts its own description but rejects the description of M2, but accepts the description of M3. I don't you know. I'm obviously I'm making this up. I don't know what M1 is. But just hypothetically, that's what the machine M1 does.
So I'm just filling out this table. H could get the answer to any of the elements in this table because it can test whether M4 accepts the description of M3. So H could fill out this table. So maybe M2 is a machine that always rejects. It's a very unfriendly, rejecting machine. M3 is a very friendly machine. It accepts all inputs. M4 rejects some and accepts others, dot, dot, dot.
Now, I want to look and see, what does D do? Now based on the information that I've already given you, we can understand what D does. So for example, what does D do when I feed it the description of M1? What does D do?
Well, we can look here. D accepts M if and only if M doesn't accept M. So D is going to accept M1 if and only if M1 does not accept M1. Well, M1 does accept M1. So D does the opposite. D rejects. So OK, I'm highlighting here. D rejects because D is going to do the opposite of what the machine does on its own it on input. So D on M2, you have to look what M2 does on M2. It rejects, so D does the opposite of that. It accepts.
And similarly, each one of these things-- D's answer is going to be the opposite of what the machine does on its own description, just by virtue of the definition of D. OK, and so on-- so far, so good. But the problem is, what happens when D is fed itself? Because, as you can see, we're heading for trouble because this is a diagonal down here. D is just one of the rows. That diagonal is going to intersect that row at this point. And D is defined to be going to be doing the opposite of what that element is, but it can't be the opposite of itself. And so that's the contradiction.
So I think we're-- I'm going to call us, give us a little break here. And then you can also text me in the meantime. I'll be happy to answer some questions during that. A little over 4 minutes to go-- so let's see. Let me see if I can answer your questions.
OK, what's so special about A,TM that enables us to do this? Why can't we do this on ADFA, for example? That's a good question. And the answer is that, in a sense, we can do this on DFA. I mean, I think this is, perhaps, a bit of a stretch. But DFAs could not answer ADFA. I mean, we could prove that in other ways as well by just-- we could use things like the pumping lemma, and it's not clear, even how you'd formulate ADFA.
But what's important here is that it's really the model talking about itself that really is where the problem comes up. So if you try to push this argument through to show that ADFA is not decidable by Turing machines, you're going to fail because we're starting off with a Turing machine. And I think I'm going to confuse myself if I try to just repeat it. But you won't get a contradiction because the Turing machine is not a finite automaton. OK.
Will this argument get into self-loops? I don't see why it would-- there is some self-reference, perhaps. We're going to talk about that a little later. So we're going to come back and revisit this argument in a week or so when we talk about the recursion theorem which talks about machines that can refer to themselves. But this is a little bit of a head-- getting ahead of ourselves.
So somebody's commenting on this reminding them of the barber paradox, if you remember that, which has some similarity. There is a town in which there was a barber which shaves every man who doesn't shave themselves. It seems he's a very good barber. So there are some people who shave themselves. And all the rest, the barber shaves. The question is, does the barber shave himself? Because he shaves only those men who don't shave themselves. So you've got a same kind of a contradiction. There is a relationship there.
So someone wants to know, where did we use the decidability? So we used the decidability to come up with H. Once we know that A,TM is decidable, then we have that H function, and then we can build D. So that's the chain of reason. So you assume A,TM is decidable. Then you have the decider called H, and then you can build D. Somebody wants to see the previous slide. What part of the slide do you want? So I'll leave that up there.
Why can we apply the proof that all Turing machines are accountable to all languages? Well, because Turing machines have descriptions. General languages don't have descriptions. And so that's why. OK, the candle has burnt to the bottom, and it's time to move on.
So now, let's look at the acceptance problem for queue automata. I'll give you a queue automaton on input, and I want to know, does it accept the input? Is that going to be decidable? And you have your choices. It's either yes, it is decidable because these are similar to pushdown automata and APDA is decidable, or no because yes contradicts results that we know at this point, or we don't have enough information to answer the question. OK, let's put that up. One second-- [LAUGHS] all right, that's it. Ready, going, gone.
So yes, the answer is, well, no. [LAUGHS] The answer is, indeed, the answer is B. True, that queue automata are similar to pushdown automata, but all these automata are similar to each other, and that's not going to be good enough.
What the homework has asked you to do is to show that you can simulate Turing machines with queue automata. So if you can answer the question about whether the queue automata will accept their input, that would allow you to be able to answer questions about whether Turing machines accept their input. And we just proved that's not possible. So it would be a contradiction if we could answer-- if we could decide A, queue, A.
Now, we have an example of an undecidable language. Let's look at an example of an unrecognizable language. Now A,TM is not going to serve that purpose because A,TM is Turing-recognizable, as we pointed out by the universal Turing machine. So A,TM is undecidable, however. How about an unrecognizable language?
For that, we will see that the complement of A,TM will serve. So the complement of A,TM is neither decidable nor even recognizable. Now it's not Turing-recognizable. And that's going to follow from a pretty basic theorem that connects recognizability and decidability that I've put up here on the screen, which is that if you have a language where it and its complement are both recognizable, then the language turnout turns out to be decidable. In fact, a language and its complement are decidable. But being decidable is closed under complement, so that's something you should be aware. But being-- OK. We'll get to that in a minute. But if-- so anyway, so if you have a language and it's complement both recognizable, how do we know the language is decidable?
So first of all, let's take the two Turing machines, M1 and M2 that recognizes A and A-complement. And we're going to put those together to get a decider for A. And that's going to work like this. It's going to be called T. So T says, on input w, what it's going to do, it's going to feed w into M1 and M2 both. A is the language, by the way, yes. A is-- when I say it's Turing-recognizable, you know, Turing-recognizable only applies to languages. So yes, A is often this symbol I'm going to use for languages, sometimes for an automaton. But A is typically going to be a language.
So now, I'm trying to make T be a decider for A from the recognizers for A and A-complement. So I'm going to take an input to T and feed it into both recognizers, M1 and M2. OK, I'm going to run them in parallel. What's nice is that because M2 recognizes the complement of what M1 recognizes, every string is going to be accepted either by M1 or by M2 because every string is either an A or an A-complement. So if I run M1 and M2 on w until one of the halts or one of them accepts, I know I'm not going to run forever because, eventually, one or the other one have to accept.
So and then I got my answer because if M1 accepts, then I know I'm in the language. But if M2 accepts, I know I'm in the complement of the language, so I'm out of the language. So if M1 accepts, then T should accept. But if M2 accepts, then T should reject. , So that proves that nice little theorem written at the top in blue. So I got my decider for A built out of the recognizers for A and A-complement.
Now immediately, it follows that the complement of A,TM is not Turing-recognizable because we know that A,TM itself is recognizable, but it's undecidable. If the complement was also recognizable, then A,TM would be decidable, but it isn't. So when something is decidable, either it or its complement have to be unrecognizable. And in the case for A,TM, it has to be the complement because we already showed that it is itself is recognizable. So that's the proof of that.
So here is a little picture of the way the world looks right now if you have here, in the middle are the decidable languages-- so these are all languages, this Venn diagram of languages. We showed earlier, the regular, the context-free, decidable, recognizable. Here, I've got the recognizable and what I'm calling the co-Turing recognizable. This is the collection of all complements of recognizable languages.
So A,TM bar, A,TM complement is the complement of a recognizable language. This region here are all the complements of the recognizable languages or the so-called co-Turing recognizable languages-- complement of. So A,TM is on this side. A,TM-complement is on that side. But if something's in both, by virtue of this theorem here, then it's decidable.
OK, last check-in for the day. From what we learned so far, which closure properties can we prove for the class of deterring recognizable languages? Choose all that apply. Well, as I say, you don't have to get it right. Let's not spend too much more time on this because we'll talk a little bit about it. Almost all-- its closed under almost all of them, but not all of them. Because-- are we done here? I think we're done-- 5 seconds. OK, here we go, ending polling.
I'm not sure what the meaning of [LAUGHS] deleting your answer is here. Everybody likes union, I guess. They're closed under all of these operations except complement. But we just proved it's not closed under complement, so I'm a little puzzled by why we have so many votes for closure under complement. We have here, A,TM is Turing-recognizable, but A,TM-complement is not Turing-recognizable. it's right here on the slide. I'm not trying to make you feel bad, but I'm trying to just point out that you think, please.
So now closure under union and intersection, I mean, you could kind of get those answers just by running things in parallel the way we did the proof here. You just run both machines. And if they both give-- I mean, it's a little tricky, I suppose. If either one of them accept, then you can accept. Or if they both accept, you just wait until they both have accepted. Otherwise, you just keep running. So the first two are pretty straightforward.
Closure under concatenation-- this is also going to be similar. You just try every possible way of cutting the string up into two pieces and run in parallel on. And if you ever find a way of cutting it up, and you run those two, and put those two sides in parallel, and if they both accept, then you can accept. And star is, again, very similar. So these are not too bad. But I admit, you know, it's not a whole lot of time to have to contend with something that you're just getting used to.
So let's talk about the very last topic of the day, which is really going to be setting ourselves up for Tuesday's lecture next week. And that's how we are going to be showing other languages are undecidable, which is something that I'm going to be expecting you guys to be able to do. This is the standard procedure for showing languages are undecidable using what's called the reducibility method.
And what that does is it takes, as a starting point, a language that we already know is undecidable-- typically, A,TM-- or it could be another one that you've previously shown to be undecidable-- and leverages that information to show other languages are undecidable. And it's using what's called reducibility. We're going to go into this more carefully next time. But basically, reducibility is a way of using one problem to solve another problem.
And so we are going to show, for example, let's take a look at the problem called the halting problem, which is like the famous problem for Turing machines. You just want to know whether it halts, not necessarily whether it accepts. So it's very similar, but not exactly the same.
And we're going to show that this halting problem is similarly undecidable. Now we could go back and do the whole diagonalization, but that seems like-- well, that's more work than necessary now that we already know A,TM is undecidable because we're going to show that we can reduce the A,TM problem to the halting problem. And we'll explain what that means again later. But the idea is-- and as we'll show in an illustration shortly-- that by proving by contradiction if the if HALT,TM were decidable, then A,TM would be decided. And we know A,TM is not decidable. And so that's our contradiction.
Now the way we're going to show that if HALT,TM is decidable, then A,TM is decidable is use a decider for HALT,TM to decide A,TM with a suitable modification. So basically, we want to turn a HALT,TM decider into an A,TM decider. And that's how we're going to reduce the problem of solving A,TM to the problem of solving HALT,TM.
Let's just do an example. If you've seen it before, obviously, this is not going to be hard. But for the many of you who have not seen it before, I'm partly doing it this time just so we can do it again next time. And maybe it'll sink in by virtue of repetition. So again, so as I just said, we're going to assume the HALT,TM problem is decidable and use that to show that A,TM is decidable, which we know is false. We showed it just earlier that it's not.
So assume we have a decider for HALT,TM. We'll call it R. And we're going to construct from R a decider for A,TM we'll call S. OK? So we're, again, typical proof by contradiction. We're assuming the opposite of what we're trying to prove. And then we're going to get something crazy.
OK, so here, my job now, is I'm assuming I have R, which is a HALT,TM decider. So now, I'm assuming I know how to decide if a Turing machine and an input eventually halts-- Not. Necessarily would it accept, just whether it halts. It's conceivable. You have to bear with me here.
It's conceivable that you could find a way to test whether Turing machines halt on their input, even though we now know that testing whether they accept their input is not decidable. So you have to be open-minded to the possibility that the HALT problem is decidable, and we're going to show that that's can't be. So we're going to show that if we could decide the halting problem, then we can use that to decide the acceptance problem. OK, so how are we going to do that? So imagine how we can solve the halting problem.
So to solve the A,TM, which is what my job is to do, so S is supposed to solve A,TM. I'm constructing a Turing machine as decide A,TM. I'm going to use first-- I'm giving it M and w. I'm going to feed it into R since that's really all I got. See if R tells me what happens. Does M on w at least halt? Well, if R says, no, it doesn't halt, well then I'm actually done because if M doesn't even halt on w, then it couldn't be accepting w. So at that point, I know that M doesn't accept w, and I can reject right off.
So R, you can see how it could potentially be helpful. But it's going to be helpful in either way because if R says M does hold, well, then, I'm also good because I don't know the answer yet, but what I do know is I can now simulate M on w until it halts because R has told me it halts. So I don't have to worry about getting into a loop. So S can be confident in being a decider for whatever it's doing because I'm running now M on w with a guarantee that it halts.
And now, that's going to tell me-- now eventually, the simulation of M on w is going to end up at an accept or reject. And that's going to be the answer I need. So if M is accepted, then accept. And if M is rejected, then reject. And that's how S solves A,TM using R, which solves HALT,TM. But S can't exist. And so therefore, R can't exist. And therefore, HALT,TM can't be decidable. OK?
So that quickly-- OK, I'm not sure which diagram you wanted me to show. But anyway, maybe we can do that. We're basically at the end of the hour or end of the 90 minutes. So let's do a quick review. And if you stick around, I'm happy to go back and look at any of the other slides that you might have missed something on.
OK, so just to recap, we showed that the natural numbers and the real numbers are not the same size using that definition of one-to-one correspondence to introduce the diagonalization method. We used the diagonalization method to show that A,TM is undecidable. We also showed that little theorem that if the language and its complement are recognizable, then the language is decidable. And from that, we concluded that A,TM complement is not recognizable. And then we showed, at least by virtue of an introduction to the method, the reducibility method to show that HALT,TM is undecidable.
And that was today's lecture. And we're and we're at the end of the hour. So why don't I-- we are finished. You can log out. And if you want, I will stick around.
OK, OK, this is kind of a good question here. So I'm getting a question about the A,TM-complement, which is-- since we have a recognizer for A,TM, if I'm doing justice to this question, we have a recogniser for A,TM, so why can't we just invert the answer? Flip the answer around, and now, we have a recognizer for the complement of A,TM, A,TM-complement. So why doesn't that work?
Well, the reason that doesn't work is because the recognizer for A,TM might be rejecting some things by looping. And now, if you just flip the accepting and rejecting, when it hits one of those halting states, it's going to give the reverse answer. But when it rejects by looping, it'll continue to reject by looping. So you won't get the complementary language coming out.
So if it would be helpful, I can go back to that slide here, which proves that A,TM-complement is unrecognizable because maybe we should start with the bottom. We know that A,TM is recognizable and undecidable, right? We already proved those two facts. A,TM is recognizable from the universal Turing machine, and it's undecidable by the diagonalization argument. Those two things together tell us that the complement has to be unrecognizable because if a language and its complement are both recognizable-- and we already know the language itself is recognizable. So now, if the complement is also recognizable, the language is going to be decidable by the upper theorem.
So it must be the case had either the language itself is unrecognizable, or its complement is unrecognizable. We know the language is recognizable. That's what the universal Turing machine told us. So the only thing left is for the complement to be unrecognizable. You should review that if you didn't get it because this is the kind of reasoning we're going to be building on things like that. So I think it's good to make sure you understand. OK.
OK, the diagram on the right-- so this is just a Venn diagram here. I threw this in at the last minute here. I was worried about it being confusing. That part is-- I'm trying to show that the three classes that we've already talked about-- the languages which are decidable, the languages which are Turing-recognizable, and the languages whose complements are Turing-recognizable. Those are three separate classes of languages. And those come up here in those three regions. These are the decidable ones. Here are the recognizable ones. And here are the ones whose complements are recognizable.
Now if a language is in both the recognizable, and its complement is recognizable-- so it's in both of these bigger regions here-- then this theorem tells you it's decidable. So that's why the intersection of these two regions is marked as being decidable because that means you're in both. OK? But we know that A,TM is sitting out here as recognizable but not decidable.
So A,TM is in the recognizable side, but it's not on the complement of recognizable, A,TM itself. The complement of A,TM is the complement of a recognizable but itself is not recognizable and not decidable. So you got this side of nice, you know-- I hope you think it's nice, but it's sort of a try to summarize things in this little Venn diagram.
So I think I'm going to then sign off. And I'll see you all on Tuesday. And have a good weekend. Bye bye.