Flash and JavaScript are required for this feature.
Download the video from Internet Archive.
Description: Quickly reviewed last lecture. Covered NP-completeness; \(SAT\) and \(3SAT\); \(3SAT\) ≤\(_P\) \(HAMPATH\); and \(3SAT\) ≤\(_P\) \(CLIQUE\). Discussed a strategy for proving NP-completeness with a reduction from \(3SAT\) by constructing gadgets that simulate variables and clauses.
Instructor: Prof. Michael Sipser

Lecture 15: NP-Completeness
[SQEAKING]
[RUSTLING]
[CLICKING]
MICHAEL SIPSER: Welcome, everyone, back to theory of computation. So we are now at lecture number 15. And this is an important lecture. Well, all of our lectures are important, but this is going to introduce one of the major topics that we're going to see going on in various forms through the rest of the semester, which is the notion of a complete problem. In this case, it's going to be an NP complete problem.
And I think some of you have probably heard of that concept already. Maybe you've seen it in some courses or other, but we're going to do that in a rather careful and formal way over the next couple of lectures. So we are following up on our previous discussions about complexity, time complexity. We defined the time and not deterministic time complexity classes, as you may remember, the classes P and NP, talked about the P versus NP problem, looked at some interesting algorithms for showing problems in P called dynamic programming.
And we started to move toward our discussion of NP completeness with the introduction of polynomial time reducibility, which is related to some of the earlier reducibility notions that we discussed in the computability section. Quick review-- what it means for one problem to be polynomial time reducible to another-- this follows our pattern of reducibility concepts, where if a problem is reducible to another problem, and that other problem is solvable, then the first problem is solvable. So if a problem is reducible to an easy problem, that problem becomes [INAUDIBLE].
And the kind of reduction that we're going to be looking at here are the mapping reductions, but now where the reductions are computable in polynomial time. And so we had this result we mentioned last time, that, if A is polynomial time reducible to B, and B is in P, then A is also in P. So that's going to be critically important for the whole of discussion that's coming.
Our intuition about P and NP, repeating that here-- the NP problems are the ones where you can verify membership easily, as opposed to being able to test membership easily. Those are the problems that are in P. The verification typically requires some kind of a certificate that establishes a proof that the input is a member of that language.
And the big question of the field, which remains an unsolved problem, is whether P equals NP. It's called the P versus NP question. And we don't know the answer-- so whether there are problems that are in NP solvable in non-deterministic polynomial time-- typically these are problems that involve searching-- whether they can be solved in polynomial time, typically without the searching.
And if P were equal to NP, then you could always eliminate searching. And if P were different from NP, then there were cases where you need to search. And we don't know the answer to that. So in the direction of exploring this question and its ramifications, we introduced this problem called SAT. These are the Boolean formulas, which have an assignment that makes them evaluate to true. So we call those satisfiable formulas.
And we mentioned, but have not yet proven-- and we will not prove until the next lecture on Tuesday-- that there is this theorem, a very remarkable theorem that says that, if you have-- take the satisfiability problem, and if it is solvable quickly, then all of the NP problems are solvable quickly. So if SAT is in P, then P equals NP.
So in a sense, SAT is kind of no the-- it's sort of the super NP problem, in the sense that all of the difficulty of any NP problem is embedded within SAT. So if SAT becomes easy, then all of the other NP problems become easy. And we'll eventually prove that, but right now we're setting up the terminology to allow us to do that. So anyway, we'll get there.
So the key ingredient for proving this Cook-Levin theorem is polynomial time reducibility. What we're going to show is that every problem in NP can be polynomial time reduced to SAT. So every NP problem can be converted into a SAT problem. And so working toward that-- because when you think about it, there are infinitely many primes in NP, and being able to show that all of them are reducible to SAT is, in a sense, kind of-- you have to prove a higher level theorem.
It's not just a single reduction. We're exhibiting a schema of reductions, which shows that all of these problems can be reduced to SAT. And developing our intuition toward that direction, we're going to look at some specific polynomial time reductions today. And we'll start out with a reduction between two problems we have not yet seen-- one of them called 3SAT and the other one called clique. So on this slide, we're going to introduce those two problems.
So remembering, again, about Boolean formulas, we're going to consider a special class of Boolean formulas, restricted form of Boolean form formulas called conjunctive normal form. And so this formula here in particular is in that conjunctive normal form. So just remember-- I was explaining some of this to someone else earlier this morning, showing some of these slides, and it was pointed out that not all of you may be familiar with these the Boolean operations of and and or.
So this is the or symbol here. This is the and symbol here. Hopefully you've seen just the concept of Booelan and and or. Or is, in a sense, a little bit like a union. And it's like an intersection. And the symbols here are a little bit similar to the union and the intersection-- union and intersection symbols themselves. The V shape is a little bit like a pointy union symbol an the upside down V shape is a little like a pointy intersection symbol. If you haven't seen those-- seen that connection before, maybe it's interesting to observe that.
But anyway, what makes this formula be in conjunctive normal form? Well, conjunctive normal formulas are organized in a certain way. They have these groups called clauses within the parentheses that are anded together, that are connected by these and operations. And within those groups, which are called clauses, the elements are ORed together. Those elements are going to be either variables or variables with negation, negated variables.
So just to repeat that, these-- well, just to state, those variables or negated variables are going to be called literals, and the ORs of a bunch of literals are going to be called clauses. This is just the standard terminology in the field. OK? So a literal is a variable or a negated variable and a clause is an or of literals-- just a definition. All right, so we have a bunch of these clauses here. Each of the clauses has an or of literals in it.
Now, a Chomsky-- Chomsky-- conjunctive normal form formula-- I guess they're both CNFs-- but conjunctive normal form formula is one that's written as an and of these clauses. So we take these clauses, which themselves are ORs-- we and them together, we get a formula that's conjunctive normal form. Conjunctive stands for and, I think. Conjunction is an and.
And then a 3CNF is a CNF where each clause has exactly three literals. So for example, this particular formula is a CNF, but it's not a 3CNF, because it has at least two clauses which violate the condition of three literals per clause. This one is OK, obviously-- this first clause. And a 3SAT problem-- so this is the first of the two problems we're going to be talking about-- the 3SAT problem is the satisfiability problem restricted to these 3CNF formulas.
So this is the collection of-- set of 3CNF formulas which are satisfiable. So you can think of it as kind of a special case of the SAT problem, where we only care about formulas of the special kind. This being a special case, you might imagine that this might be an easier problem to solve, but in general, it turns out not to be, as we will see. Solving this special case is just as hard as hard as solving the general case for satisfiability.
Now let's turn to the second of these two languages up in the headline, the clique problem. So for that we-- going to turn to graph theory. So we're going to consider graphs, points and lines-- connecting them. And we will say that a clique in a graph is a collection of nodes-- collection of the points that are all pairwise connected by lines. And a k-clique is one where you have k such nodes.
So here we have a three clique, four clique, and a five clique. And then the clique problem is to try to find cliques of a certain specified size embedded within a given graph. You're going to be given a graph, and it targets clique sized k, and you want to know, is there a subset of the nodes in the graph of size k that are all connected to one another? That's the clique problem.
So obviously, the clique problem, just as with the satisfiability problem, is a decidable problem. You can just try every possible subset of k nodes and see whether it constitutes a clique. But that's, in general, going to be an exponential algorithm. If you have-- k is a large value-- it might be half the size of the graph-- you might be looking for a very large clique.
Then you have to try many, many subsets in order to see whether any one of them is a clique. It's also going to be and-- I hope you get this intuition-- this is a problem that's in NP. The clique problem is an NP problem, because you can easily verify that a graph has a k-clique just by exhibiting the clique. OK.
Now, so the language here is you've given a graph and a k, and you want to know, does the graph contain a k-clique? And what we're going to show is that these two problems are connected, that the 3SAT problem on the clique problem are related, in that you can reduce in polynomial time 3SAT to clique.
Just standing back at it-- back for a minute, it seems kind of surprising. There's no real reason-- no obvious reason why there should be a connection between three 3SAT and clique. They look very different from one another. But we do give that-- we will give that reduction.
That implies that, if you can find a way of solving the clique problem quickly, that'll give you a way of solving the 3SAT problem quickly. And that's going to be the whole point of this. So we're going to show this over the next slide or two, this polynomial time reduction. I'm going to walk through it slowly, but this is one where it's really important to try to get a sense of how it's working, because this is the kind of thing that we're going to be doing a lot of, and you're going to be asked to do it also on the homework and possibly on the final exam.
I hate to use that as the motivating force here, but at least it might be one motivation for some of you, that you have to understand how to do this kind of a reduction. I think, at a high level, what's going on is we're showing how to recode a Boolean formula satisfiability problem into the problem of testing whether a graph has a clique. Oh, let's see if we have any questions here that are coming up in the chat.
OK, so this is an interesting question here. Can we always convert a Boolean formula into conjunctive normal form, first of all? Yes. The answer is you can always convert a Boolean formula into an equivalent one in CNF. But in general, that might make the formula exponentially larger.
So just the mere fact that you can convert formulas into conjunctive normal form doesn't mean that solving conjunctive normal form formulas for testing satisfiability of the CNF formulas is going to be as hard as testing the general case, because just the conversion might be exponential. There's something more-- little bit more complicated going on than that. Let me just see here.
If phi is a satisfiable with formula which is not in CNF, can be-- so a similar question-- so the questions that I'm getting are about converting formulas to CNF. So yeah, you can do it, but not in polynomial time, in general, because the resulting formula you get might be much larger-- if you're looking for an equivalent formula. If you're not looking for an equivalent formula, then-- of course, then, depending upon what you're looking for, you might be able to find something smaller.
This is, I guess, a good basic question. Why is clique in NP? Doesn't verifying that you have a clique require going through all the possible cliques? You have to understand what verifying means. Verifying means you can verify something if you're given a certificate. In the case of a clique-- the clique problem-- the certificate is the clique. So once you have the certificate, you can do the verification in polynomial time.
Finding the certificate, of course, might be difficult. So you only think of NP in the context of having that certificate. So in a case for compositeness, the certificate might be the factor. There are problems sometimes where what the certificate is is not necessarily obvious. But there can be a certificate for showing that the input is in the language. In the ones that we've done so far, maybe you can argue that the certificate is sort of an obvious thing, but it's not always an obvious thing.
OK. Why don't we move on then? So let's see, how do we do this polynomial time reduction from 3SAT to clique? OK, here we go. So I'm just going to give a reduction. That's what the definition means. I'm going to give a way of converting formulas to pairs, a G and a k, where the formula's going to be satisfiability if and only if the graph has a k-clique.
OK, so let's a little bit do it by example. And in order to do that-- so here's going to be a formula now. It's in 3CNF. That's what I need in order to be doing this reduction. I'm converting 3CNF formulas into clique problems. We to have a little bit understand what it means when we say-- we talk about the satisfiability of a formula like this, because it's going to be helpful in doing the reduction.
Obviously, satisfiability means that the-- you can find an assignment to the variables. So you're going to set each of these variables-- A, B, and C, and so on-- to true or false, and you want to make the whole formula evaluate the true. But what does that actually mean?
It means that, because of the structure of the formula, that making this formula true corresponds to making each clause true-- because the clauses are all anded together, so the only way for the formula to be true is to make each clause true. And to make a clause true, you have to make at least one of the literals true.
So it's another way of thinking about satisfying this formula. Satisfying these [INAUDIBLE] satisfying assignment makes at least one true literal in every clause. It's really important to think about it that way, because that's what's going to be the basis for doing this reduction and all of the reductions. It's what makes 3SAT easy to think about, in terms of its satisfiability.
If you had a general satisfiability problem and you had a satisfiability both formula, there's no obvious way of seeing what the satisfying assignment looks like, but here we understand what it looks like. It has that very special form, making one true literal in every clause-- at least one true literal in every clause. So now we're going to do the reduction. So I'm going to take from this formula-- I know, for some of you, you're going to be chafing. Why am I going slowly? But I want to make sure that we're all together and understanding what the rules of the game are and what we're trying to do.
We're trying to convert this formula into a graph and a number. So right now my job is, to do this reduction, is to exhibit that graph. So I'm going to do that and two steps. First, I'm going to tell you what the nodes of the graph are. Then I'll tell you what the edges of that graph [AUDIO OUT]
Finally, I'll tell you what the number k is. That's the way this polynomial time reduction is going to work. And we have to also observe at the very end that the reduction that I'm given-- giving you, this procedure for building this graph can be done in polynomial time, but that, I think-- you'll see, once I'm done, that that's pretty obvious.
OK, so first, as I promised, the nodes-- so the nodes of this graph are going to correspond to the literals of the formula. Every literal is going to become a node in the graph, and it's going to be labeled with the name of that literal. Every node is going to be labeled an a, a b, or c bar, and so on. So here it goes. So those are the nodes of the graph G, one for each literal in the formula, labeled as promised.
Now I have to tell you what the edges look like. I'm going to tell you what the edges are by first telling you what they are not. I'm going to first explain to you what the forbidden edges are, what edges I'm going to promise not to include.
And then the ones that I will include are going to be all the others. So what are going to be the forbidden edges? First of all, the forbidden edges are going to be of two types. One is the edges between nodes that come from literals in the same clause. So I would just call that no edges within a clause.
So for example, these three nodes will not be connected to one another. And I'm going to indicate that by writing red dashed lines, which means there's not an edge there. Those are forbidden from having an edge. So these three have no edge, and the same thing for every other triple of three nodes that come from clauses. Those are no edges there.
And there's one other category of edge that I'm forbidding, and that are edges that can-- that go between inconsistent labels, and the nodes with inconsistent labels. So for example, between a and a bar-- those are inconsistent. Nothing's wrong with a going a to d. Those are not inconsistent.
Those are just different labels. Or going from a to a-- that's OK. But from a to a bar-- that's not allowed. So that's going to be another forbidden edge-- or for example, a to a bar here, a bar to a, or for example, from this c bar to c-- forbidden. OK? So you imagine, you're going to write down all of those forbidden edges.
Those are all of the forbidden edges, and then, after taking those away, you're going to be putting in all the other edges possible. So for example-- let me just gray those out so they don't interfere with the picture-- from a to d, there's going to be an edge, because those are not forbidden. a to a-- not forbidden, because they-- they're in different clauses and they're not inconsistent. They're consistent with one another.
So here are a whole bunch of others. I'm not showing them all. It becomes very messy. But here are all of the other edges between nodes which are-- where they're not forbidden. OK? And that's the graph. That's G. G is all those nodes and those edges which are not forbidden. And I just have to tell you what k is.
k is going to be the number of clauses. That's going to be the size of the clique I'm looking for in this graph G that I just spelled out for you. And I'm going to claim that this graph here that I just described will have a k-clique. k is the number of clauses exactly when phi was satisfied. It's kind of cool. If phi is satisfiability, then there will be a k-clique here. And if phi was not satisfiability-- no k-clique. We're going to prove that as a claim on the next slide.
OK, so k is the number of clauses. All right? Any questions on that construction? So I've done with the construction. What's left is to argue why the construction works. So far, so good-- let's move on. OK. So here is that very same construction. I eliminated the red forbidden edges. These are the ones that were remaining, plus anything else that was not forbidden.
That was the same formula that I had from the previous slide. Now I want to claim that that formula is satisfiability exactly when G has a k-clique. Now, why in the world is that? So this is an if and only if. It's proved in both directions. And this is going to be the typical kind of thing that you're going to want to do when you're exhibiting a-- one of these reductions, which is you're going to have an opportunity to do that. And we'll do that too in our examples.
OK, so now what I want to show is that, if phi is satisfiable-- so let's prove the forward direction-- if phi is satisfiable, then G has a k-clique. OK, so first of all, if phi is satisfiable, that means it has a satisfying assignment. Now, here's a common confusion. I'm not sure whether it's helpful to sprinkle the confusions alone with the discussion, but in case you're worried, you might ask, well, how do I find that satisfying assignment? I thought that was exponentially hard to do.
This is a proof. This is not an algorithm. We're just trying to argue the correctness of this construction. So there's no longer any concern about how to find an assignment. I'm just saying, if there is an assignment-- if there is a satisfying assignment, then something will happen something. Then something will happen-- in particular, then we will show that G has a k-clique.
So let's say phi is satisfiable, so it has some assignment. Let's take that satisfying assignment. And remember that, in any satisfying assignment to a formula in CNF, that makes at least one literal true in every clause. So let's just pick one of those. There might be several clauses which have multiple true literals. In that case, just pick one of them arbitrarily.
I don't have this indicated on this diagram here, but imagine maybe, in the very first clause, a was true; in the second clause, b was true; in the third clause, e complement-- e bar was true, not e-- e bar was true, which means e itself was false, but e bar was true. And that's the way each of those clauses, in turn, got to be true in this particular satisfying assignment-- because you're going to pick one true literal every clause.
And now, from that choice of literals-- one per clause-- I'm going to look at the corresponding nodes in G, and I'm going to claim that those nodes taken together form a k-clique. OK, so first of all, do they have the right number of nodes? Well, sure, because I'm picking one node per clique. I already said k is the number of cliques, so I'm getting exactly k nodes. So I have at least the right number of nodes.
But how do I know they're all connected to one another, those nodes that I just picked? Well, they're all connected to each other because I'm going to say-- well, because there were no forbidden edges among them. And remember that we put in all possible edges except for the forbidden ones. So how do I know there are no forbidden edges?
Well, what were the rules for being forbidden? It means they-- two nodes in the same clique-- in the same clause. Well, I'm picking one node per clause, so I can never be having two nodes from the same clause. So I'm never going to run into trouble with the first condition of being forbidden.
So what's the second possibility for being forbidden, is that I'm picking two nodes which are inconsistent-- well, how do I know that I didn't end up with two nodes with inconsistent labels? That would be bad, because then they would have a forbidden edge, and what-- my result would not be a clique. How do I know I didn't end up picking-- in this group I pick a, and in this group I pick a bar?
Well, because they all came from the same assignment. They were all true liberals in clauses. It's not possible that a was true in this clause and a bar is true in that clause, because if a is true here, a bar's got to be false. It can't be the true literal in this clause. So I cannot have any inconsistent nodes appearing among the nodes of my clique.
And so they're not in the same clause. They're not inconsistent, so the edge has to be there. And that's going to be true for every pair of nodes in that clique-- in that group of nodes, and that's why it's a clique. So that proves one direction. Now we still have to prove the other direction, because we have to say, well, if G has a k-clique, how do we know that phi is satisfied? So that's the reverse. So let's just take any k-clique that's in G.
And how do I know I can get from that, a satisfying assignment to the formula? Good-- getting some good questions here in the chat. But let me just move on. So proving the reverse-- the reverse direction, assuming we have a k-clique-- take any such k-clique. Now, first of all, you observe that it's got to have one node in every clause. It can't have two nodes in the same clause or zero nodes in a clause.
First of all, it can't have two nodes in the same clause. Why? Well, because those nodes are never connected to one another. So they cannot be both in the same clique, because all the nodes in the inner clique are connected to one another. By the way we constructed G, nodes from the same clause are not connected, so they cannot appear in a clique together. So there's going to be at most one node from every clause appearing in this clique, but-- appearing in this clique.
But we also have-- we know we have k nodes, so that means, if there is at most one per clause, and we only have k clauses, that means every clause has got to have one. If some clause is missing a node, then there's got to be some other clause that has two. So we know that every-- that this clique that G has exactly one node in every clause.
So how do we get from that satisfying assignment? So what we're going to do is take the corresponding literals that correspond to that one node in that was in the clique, take those liberals and set those literals to be true, which means setting the variable to be true or setting-- if the literal was a bar, then setting a to be false, because you want a bar to be true. You want to set the literals to be true.
Well, now we're setting-- there's one node in each clause-- we're setting one literal-- the corresponding literal-- true, so that's going to set one true literal in every clause. So that means it's going to be satisfying. There's one thing that one has to really be careful here to double check-- to make sure that we're doing this consistently, that we're not being asked to set a true and also a bar true, because then that would not be possible.
But a and a bar are not connected, so it's-- we're never going to be trying to set both a and a bar true. If we're going to be trying to set a true, we're never going to try to set a bar true, because those are not-- a and a bar cannot be in the same clique. They're not connected to each other. OK? So that is the argument.
And lastly, I claim-- we're not going to look at this in detail-- that it's kind of obvious that this reduction can be done in polynomial time. Namely, if I give you that formula, you can write out that graph in-- pretty easily. There's no hard work to be done in writing out that graph or counting up the number of clauses. So that's the proof that I can reduce 3SAT to clique.
And it tells you also that, if I could solve clique quickly, then I can solve 3SAT quickly. And this is the whole point, because I can convert now clique problems to 3SAT problems. No. I'm sorry. I got it backwards-- convert 3SAT problems to clique problems. So if I can solve clique easily, then I can solve 3SAT easily.
I just showed a way of converting these formulas to graphs. So that says, if I can solve the graphs easily, then I can solve the formulas easily. OK, why don't we turn to some-- oh, question, check-in. But we can now also-- I'll launch a check-in, but we'll-- let's just see. I'm seeing many, many questions here, which are actually what my check-in is about. I'll turn it back to you guys.
OK, so where did we use the fact that we have three literals per clause? Does this thing just work even if we had any number of literals per clause? What do you think? We got a tight race here. Truth is losing out, unfortunately. All right. Oh, it's really close, but still-- OK.
One more-- OK, it's neck and neck. OK, almost done-- 10 seconds-- are we done here? OK, that's it-- ending polling. Yes, truth has won out. Thank God. Yeah, it works for any size clause. We didn't use the fact that it's a 3CNF. It could have been any number here.
Well, we've got a-- I guess it's a plurality here, though, not a majority. Where did we use 3 in any of these argument? I didn't mention it. Maybe you were imagining that it's going to be part of it, but 3 does not come into this discussion at all. If you think about how it's-- why it's working, even if we had-- one of these clauses had 10 literals in it, as long as k is the number of clauses, and we don't connect any variable-- any literals internal to a clause, this whole argument is going to still work.
So please check that make. Sure you understand what's going on, because I can see that a good chunk of you have not got this right. So I got a good question here. What if it's only just one big clause-- so it's like the whole formula's just one big OR. So what does happen in that case? Good question-- so in that case-- suppose there's one big clause with 100 literals. That's the whole formula.
It's just a big OR. So we know the formula's going to be satisfiable, by the way, obviously. So if you look at the corresponding G, it's going to have 100 nodes, because there's one for every literal. None of them are going to be connected to each other, because they're all in the same clause. So it looks like we're going to be in tough shape trying to find a clique there, because there's no edges at all. Everything's forbidden.
But what is k? k is going to be 1 in that case, because there's only one clause. And kind of a degenerate case, but a clique with just one-- just a single node is-- counts as a one clique, because it's just one node. There's no need for any edges at all. It still counts as a clique. So it'll still work out. Let's see. What else here? That was kind of a fun question that you asked me. Thank you.
There are a lot of questions. I think we're actually pretty near the break also. We'll just see. Yeah. Oh, many, many questions here-- so I think I'm going to start the clock going down for our, coffee break and I will take some of these questions. I'll try to answer some of these questions-- oops-- try to answer some of these questions afterward.
All right. Good-- all right. So I got a question about making sure why nodes don't have inconsistent labels, if I understand the question correctly. So I never put edges between nodes with inconsistent labels. That's the rule. That's my construction. I get to say how the construction works. So those two nodes with inconsistent labels can never be in the same clique, because there's no edge between them.
So I'm not sure that answered the question there, but that was-- let's see. So we're getting another question here. Since any Boolean formula can be converted to CNF, does that mean-- is SAT polynomial time reducible to clique as w-- [INAUDIBLE] time reducible to clique, but not for the reason that any Boolean formula can be converted to CNF, because that conversion is going to be an exponential conversion in general-- the conversion to CNF. So you have to be careful about what we mean by converting a formula to CNF.
So also getting question-- what happens in the case of 1SAT? Well, we didn't really talk about 1SAT. So the question is, suppose there's only a single literal per clause. That's an interesting-- another sort of edge case here. What happens under those circumstances? So there's only a single literal per clause.
So you have to work out what happens. But in that case, all the clauses just have one literal in them, and now they have-- when you have a 1CNF, so there's only one literally per clause, the only way that can be satisfiable is you have to make each literal true, because there's no ORing anymore. So every literal has to be true in the assignment.
So that means you can never have any consistent literals. And that's the only case when you would not have an edge, because you're not-- the forbidden condition won't happen because you don't have more than one node per clause. Every clause is going to have just a single node in it. So it really comes down to whether or not you have an inconsistently labeled-- inconsistent clauses. I didn't explain that super well, but it's-- the argument still works, though. You should check it for yourself.
So this is a basic question. How do we see that F is polynomial time? I'm not sure I want to spend a lot of discussion on that. The conversion from the formula to the graph is kind of a one-to-one-- every literal becomes a node. The rule for when edges are present-- it's a very simple rule. I'm not sure what more to say. It seems pretty clear that the conversion-- that reduction is polynomial time computable. If you wrote a program to do that, it would operate very quickly.
Is it possible for G to have a k plus n clique, where n is greater than 0? Does that matter? If you think about it, the biggest possible clique that this graph G could have is k. It could never have more than a bigger clique, because two nodes in the same clause are never connected. You only have k clauses. So the answer is no, you cannot have a-- bigger than a k-clique-- just not going to happen.
Does each clause need to have the same number of literals? I don't see why. So question is, why are we worrying about 3SAT, since it didn't seem to matter here? There are other examples where it does matter. I'm actually not sure if we'll see one or not, but there are cases where it does matter. Let me just see if there's any quick questions here.
Somebody's saying, does-- do we have to worry about there being a polynomial number of literals? So you have to think about what that question means. The size of the input, which includes all of the literals, is n. So there's going to be, at most, n literals, because that's the size of the input, at most, in literals appearing. So you don't have to worry about that being polynomial, because-- by the way, we're defining the size of the input is going to be, at most, n literals. I hope that's clear.
OK. OK, so this is a good question. When we talk about polynomial time, it's polynomial in the representation of the input, which is-- if you want to think about it in terms of bits, that's fine-- or whatever. It's not going to matter if you use some larger alphabet. But the number of symbols you need in your fixed size alphabet, the number of symbols you need to write down that input in whatever your reasonable encoding is is going to be n. And it's going to be polynomial in n, polynomial in that length of the input.
So another question is, does SAT reduce the 3SAT? Yes. That we will see. SAT actually does polynomial time reduce to 3SAT. Not by converting to an equivalent formula, but by some more-- somewhat more involved argument than that. OK, why don't we move on? Some of these are going to come out anyway in the lecture.
OK, so now let's talk about NP completeness, because we've kind of set things up. We're not going to prove that the basic theorem, the Cook-Levin theorem about NP completeness, but at least we'll be able to make the definition. OK, here is our definition of what it means for a problem to be NP complete.
So a language B is called the NP complete if it has two properties. Number one is that it has to be a member of NP, and number two-- every language in NP has the polynomial time reduce to that language-- to that NP complete language in order for it to be NP complete.
So simple picture-- has to be in NP and everything an NP reduces to it. And so that's kind of the magical property that we claim that SAT has. sat, for one thing, is obviously in NP. And as we-- the Cook-Levin theorem shows-- or will show-- everything in NP is reducible to SAT, so SAT's going to be our first example of an NP complete problem.
And we're going to get what we claimed also for SAT-- that, if SAT or any other NP complete problem turns out to be solvable in polynomial time, then every NP problem is solvable in polynomial time. And that's immediate, because everything is reducible in polynomial time to the NP complete problem. So if you can do it easily, you can do everything easily just by going through the reduction.
OK. So the Cook-Levin theorem, as I mentioned, is that SAT is NP complete. And we're going to actually prove it next lecture, but let's assume for the remainder of this lecture that we know it to be true. So I'll use the terminology of problems being NP complete, assuming that we know-- that we have SAT as NP complete. OK? So we're going to be using some of the things that we're proving next lecture just in the terminology that we're going to be talking today.
OK, so here's the picture. Here's the class NP. And everything in NP is polynomial time reducible to SAT. SAT itself is a member of NP, but I didn't want to show it that way because it makes the picture kind of hard to display. So just from the perspective of the reduction, everything in NP is polynomial time reducible to SAT. We'll show that next lecture.
Another thing that we'll show next lecture is that SAT, in turn, is polynomial time reducible to 3SAT. So 3SAT, as you remember, are just those problems that are in conjunct-- are in 3CNF. And then what we show today is that 3SAT is polynomial time reducible to clique. So now, taking the assumption that SAT is NP complete-- so everything is polynomial time reduce both the SAT, which is, in turn, polynomial time reducible to 3SAT, and in turn, reducible to clique.
These reductions, as we've seen before, composed. You can just apply one reduction function after the next. If each one individually is polynomial, the whole thing as a combination is going to be polynomial. So now we know that 3SAT is going to be also NP complete, because we can reduce anything in NP to SAT, and then to 3SAT, and then we get a reduction directly to 3SAT by composing those two reductions-- and then, furthermore, at the clique.
So now we're-- have several NP complete problems. And moving beyond that, we have the HAMPATH problem, which we are going to talk about next. And we'll show another reduction in addition to the one we just showed to clique, now one going from 3SAT to HAMPATH. OK. So in general, I think the takeaway message is that, to show some language is NP complete, you want to show that 3SAT is polynomial time reducible to it.
OK, some good questions coming in-- I'll try to answer those. So you're going to take 3SAT and reduce to C. That's the most typical case. There's going to be other examples too, we might start with another problem that you've already shown to be in NP complete, and reduce it to your language. So it doesn't have to start with 3SAT, though often, it does.
Why is this concept important? So I would say there are two reasons. And this is going to get a little bit at some of the chat questions. First of all, if you're faced with some new computational problem-- you've got some robotics problem that you want to solve in your thesis, and you need some algorithm about whether the robot arm can do such-- move in a certain such a way, and involves searching-- possibly searching through a space of different kinds of motions, and you want to know-- I'd like to find a polynomial algorithm to solve that problem.
I'm using this as an example because this actually did happen to one of the former students from this class, who was working in robotics, and they actually end up proving that the problem that they were trying to solve is NP complete. So that's useful information, because even though knowing a problem is NP complete doesn't guarantee that it's not in P-- because conceivably, P equals NP-- what it does tell you-- if you have a problem that's NP complete and it does turn out to be in P, then P equals NP.
So there would be tremendous surprising consequences of your problem, if it was known to be NP complete, ending up in P. So generally, people take a proof of NP completeness as powerful evidence that the problem is not in P. Even though it's not quite a proof, it's powerful evidence that it's not in P. And so you might as well give up working on trying to find a polynomial algorithm for it, because if you do, you don't have to worry about robotics anymore.
You're going to become famous as a So I wouldn't worry about-- so if you throw problem's NP complete, you can pretty much assume-- almost certainly not in P. Now, there's another reason related to-- for the theorists to be-- care about NP completeness, and that is, if you're trying to prove P is different from NP-- or P equals NP, as one of the chat questions is raising-- order to prove P different from NP, the most likely approach is you pick some [? pre ?] problem in NP and show it's not in P.
That's what it would mean for them to be different. You're going to pick some NP problem and show it's not a P problem. Well, one thing would be terrible is possibly, P is different from NP, and you pick the wrong problem. Suppose I'd spent all my time trying to show-- back 20 years ago, I'm working really hard trying to show composites is not in P, which is-- would have been perfectly reasonable to do, because composites is an NP, and it was not known to be in P 20 years ago.
And then I invested tons of effort to try to prove-- because I like number theory or God knows what-- to prove that composites is not in P. And then, turns out, composites was in P. It was the wrong problem to pick, even though P might be different from NP. But what NP complete is guarantees is that, if you work on a problem, which is NP complete, you can't pick the wrong problem, because if any problem is in NP, and not in P, an NP complete problem is going to be an example of that-- because if the NP complete problems in P, everything in NP is in P.
So if NP is different from P, you know the complete problems are not in P, so you might as well work on one of those. So those are two ways in which NP completeness has turned out to be-- is an important concept. OK, so here's a check-in. You guys are getting-- maybe you're getting-- starting to think the way I think. You're getting-- some-- at least one of you asked this question in the chat.
Which language are we've probably seen is most analogous to SAT-- ATM, ETM, or 0 to the k, 1 to the k? Obviously, this is maybe subjective. You may have your own interpretation of what that means. There's, in a sense, perhaps no right answer, but what do you think? OK, that's it. I don't know how in the world that you could see this problem as analogous to 0 to the k, 1 to the k, but OK. I'm sure you have your reason.
Yes, this is a lot like ATM. Why? well, because for one thing, we showed in a homework problem that all Turing recognizable languages are reducible to ATM-- mapping reducible to ATM. So that's a little bit like the notion of completeness that we have for satisfiability, because all NP problems are going to be reducible to the SAT.
And the other thing too is that, once we start-- we want to show other problems are undecidable, we reduced ATM to them. And that's also very similar. We're going to be reducing SAT or 3SAT-- so it's indirectly from SAT-- to other problems in order to show that there shouldn't NP complete, so that they're hard-- that they're hard, in a sense. And so in both those ways, ATM and SAT are kind of playing similar roles.
One key difference, however, between ATM and SAT is in that for ATM, we can prove that it's undecidable, but for SAT, we don't know how to prove it's outside of P. Those would be the analogous situations. And so the story for SAT, which is easily solved by a diagonalization argument for ATM-- there's reasons to believe-- that we will see later-- that the diagonalization is not going to work to prove SAT outside of P. And besides that, we don't really have any good methods.
So anyway, let's move on. Why is ETM less analogous? Because the ATM was the first problem we showed undecidable, and SAT is the first problem that we're going to be showing NP complete. I guess that would be my answer. OK, let's continue. So let's show now that HAMPATH is NP complete, assuming that we know SAT or 3SAT is NP complete. So we're going to give a reduction from 3SAT to HAMPATH.
That's what this is about. It's just like what we did for clique, but now for HAMPATH. And this is going to be very typical. In these reductions, typically what happens is that you're trying to simulate a formula-- Boolean formula for satisfiability-- from the satisfiability perspective. You're trying to simulate that formula with some sort of structures inside the target language, which would be HAMPATH. The lingo that people use is that you're going to build gadgets to simulate the structures in the formula-- namely, the variables, the literals, and the clauses.
OK, these are going to be substructures of the graph, in this case, that you're building. We'll see what that means. So let's take a formula here and let's, again, try to imagine how we would reduce that to the HAMPATH problem. So the reduction would produce a graph-- no, it would produce an HAMPATH instance-- so a G, s, and t. Want to know, is there a Hamiltonian path from s to t in the graph?
And this is going to be not the whole graph, but this is going to be a substructure in that graph. The next slide is going to have the global structure of the graph. But here, this is going to be a key element, and we're going to call that the variable gadget. OK, what does it look like? I don't know if you can see it on your-- clearly enough, but these edges-- so there are four outside nodes here. The edges connecting them are all kind of pointed downward. And then there were these horizontal nodes here, and there are edges connecting them both left to right and to left. There's a row of these horizontal nodes.
OK, so you get the picture of what this looks like? You have to look carefully to see the arrowheads. I maybe should have made those a little bigger. Whoops-- so now, if we're trying to get from this node to that node-- imagine now you're trying to build a Hamiltonian path, because this is going to be a part of that graph G that I'm constructing.
Now, remember, for them-- for there to be a Hamiltonian graph, that means you have to go through every node in the graph. So if I want to get from s to t, the only way I'm going to be able to go-- to hit these horizontal nodes here is by picking them up as I go from s to t. So the only possibility is-- if you think about it, is-- would be for the sequence to go like this, if you can-- if that comes through for you.
So the path would go from s to this node, and then through these hops of-- along these horizontal nodes, and then down to the bottom node here. So that's one way that you can get from here down to there and pick up all the other nodes along the way, which is-- they're not going to have any other possibility-- possible ways of getting to them. So I'm going to call that a zig-zag. OK?
But there's another way to get from s to the bottom node, which is going to be by doing sort of the dual-- going to the right, then going out to the left, and then down to the bottom. I'm going to call that a zig-zig-- and with a little diagram here just to summarize what it means.
And this is the classic thing for a variable gadget, because it's a structure that when you're trying to think about how the object that you're asking whether it exists or not-- the Hamiltonian path-- how it relates to that object, it's going to have two possibilities, which are going to correspond to the variable being set true or false in the formula.
So we're showing how to set the-- simulate the variable in this HAMPATH instance. So setting the variable is going to correspond to constructing the path. OK? Now, we also have to make sure not only that the variable gets set to true or false, but that it gets a set of true and false in a way that makes this a satisfying assignment-- namely, that we get one true literal in every clause. So I'm going to add another gadget here called a clause gadget, which is just a single node.
And visiting that node here is going to correspond to satisfying that clause, to having a true literal in that clause. I'm going to have [? enabled ?] a detour from these horizontal nodes out to this clause gadget. So here it is. Here's that detour. As I'm going from left to right, I can-- instead of doing a single jump here, I could branch out and visit that clause node, and then come back, and pick up my left to right path, as I was doing before.
I hope you're seeing the big picture, because this has to be a Hamiltonian path. It has to hit every node. That's one of the requirements. This is going to be one of the nodes in my graph. The path has got to hit that node. The only way it's going to be able to hit that node is by taking a detour off of this horizontal path here.
But notice-- and this is the key-- if I'm doing a zig-zag, then I can make the detour and visit that clause gadget-- that clause node. But if I'm doing a zag-zig, the way this detour is structured does not allow me to visit that node, because if I'm going from to right left here, by the time I get to this outgoing edge-- now I want to come back-- that note has already been taken.
It only works if I'm going zag-zig, if I'm going from right to left-- left to right. If I'm going from left to right, then I can do it, but if I'm going right to left, no. If you think of left-- the left to right direction as true, that's going to correspond to that variable appearing in the positive way in that clause, but not in the negative way. The negative way-- I would reverse in and out of the detour, flip that around so now I could only do the detour when I'm going right to left, instead of left to right.
I hope you get the picture. This is how the structure is working. Now what's only left for me to do is put it all together. But this slide contains the guts of what's happening. Is there any question that I can answer for you on this? Let's move on to the next slide, and then, as questions come up, you can be typing them in, and we can maybe answer it there. OK, so here is the big picture.
So imagine we have that formula that we're start-- we're reducing that formula. And let's say it has m variables and k clauses. I'm going to call them clause c1, c2, up to ck-- here, the m variables appearing either positively or negated in that formula. And this is the way the global structure of G is going to look like. Now, I'm getting a question here. What do those horizontal nodes-- what role do they play?
Those nodes are there to allow me to visit those clause nodes, the nodes which represent the clause gadgets, which I'm going to place over here. This is almost a whole of G. I'm just missing a few edges, but these are all the nodes of G. So remember, I'm trying to find out, is there a Hamiltonian path from s to t?
Now, if I didn't have to worry about these nodes, the answer would be just yes. In fact, there would be many Hamiltonian paths from s to t, because I can do a zig-zag or a zig-zag through each of these variable gadgets, and that would take me from s to t, and I'd be good. It's just the c nodes, these nodes-- these ci nodes-- I have to hit them too. So they're going to be-- visiting them is going to be enabled by detours from here.
So let me just try to show you what that looks like. So I'm going to magnify little pieces here from these gadgets here and show you how these guys are connected up. So here is the x1 gadget. So x1 appears positively in c1. Here's x1 and c1. And so that means, when I'm going left to right, I'm going to be a possibility of visiting c1. But now, let's look what happens with-- which was the next one I had here? OK.
Right. So the next one is-- oh, yeah. So x1 appears positively in c1. That's why I have the connection like this. Now, x1 appears negated in c2, so I only want to do-- enable the detour to visit c2 when I'm going to left, as opposed to left to right. So this set of horizontal nodes is only going to allow me to take a detour either out to c1 or out to c2, but not to both, because the Hamiltonian path is either going to go left or right or to left. It can't do both, when it's going through the x1 gadget.
You need to-- [INAUDIBLE] I'll try to help you through it, but you have to try to think about why it's working. Let's think together. So x2 also appears in c1, but now it appears negated.
So I'm going to have edges from this x2 gadget-- oops-- the x2 gadget, but now look at the-- look at how I've arranged that detour. I can leave on the left leftward side and return on the right side, which means I can only do that detour when I'm going to left. And that's because x2 is negated in c1.
Maybe it's a lot here, if you're not quite getting it, but the point is, let's say, try to quickly prove-- so that's the whole construction. You just do that for every single appearance of the literal in a clause. You're going to add these detours, which allow you possibly to go visit the clause. So the forward direction is-- why is this true? So you take any satisfying assignment, as I suggested, make the corresponding zig-zags or zag-zigs through the variable gadgets from s to t, and then take the detours to visit the clause nodes.
The reverse direction actually is slightly trickier. We're not going to have time to go through the subtlety of it, but what you want to make sure here is that you don't have a weird path occurring, because I'm going to start with a Hamiltonian path now and build an assignment. And we want to make sure that the path that I'm constructing doesn't go from one-- from this horizontal nodes to a closed node, and then back to somebody else's horizontal nodes, and is kind of a hodgepodge of things which don't make any sense in trying to reconstruct a satisfying assignment.
What you really want to have happen is the Hamiltonian path should have clear zig-zags and zags-zigs that allow you to decide how to set the variables. And so that's the role of these little nodes here. These are spacers that separate the detours from one another, and that force a visit to the clause node to come back to the same place from which it left. Otherwise, you would never be able to visit those spacer nodes.
You have to look in the book for that, but there is a little bit of a detail that you have to go through. You have to show it must be zig-zags and zag-zigs, and then you get the corresponding truth assignment, and it must satisfy phi for all paths. OK. Again, the reduction is polynomial time computable. I'm not going to say more about that. We're a little bit low on time.
Last check-in-- would this construction still work if G was undirected? Suppose I just eliminated all the directions from the edges, made them lines, instead of arrows. Would that now show that the undirected Hamiltonian path problem is NP complete?
Let me see. OK, what do the c nodes represent here? There's one c node for every clause. So there are k clauses named c1 to ck, and there are k c nodes. These are the so-called clause gadgets, which are going to force there to be one true literal in every clause for the satisfying assignment. You have to look at it, or maybe we can spend a little bit more time explaining it, but that's what the purpose is.
Does that mean we need only two inside nodes? So the horizontal nodes-- do we only need two of them? You won't be able to reuse these nodes for multiple detours. For one thing, once you've gone to a detour, you come back to the node next over, and so you better not overlay multiple detours. And also, you need to keep them separated from each other. Don't forget, this node x1 can appear in many, many different clauses, so you would need to have possibly many of these horizontal nodes.
So someone now says 2k inside nodes would suffice. Probably 2k-- I would say 3k, just to be safe for the spacer nodes. You need to look carefully at the argument, which is laid out in the textbook. You may not actually need the spacer nodes, but then it makes the argument just more ugly. So the way the construction is done is you have 3k inside nodes.
OK. Again, several questions like that-- the graph would start looking messy if x9 was in c1. Yeah, if x9 down here was in c1? Yeah, it would be messy. It's OK. Messy is allowed. All right, I think we're-- let's end the polling. Are you all in? All right-- share results. Yes. The answer's no. The construction depends on this being directed.
You can see that all over the place, but for one thing, the whole point of these detours is the directions of the edges. And so without that, this construction is going to be just a bunch of-- is not going to mean anything. It's not going to prove anything. It's probably always going to be-- have a Hamiltonian path without the directions.
So I think we're out of time. A quick review-- these are the topics we've covered. I think we're out of time, so I should let you go. But I'll stick around for a few minutes, in case any of you have any questions. But I need to run off at 4:00 myself for another meeting, so I don't have much that much time.
Clarify my comment about picking the wrong problem to tackle P versus NP, when I used composites as an example-- if I worked hard to prove that composites is not in P as a way of proving that there is some NP language which is not in P, that would have been a mistake, because composites is in P. I would have been working hard to prove something which we now know was false.
So we don't want to spend time working on the wrong language. But the nice thing about NP complete languages is that we have a guarantee that, if P's different from NP, that that language is not in P. Are there problems that are not in P that are in NP, but are not NP complete? Oh, that's a good question. Are there problems in between P and NP complete?
So NP complete is sort of like the hardest problems in NP, and the P problems are obviously the easy problems that are in NP. Is everything either NP complete or in P? So for one thing, there are problems that are not known to be in either category. So we'll discuss some of those in due course, but one of them is the graph isomorphism problem, testing two graphs-- if they're really just permutations of one another. It's clearly a problem in NP, but not known to be in P. So there are problems that are not known to be either NP complete or in P, so there are problems that might be in between.
But then there was another theorem out there, which says that, if you assume that P is differ from NP, then you can construct problems which are in between, which are neither NP complete nor in P. They're NP, problems but they're not NP complete, not in P. So those problems themselves are perhaps somewhat artificial, but they at least prove the point that it is possible to have these intermediate problems.
Oh, so somebody's asking, isn't factorization one. Not [? known-- ?] for the case of factorization-- or you have to make a language out of that, by the way. But it's a because factorization's a function, so we won't really want to be talking about languages. As the homework suggests, NP and P are classes of languages, but OK, that's a separate note there.
Factorization is not known. Factorization could be in P and it could be NP complete. Both of those are not ruled out. So I think most people would probably venture to guess that it's a problem that's in the in-between state, that's neither P nor NP complete, but not known.
Who first thought of this reduction from 3SAT to HAMPATH? It's so clever. Well, it wasn't me. I think that is due to Dick Karp, who was one of my professors at Berkeley, where I was a graduate student. That was done before I got there. It was around 1971 when-- so there were two famous papers. There's the Cook paper. There's also the Levin paper. That was in Russian. That took a while for people to discover out here in the West.
But the Cook paper was 1971, and very quickly followed after-- he just showed SAT is NP complete, but after that, Karp was-- he had a paper called "Reducibility Among Combinatorial Problems," and he had a list of about 20 problems that he showed were NP complete-- by reduction from SAT-- include clique, include HAMPATH, and a bunch of other things. And that was also a very famous paper. Both of those are-- people often talk about Cook-Karp as, together, they really show the importance of NP completeness and the whole notion of NP completeness.
Yeah, 21 problems, so-- yeah, so Karp proved 21 when problems were NP complete in 1972. So that was shortly after Cook showed that SAT was NP complete. Incidentally, I think the terminology NP complete wasn't around until a little later. And that might have been-- might be due to Knuth. I'm not sure.
I remember he did a big poll of people about what should be the right language to use for that term, and I think he came up with it. All right, I'm going to head off, guys. Nice seeing you all-- so until Thursday-- oh, until Tuesday. Bye bye.