Flash and JavaScript are required for this feature.
Download the video from Internet Archive.
Description: Quickly reviewed last lecture. Introduced nondeterministic finite automata (NFA). Proved that NFA and DFA are equivalent in power. Proved that the class of regular languages is closed under ◦, ∗. Showed conversion of regular expressions to NFAs.
Instructor: Prof. Michael Sipser

Lecture 2: Nondeterminism, ...
MICHAEL SIPSER: Hi, folks. Why don't we get started? Welcome back. Good to see you all here. So I am going to first-- well, we'll recap what we did last time and what we're going to do today. I'll talk a little bit about the problem set. And we'll also have a break, as requested, halfway through.
So why don't we jump in? What we did last time was besides introducing the course, we introduced finite automata in regular languages, which are the languages that the finite automata can recognize. We talked about these regular operations. Those allow us to build what we call our regular expressions. These are ways of describing languages.
So we have finite automata can describe languages, and regular expressions can describe languages. And one of our goals is to show that those two systems are equivalent to one another, even though they look rather different at first glance. So to move in that direction, we're going to prove closure properties for the class of regular languages over these regular operations. So we'll show that-- well, we already showed that any two regular languages have their union, also being regular. And we'll show that for the other two operations as well.
So let's just look ahead to what we're going to do today. We're going to introduce an important concept which is going to be a theme throughout the course, called nondeterminism. And having that as a tool that we can use, we'll be able to show closure under concatenation and star, finishing up what we started to do last time. And then we'll use those closure constructions to show how to convert regular expressions to finite automata. And that's going to be halfway to our goal of showing that the two systems are equivalent to one another. And the following lecture, we will show how to do the conversion in the other direction.
So I thought we would just jump in, then, and look at-- return to the material of the course. As you remember, we were looking at the closure properties for the class of regular languages. We started doing that. And if you recall, hopefully, we did closure under union. And then we tried to do closure under concatenation, which I have shown here on this slide, the proof attempt that we tried to do last time. And let's just review that quickly, because I think that's going to be helpful to see how to fix the problem that came up.
So if you remember, we're given two regular languages, A1 and A2. And we're trying to show that the concatenation language A1A2 is also regular. And so the way we go about all of these things is we assume that A1 and A2 are regular. So that means we have machines, finite automata, for A1 and A2. We'll call them M1 and M2, that recognize A1 and A2, respectively.
And then what we need to do in order to show the concatenation is regular is to make a finite automaton which recognizes the concatenation. And we tried to do that last time. So if you remember, that concatenation machine-- M, we're calling it-- what is it supposed to do? It's supposed to accept its input if it's in the concatenation language. And that means that the input can be split into two parts, x and y, where x is in the A language, and y is in the B lang-- y is accepted by M1-- and x is accepted by M1, and y is accepted by M2. Sorry I garbled that up.
So x should be in A1, and y should be in A2. if you can split w that way, then M should accept it. So M has to figure out if there's some way to split the input so that the first machine accepts the first part, the second machine accepts the second part.
And the idea that we came up with for doing that was to take these two machines, build them in to a new machine M, and then connect the accepting states for M to the start state-- connect the accepting states for M1 to the start state for M2. Because the idea would be that if M1 has accepted an initial part, well, then you want to pass control to M2 to accept the rest.
But as we observed, that doesn't quite work. Because the first place to split w after you found an initial part that's accepted by M1 may not be the right place. Because the remainder may not be accepted by M2. You might have been better off waiting until you found another place that M1 accepted, later on in the string, say, over here. And then by splitting it over there, then maybe you do get successfully find that the remainder is accepted by M2. Whereas if you tried to split it in the first place, the remainder wouldn't have been accepted by M2.
So all you need to do-- M has to know, is there some place to split the input so that you can get both parts accepted by the respective machines? The problem is that M might need to know the future in order to know where to make the split. And it doesn't have access to the future. So what do we do?
So what we're going to do is introduce a new concept that will allow us to basically get the effect of M1-- and the-- sort of being able to see the future. And that new concept is going to be very important for us throughout the term. It's called nondeterminism. And so we're going to introduce a new kind of finite automaton called a nondeterministic finite automaton. And first, we'll look at that, and then we'll see how that fits in with the previous deterministic finite automaton, that we introduced last time.
So here's an example. It's always good to start off with an example. Here is a picture of a nondeterministic finite automaton. It looks very similar, at first glance, to the former kind, the deterministic finite automaton. But if you look a little carefully, you see that there are some key differences.
The most important difference is that in state q1, for example, whereas in the machines that we introduced last time, there had to be exactly one way to go on each possible input symbol so you knew how to follow along through the machine it's computing, here there are two ways to go. In q1, you can either stay in a1, or you can go to q2. That's the essence of nondeterminism. There could be many ways to proceed.
And furthermore, on q1, if you get a b, then there's nowhere to go. So that's also possible within nondeterminism. So let's just start looking at these features.
There are multiple paths forward-- multiple paths possible. You might be able to have one, as we had before, or many ways to go at each step, or maybe 0 ways to go at each step. Those are all legitimate for a nondeterministic machine, which is doing a nondeterministic computation.
Another difference, if you look carefully, is that we're allowing here the empty string to appear on a transition. That's perhaps a little less essential to the spirit of nondeterminism. But it's going to turn out to be a convenience when we actually apply nondeterminism to build machines, as you'll see very shortly.
Now, if there are many different ways to go-- and some of those ways to go might have different outcomes. As we remember from before, we accepted the input, if you end up in an accept state, and we rejected the input, if you end up not in an accept state, in a non-accepting state. Then you reject.
But now there might be several different ways to go. And we'll do an example in a minute. But there might be several different ways to go. And they might disagree. Some of them might accept. Other ones might reject.
So then what do you do? Well, in that case, acceptance always overrules rejection. That's the essence of nondeterminism the way we're setting it up. You may ask why that is. And the spirit of that will become clear in due course. But right now, just take it as a rule.
When we're having a nondeterministic machine, acceptance overrules rejection. So as long as there is-- one of the possible ways to go ends up at an accept, we say the whole thing is accepted. The only way we can possibly reject-- if all of the possible ways to go end up at rejection, end up at a non-accept state. So we'll see example of-- I think we're going to do an example right now, yeah.
So if we take, for example, this machine N1 now on an input ab-- and we're going to process the symbols one by one, just like we did before. But now, to follow along, there might be several different ways to go. So if we take the first symbol a, and we run the machine-- so when the machine, it starts at the star state, as before-- but now an a comes in. And there might be two-- now there are two different ways to go.
So we're going to keep track of both of them. After the machine reads an a, you can think of it as being in two states now simultaneously. It can be in state q1, and it can be in state q2. So those are two different possible places it could be at this moment. OK?
Now we read the next symbol, the b. And from a b, you take each of the places where the machine could be at the end of the previous symbol, and you then apply reading a b, the next symbol, from each of those states where the machine could be in from the previous symbol. So the machine could be in q1 and q2 after reading an a. Now we apply a b.
Well, q1 on a b goes nowhere. So you think of that branch, if you will, of the computation as just dying off. It has nowhere to go. It just vanishes. However, the other possibility, which was state q2 on a b, does allow, does have a place. So the machine is now going to go from q2 to q3 on that branch of the computation, which reading-- on reading a b.
And then it has, coming out of q3, there are two symbols. There's an a and an empty string symbol. Now, on an a, the machine would have to read an a in order to transition along that arrow. But when there's an empty symbol on the arrow, that means the machine can go along that arrow for free without even reading anything. As long as it gets to q3, it can automatically jump the q4.
And so once it has read a b and gone to q3, now it can either stay in q3, or it can go along the empty transition and go to q4. So again, it is going to be a nondeterministic step at this point. The essence of having a empty transition is that there is going to be nondeterminism. That's why we didn't introduce that for deterministic automata, because you don't have to transition along an empty string transition. You can stay where you are, or you can go along the empty string transition without reading any input and go over to the next state, which, in this case, is q4.
So let's just see where we are. After reading an a, we're in states q1, q2. But now after reading a b, we're in states q3 and q4 as possibilities. And now we're at the end of the input, and we look and see what we got.
If any one of the states as possibilities that we are right now at the end of the string is an accept state, then we say, overall, the machine has accepted. So that corresponds to what we said over here before-- accept the input if some path leads to an accept. So if any way of proceeding through these nondeterministic choices will lead you to an accept, then you will say, we're going to accept the input. OK? So this input here is accepted.
Let's do another example. Suppose we have the input aa instead of ab. So aa, after the first day, as before, we're in states q1 and q2 as possibilities. Now we read an a again. Now, the one that's on state q1, that possibility, q1 possibility, after reading an a, it again branches to q1 and q2. So we know after reading the second a, we're going to be in at least q1 and q2.
Now how about the state that had been on q2 on reading an a, the one from before? After reading the first a, you were in q2. Now reading the second a, there's nowhere to go. So that one just gets removed.
So after reading aa, we remain in states q1 and q2 as possibilities. Neither of those are accept states. So therefore, on input aa, the machine rejects. OK? Let's just do a couple more, and then I'll ask you to do one.
So we have aba as an input. Let's see what happens then. So remember, after reading ab, the machine is in the two states q3, q4 as possibilities. That's what we have from the first example. So after reading ab, we're in states q3, q4.
Now we read another a. The q4 on an a has nowhere to go. In fact, it has nowhere to go in any input. So no matter what comes in after you're in state q4, that branch dies.
But on q3, which is another one of the possibilities reading an a, it can follow along, just transition. Because that's one of the labels on that transition, is a. So you can follow along just in transition on reading an a, which is the last symbol in the string. And so now after aba, you are in only state q4 as a possibility. But that happens to be an accept state, so the machine accepts. OK.
And now lastly, let's take our final example. What happens if we have abb? So as we remember before, after reading ab-- that was the first example-- we were in states q3, q4 as possibilities. Now we read a b. Well, neither of those states have anywhere to go on a b.
So now all threads, all branches, of the computation die off. And at this point, the machine is totally dead. It has no active possibilities left. So certainly, it's going to reject this input, because none of the active states-- there are no active states or accepting states.
And in fact, if you looked at anything that came later, anything that extended the string abb would also be rejected. Because once the machine had all-- all possibilities have died off, there's no way for them to come back to life on any extensions. So with that-- oh, here's an important point before I'm going to jump to a check-in on you. But I think one thing that might be on your mind about this nondeterminism is how does that correspond to reality? Well, it doesn't.
We're not intending for nondeterminism as we're defining it to correspond to a physical device. But nevertheless, as you'll see, it's a very mathematically useful concept, this nondeterminism. And it's going to be playing a big role throughout the subject as we'll experience it during the rest of the term.
So with that, I'm going to have a little check-in. I'm going to ask you to consider what happens on one of the inputs. So here we go. What does it do on input aab? So here's the machine. You can look at it. And suppose, hopefully, there's a poll here for me to give to you so you can give me your input. So what does the machine do on input aab? Most of you have answered.
Again, you're not going to be penalized for getting the wrong answer. But hopefully, you'll get the right answer. Anyway, let's just take a look here. So time is up. Let's end the polling and share the results.
So the majority of you, majority have gotten the correct answer, which is a. The machine does accept aab. Because when you have a-- so I'll show you the path that corresponds to accepting. You go a, a, b, and then empty string. And so that sequence of steps is one of the nondeterministic possibilities that the machine can follow. And that shows that the machine does accept the input aab.
You can think about it, the way we did it before also. If you read an a, it's in the two possibilities q1, q2. You read a second a, again, in the possibilities q1, q2. Now you read a b. It's in the possibilities q3, q4. And that's it, aab. So now you read the b. It's in possibility q3, q4. q4 is an accepting state. That overrules the non-accepting state. And so the machine accepts.
You have to understand this. So if you didn't get it right, go back and think about where you slipped up. OK? Because this is just getting-- we're just getting warmed up here. It's going to get a lot harder. OK, so stop sharing the results. And so let's continue.
So just as we did last time, we can formally define a nondeterministic finite automata. Here's the picture again. OK. So it looks a lot like the case we had before, the Deterministic Finite Automata, or DFA, as we'll call them. It's a 5 tuple. So I've written down little reminders for what those components of that 5 tuple are, that list of five components.
So they're all the same as before-- states, alphabet, transition function, start state, and accepting states. So that the formal definition looks exactly the same except the structure of the transition function. So now, before, if you remember, you had a state and an input symbol, and you got back a state.
Now we have something more complicated-looking. We have a state and an input symbol, but instead of just sigma, it's sigma sub epsilon. And that that's a shorthand for sigma union epsilon. And that's a way-- my way of saying that you're allowed to have on your transition arrows either an input symbol or an empty string. So the transition function has to tell you what to do when you have an empty string coming in as well. So that would be part of your table for the transition function.
Now, over here, what's going on over here? Well, now, instead of just producing a single state, when you've read, for example, an a from q1, there's a whole set of possibilities. So here we have what's called a power set. That's the set of subsets of the collection Q. So here we're going to produce an entire subset of states. Instead of just one state coming out, there might be a subset of possible states that you can go to.
So the power set of Q is a set of subsets of Q. So that's what this notation means. Again, this is something that I'm, hopefully, presenting to you as a bit of a reminder. You've seen this somewhere else before. But please make sure you understand the notation, going forward, because we'll be doing less hand-holding as we start moving forward. OK.
So just let's take a look. In the N1 example here, just to illustrate what's going on, when you're in state q1 reading an a, now you get a whole set of possibilities, which, in this case, is q1 and q2. Whereas, if you're reading a b, what would be that set? Coming out of q1, what's the set of possible successor states? Well, there are none. So it's the empty set. OK? So hopefully, you're understanding the notation here.
So now here's, I think, really important. How do we understand nondeterminism, intuitively speaking? And there are multiple different ways, which each has their value under different circumstances.
So one way is thinking about nondeterminism as a kind of parallelism. So every time the machine has a nondeterministic choice to make, where there's more than one outcome, you think of the machine as a branching, forking, new threads of the parallel computation at that stage, where it makes an entire copy of itself when there's a choice of possibilities. And then each of those independently proceeds to read the rest of the input as separate threads of the computation.
So if you're familiar with parallel computing, this should be reasonably familiar to you. The only key thing to remember is that as this thing forks a number of possibilities, the acceptance rule is, that if any one of those possibilities gets to an accept at the end of the input, it raises a flag and says, accept. And that overrules everybody else. So acceptance dominates.
So another way of looking at it is the mathematical view, where you can imagine-- and we're going to use all these. So you really need to understand them all. The mathematical view is you can think of the computation as kind of a tree of possibilities. So you start off at the very beginning at the root of the computation, which is when it really begins. But every time there's a nondeterministic branching that occurs, that node of the tree has multiple children coming out of that node. And so the different threads of the computation correspond to different branches of that tree.
And now you're going to accept if any one of those branches leads to an accepting state-- OK, obviously, somewhat similar to what we had before. But I think it's a little bit of a different perspective on how to think about nondeterminism. And the last one is going to sound a little weird.
But actually, I think for people who are in the business, it's the one they use the most. And that's the magical way of thinking about nondeterminism. And that is, when the machine has nondeterministic choices to make, you think of the machine as magically guessing the correct one at every stage, and the correct one being the one that will eventually lead it to accept. OK?
So you can think of the machine as guessing which is the right way to go. And if there is some way right way to go, it always guesses right. Of course, if the machine ends up rejecting, because there is no right way to go, then it doesn't matter. There is no good guess. But if there is some good guess, we'll think of the machine as taking that good guess and going that way. OK.
So now here is a very important thing. We introduced this new model, the Nondeterministic Finite Automaton, NFA. It turns out, even though it looks more powerful, because it has this nondeterminism, it isn't any more powerful. It can do exactly the same class of languages, the regular languages. And we'll show that with this theorem here, that if an NFA recognizes a, then a is regular.
So we'll prove that by showing how to convert an NFA to an equivalent DFA, which does the same language. So we can take an NFA that has the nondeterminism and find another DFA which doesn't have nondeterminism, but does the same language. It accepts exactly the same strength, even though it lacks that nondeterministic capability.
This is going to be extremely useful, by the way, and for example, in showing that closure under concatenation. OK, so in this presentation here, I'm going to ignore the epsilon transitions. Because once you get the idea for how to do this, you could figure out how to incorporate them. They just make things a little more complicated. So let's just focus on the key aspect of nondeterminism, which is that the machine could have several ways to go at any point in time. There could be several next states on an input. OK?
Now the idea for the construction-- so we're going to start with a nondeterministic machine M, and we're going to build a deterministic machine M prime, which does exactly the same thing. And the way M prime works is it's going to do what you would do if you were simulating M. What would you do? This is what we were doing as I was explaining it to you.
If you were simulating M, every time you get an input symbol, you just keep track of what is the set of possible states at that point in time. That's what the DFA is going to do. it's going to have to keep track of which possible set of states the NFA could be in at the point on that input where we are right now.
And then as you get to the next symbol, the DFA is going to have to update things to keep track of the next set of states the NFA could be in at this point, just like you would do. OK? And so here's a kind of a picture. And how do we implement that?
So here's the NFA that we're starting with, M, and we're going to make here the DFA. But in order to remember which set of states that DFA could be in at a given point-- so maybe it's in the set of states that M could be in. Did I say it wrong? Which set of states the NFA could be in a given time-- so maybe M, the NFA, could be in, at some point, state q3 and q7. The way the DFA keeps track of that, it's going to have a state for every possible subset of states of the NFA. That's how it remembers which subset of states the NFA is in.
That's the way DFAs work. They have a separate state for each possibility that they need to keep track of. And the possibilities here are the different subsets of states that the NFA could be in at a given point. OK? So corresponding to this subset, to these two possibilities q3, q7, the DFA is going to have a state with the subset q3, q7. And it's going to, for every possible subset here, there's going to be a different state of M prime. So M prime is going to be bigger.
OK. So quickly, the construction of M, the states of M prime now, q prime, are going to be the power set, the set of subsets of states from the original machine M. And now we have to look at how the transition function of the DFA, when you made the primed machines of the DFAs, the DFA machine. So these are the deterministic components.
So delta prime, when it has a subset, something like this, has one of its states, which corresponds to a subset of states of M, and it reads an input symbol, you just have to do the updating the way you would naturally do. You're going to look at every state in R, look at where that can go under a-- so there's a bunch of sets there. And look at all the possible states that could be in one of those subsets, and that's the set of states that you could be.
That's going to be the new set of states, and that's going to be in the new state of M prime. OK? So it's going to be the subset corresponding to all of the states that could be in, when you apply the transition function of the nondeterministic machine, to one of the states in the subset of states that the nondeterministic machine could be in. OK?
It's a little bit of a mouthful. I suggest you look at this, if you didn't quite get it, after the fact. Good to understand.
The starting stage for the NFA-- for the DFA-- I'm sorry-- is going to be which subset now we're going to start off with. It's going to be the subset corresponding to just the start state of M. And the accepting states are going to be-- of the deterministic machine are going to be all of the subsets that have at least one accepting state from the NFA. OK?
So I hope you got that. Because I'm going to give you another little check-in here. Which is I'm going to ask you, how big is M prime? How many states does M prime have? I told you what those states are. So just go think about that. So check-in two-- if M has n states, how many states does M prime have by this construction? OK, so let's launch the next poll. OK, five seconds-- and I think we're almost done here. good.
All right, share results-- I don't know if sharing results is a good thing. I'm not trying to make you, if you didn't get the right answer-- because most of the people did get the right answer-- but if you didn't get the right answer, trying to make you feel bad. But it's a little bit of suggestion that you need to review some basic concepts.
So the basic concept here is if you have a collection-- you have a set of states, how many subsets are there? And the number of subsets is going to be exponential. So if you have a collection of n elements, the number of subsets of those n elements is 2 to the n. That's the fact we're using here. And that's why M prime has 2 to the n states, if M had n states. And you should make sure you understand why that is.
All right, so with that, as requested, we're going to have a little break. And that break is going to last us exactly five minutes. So we will return in five minutes. I'm going to be prompt. So I gave you a little timer here. So please, I'm going to begin it right when this is over.
OK, almost ready. I hope you're all refreshed and ready for the second half. So now that we have nondeterminism, we're going to use that as a tool to prove the closure properties that we were aiming for, starting from last lecture. OK.
So remember, let's look at closure under a union. Now, we already did that, but I'm going to do it again, but this time, using nondeterminism. And you'll see how powerful nondeterminism is. Because it's going to allow us to do it almost with no effort.
We'll start off the way we did before. I'm going to start off with two DFAs. But actually, these could be NFAs even. But let's say we started with the two DFAs for the two languages A1 and A2. And now we're going to construct an NFA, recognizing the union. And that's good enough, because we already know that we can convert NFAs to DFAs. And therefore, they do regular languages, too. OK.
So now here are the two DFAs that do the languages A1 and A2. And what I'm going to do is I'm going to put them together into a bag of states, which is going to be M, the NFA that's going to do the union language. So remember-- what does M supposed to do? M is supposed to accept its input, if either M1 or M2 accept.
So how is it going to do that? What it's going to do, we're going to add a new state to M, which is going to branch under epsilon transitions. And now you can start to see how useful these epsilon transitions are going to be for us. Going to branch under epsilon transitions to the two original start states of M1 and M2.
And we're done. Why? Well, now, nondeterministically, as we get an input, w coming in to M-- and at the very beginning, even just right after it gets going, the very first thing that happens is it's going to branch to M1 and also branch to M2 nondeterministically as two possibilities. And then inside M1 and M2, it's going to actually start reading the input. And each one is going to be now following along as it would have originally the states corresponding to reading those input symbols.
And M, as a combination of M1 and M2, is going to have a possibility for one state in M1 and one state in M2. And so M is going to have those combined into one package. And now at the end of the input, if either of these end up at an accepting state, then M is going to accept as a nondeterministic finite automaton. Because that's how nondeterminism works.
You accept if either-- if any one of the branches ended up accepting-- which is just what you need for union. So when we're doing union, you want either one of these to be accepting. And the nondeterminism just is built conveniently to allow us to do the union almost for free.
So you can again, thinking about nondeterminism as terms of parallelism, you could think of the nondeterministic machine as running in parallel M1 and M2 on the input. And if either one of them ends up accepting, M will accept. Or you can think about it in terms of that guessing that I referred to before, which means that as M is getting-- when it's just about to read the first symbols of its input, it guesses whether that's going to be an input accepted by M1 or an input accepted by M2.
And the magic of nondeterminism is that it always guesses right. So that input happens to be an input that's going to be accepted by M2. M is going to guess that M2 is the right way to follow. And it's going to go in the M2 direction. Because nondeterminism, the magic is you always guess right. I wish that was true in real life. It would make exams a lot easier.
Anyway, so now let's see how we can use that to do closure under concatenation. OK, so now we're going to actually have a picture of very similar to the one we had originally. But now using nondeterminism, we can make it work. So here we have the two machines doing the two languages, A1 and A2. And we're going to combine them into one bigger machine M, as shown.
Remember, what M is supposed to do is accept its input, if there's some way of splitting that input, such that the first half is accepted by the M1, and the second part is accepted by M2. The way we're going to get that effect is by putting in a transit empty-- empty transitions, epsilon transitions, going from the accept states of M1 to the start state of M2, just as I've shown in this diagram.
So these were the original accepting states of M1. And now they're going to be declassified as accepting states. But they're going to have new transitions, empty transitions, attached to them, which allow them to branch to M2 without reading any input.
And so intuitively speaking, this is going to do the right thing. Because once M1 has accepted some part of w, then you can nondeterministically branch to M2. And you're going to be start processing inside M2.
And the point is-- I jumped ahead of myself-- is that the reason why it fixes the problem we had before is that the epsilon transitions don't-- the machine does not have to take that. It can stay where it is as one nondeterministic option, or it can move along the epsilon transition, without reading any input, as another nondeterministic option. So it's using this nondeterminism now to both stay in M1 to continue reading more of the input and to jump into M2 to start processing what might be the second half or the second part of the input which M2 accepts.
And you can think of it in terms of the guessing as that the machine is guessing where to make that split. Once it found an initial part that's accepted by M1, it guesses that this is the right split point. And that passes to M2. But there might be other guesses that it could make corresponding to other possibilities.
And so with nondeterminism, it always guesses right. If there is some way to split the string into two parts accepted by M1 and M2, the machine will make that good guess. And then M1 will accept the first part, and M2 will accept with the second part. And we'll get M accepting that whole string altogether.
And so that is the solution to our puzzle for how do we do closure under concatenation. OK, I hope that came through. Because we're just getting going with nondeterminism. We're going to be using nondeterminism a lot, and you're going to need to get very comfortable with it. OK?
Now let's do closure under star. And closure under star works very similarly, but now we're just going to have a single language. If A is regular, so is A star. So they're not a pair of languages, because a star is a unary operation applying to just a single language. So if we have a DFA recognizing A, in order to show that A star is regular, we have to construct a machine that recognizes A star. And the machine we're going to construct is as before and then an NFA. OK?
So here is M, the DFA for A. And we're going to build an NFA M prime that recognizes A star. And let's think now, what does it mean to recognize A star? So if I'm going to give you an input, when is it in the star language? What does M prime have to do?
So remember what star is. Star means you can take as many copies of you lot as you like of strings in the original language, and that's in the star language. So to determine if something is in the star language, you have to see, can I break it up into pieces which are all in the original language?
So you want to see, can I take my input w and cut it up into a bunch of pieces-- four, in this case-- where each of those pieces are members of A, the members of the original language? So that's what M prime's job is. It has its input and wants to know, can I cut that input up into pieces, each of which are accepted by the original machine M? That's what M prime does.
And if you think about it a little bit, really what's happening is that as soon as M-- so M prime is going to be simulating M. That's the way I like to think about this, as having M inside. So if you were going to be doing this yourself, you're going to take w. You're going to run it for a while. You'll see, oh, M is accepted. Now I have to start him over again to see if it accepts the next segment.
So every time M accepts, you're going to restart M to see if it accepts another segment. And so by doing that, you're going to be cutting w up into different segments, each of which is accepted by M. Of course, it's never totally clear whether you should, for any given segment, you should cut it there or you should wait a little longer and find another, a later place to cut. But that's exactly the same problem that we had before with concatenation. And we solved it using nondeterminism, and we're going to solve it again using nondeterminism.
So the way we're going to get that effect of starting the machine over again, once it's accepted, is by adding in epsilon transitions that go from the start states back to-- from the accept state back to the start state. So now every time M has accepted, it has an option-- not a requirement, but has an option. It can either stay continuing to process, or it could restart, making a cut at that point and trying to see if there's yet a second, another segment of the input that it's going to accept.
And this is basically the whole thing, with one little problem that we need to deal with. And that is we need to make sure that M prime accepts the empty string. Because remember, the empty string is always a member of the star language. And as it's written right now, we're going to be requiring there to be at least one copy of at least one segment. We're not taking into account the possibility of no segments, which is the empty string.
And the way we're going to get that is-- well, I mean, one thing, one way to get to add-- so we're missing the empty string right now. So how do we fix it? Basically, we're just going to take the construction we have on the screen, and we're going to adjust it to add in the empty string. Because it's possibly missing.
One way to do that, which is tempting, but wrong, is to make the start state of M an accepting state for M prime. So we could have made this an accepting state, too. And now M prime is also going to accept the empty string. That's the good news.
The problem is that the start date might be playing some other role in M besides just being the start. There might be times when M comes back to the start state later on. And if we make the start state the an accept state, it's going to suddenly start accepting a bunch of other things too, which might not be intended. So it's a bad idea to make the start state an accept state.
Instead, we'll take the simple solution alternative of adding a new start state, which will never be returned to under any circumstances, and make that a new start-- an accept state as well. So here, we'll have to make this additional modification. So as I'm saying, this is what we need to do. And the way we'll do that is by adding a new start state, which is also an accept state, to make sure it accepts the empty string. And then that also can branch to start off M as before, if the string that's input is not the empty string. And so then M prime is actually going to have to do some work to see if it can be cut off, as it was doing before.
So that's the proof of closure under star. I'm not going to do it anything beyond what I've just described. These proofs by picture are convincing enough, I hope. And if not, they are explained in somewhat more detail, somewhat more formally, in the textbook. But for the lecture, this is where I'm going to stop, with these two arguments.
And so now-- oh, we have one quick check-in on this. So if M has n states, how many states does M prime have by this construction? So I'm not intending these to be very hard, more just to keep you awake. So how many states does M prime have? OK, maybe a little too easy even for a check-in. Yeah, everybody is getting this one.
Because all you did was-- we added one new state. So the answer is as you have-- I think pretty much everybody is observing that it's number b. So I'm going to end the polling, and I'm going to share the results. And everybody got that one. And so let's continue on.
And so the very last thing we're going to do today is show you how to convert regular expressions to NFAs, thereby showing that every language that you can describe with a regular expression is a regular language. On Tuesday, we'll show how to do the conversion in the other direction and so thereby showing that these two methods of describing languages are equivalent to one another.
So here's our theorem. If R is a regular expression, and A is the language-- a set of strings that that regular expression describes, then A is regular. OK? So we're going to show how to convert. The strategy is to convert R to an equivalent NFA M. And so we have to think about, remember, these regular expressions that we introduced last time. These are these expressions that look like ab union b star, something like that-- so built up out of the regular operations from the primitive regular expressions that don't have any operations, that we're calling atomic.
So if R is an atomic regular expression, it just looks like either just a single symbol or an empty string symbol or an empty language symbol. Or R can be a composite regular expression-- whoops. We're having a little-- yeah, so we have two possibilities here. R is either atomic or composite. And so let's look at what the equivalent expression is in each case.
So if R is just the single letter regular expression-- that's a totally legitimate regular expression, just a regular expression 1. So that just describes the language of the string 1. So we have to make an NFA which accepts-- which recognizes just that language, accepts only the string 1. So it's a very simple NFA. It just starts in the start state.
And on that single symbol, it branches to an accept state. And there were no other transitions allowed. So if you get anything else coming in besides that one, that string, which is just that one symbol, the NFA is going to reject it. If it's too long, if it gets aa coming in, well, there's nowhere to go from this accepting state on an A. So the machine is just going to die. It has to be in an accept state at the end of the input.
Now, I want you think for yourself for a minute, how do we make an NFA which accepts only the empty string and no other strings? You can do that with just one state with an NFA, just this one here. The machine is going to start off in the start state, which is also immediately an accept state. So it accepts the empty string. But if anything else comes in, there's nowhere to go when the machine dies.
So this machine accepts just the empty string. Or its language is the language with one element, the empty string. How about the empty language? Well, here's an NFA which has no accepting state, so it can't be accepting anything.
Now, if we have a composite regular expression, we're already finished. Because we showed how to build up-- we showed constructions which give us closure under union, concatenation, and star. And those constructions are going to enable us to build up the NFAs that do the language of these more complex regular expressions built up out of the NFAs that do the individual parts.
So if we already have NFAs that do R1 and R2, then the closure under union construction gives us an NFA that does R1 union R2 as a language. So I hope that's clear, but I'm going to do an example which will hopefully illustrate it. And it's going to show you-- basically, what I'm giving you is an automatic procedure for converting a regular expression into an equivalent NFA.
So let's just see that procedure in action, which is really just following this recipe that I described for you. So here is a regular expression a union ab star. So this is a regular expression. It's some language. Whatever it is, I don't care. But I want to make an NFA which recognizes that same language.
And the way I'm going to do that is first build NFA for the components, the subexpressions of this regular expression, and then combine them, using our closure instructions, to be NFAs for larger and larger subexpressions, until I get the NFA that's the equivalent of the entire expression. So let's just see how that goes.
So the very most primitive parts, the smallest subexpressions here, are just the expressions for a and for b. So here's the one just for a. So this is the NFA which recognizes the language, which is just the one string a. Here is the NFA whose language is just the one string b.
And now I want an NFA which accepts only the string ab. Now, of course, you could just do that by hand yourself. It's simple enough. But what I'm arguing is that we can do this automatically, using the closure construction for concatenation. Because really there's a hidden concatenation symbol. This is a concatenate b.
So now for ab, I'm going to take the thing from a and the part from b-- so these two things that I had from before, and use the concatenation construction to combine them. You see that? So now I have automatically an NFA which does the language whose string is just ab, just the ab string. And it's not the simplest NFA. You can make a simpler one, but the virtue of this one is that I got it automatically just by following the closure construction.
So now I'm going to do a more complex one, just the inside here, a union ab. So the way I'm going to build that is from the two parts, the a part and the ab part, the a part and the ab part. So here is the a part. Here's the ab part. I've already got those from before. It's really kind of a proof by induction here. But I think it's simple enough, we don't have to use that language.
So we have the a part, the ab part. And now we are going to apply the closure under union construction to combine those into one machine. And remember how that worked. We had a new symbol here, which branches under empty string to the previous-- we're adding a new start state, which branches to the original start states under empty transition. And now this is an NFA for this language, a union ab.
And lastly, now we're one step away from getting the star of this. And how are we going to do that? We're going to take this thing here and apply the construction for the star closure. And that's going to be an NFA which does a union ab star, which is what we wanted in the first place.
So first, we're going to bring that one down. Because we've already built that one. And now remember how we built the closure under star. We made the accepting states return back to the start state, and we added a new start state to make sure we got the empty string in there that transitioned to the original start state under epsilon. OK?
So that's all I wanted to say for today's lecture. Let's do a quick review. Very important concept, nondeterminism and nondeterministic finite automata-- we proved they were equivalent in power, showed the class of regular languages closed under concatenation in star. We showed how to do conversion of regular expressions to NFAs.
So I think that is it for today's lecture. And thank you, all, for being here. I'll try to answer a few of these.
"Why does concatenation have order?" Well, because it's an ordered construction. Is there a simple way to prove closure under concatenation without using nondeterminism? No.
"Why are the empty strings at the accept state? Can't they be at any state? Doesn't star make copies of any part of the input?" No, it's only-- you have to think about what's going on. You have to branch back to the beginning only on an accept. Because that means you found a piece that's in the original language.
"Is there an automaton that can add some or subtract memory automata?" Well, depends on what you mean by all that. But certainly, there are more powerful machines that we're going to study than finite automata. But yes, there is. And even finite automata can add and subtract, if you present the input in the right way. I would refer you to the first problem on the homework. So I think I'm going to check out then. Take care, everybody. Bye-bye.