The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare
continue to offer high-quality educational resources for free.
To make a donation or to view additional materials
from hundreds of MIT courses, visit MIT OpenCourseWare
at ocw.mit.edu.
CHARLES LEISERSON: So today we're
going to do some really cool stuff
having to do with nondeterministic parallel
programming.
This is where the course starts to get hard.
Because nondeterminism is really nasty.
We'll talk about it a little bit.
It's really nasty.
Parallel computing, as you know, is pretty easy, right?
It's just work and span.
Easy stuff, right?
It makes sense.
You can measure these things, can learn some skills
around them, and so forth.
But nondeterminism is nasty, really nasty.
So first let's talk about what we mean by determinism.
So we say that a program is deterministic on a given input
if every memory location is updated with a sequence--
the same sequence of values in every execution.
So the program always behaves the same.
And you may end up-- if it's a parallel program having
different memory locations updated in different orders--
I may do A and then B, versus updating B and then A--
but if I look at a single memory location, A say,
I'm always updating A with the same sequence of values.
There are lots of definitions of determinism.
This is not the only one.
There are some where people say, well, it only
matters if the output is always the same.
And there are others where you say not only does it
have to be the same but every write to a location
has to be in the same order globally.
That turns out to be actually pretty hard,
because if you have parallel computing
you're not going to get them all updated the same unless you
only have one processor executing instructions.
And so we'll talk about this.
We'll talk a little bit more about this kind of thing.
So why-- what's the big advantage
of deterministic programs?
Why should we care whether it's deterministic or
nondeterministic?
Sure.
AUDIENCE: It's repeatable.
CHARLES LEISERSON: It's repeatable.
OK.
So what?
AUDIENCE: [INAUDIBLE] a lot of programs [INAUDIBLE]..
CHARLES LEISERSON: Why is that?
AUDIENCE: [INAUDIBLE] like a--
Why?
Because sometimes that's what you want.
CHARLES LEISERSON: Because sometimes that's what you want.
OK.
That doesn't-- so if--
I mean, there's a lot of things I might sometimes want.
Why is that important to want that?
Yes.
AUDIENCE: Because consistency makes
it easier to debug source code.
CHARLES LEISERSON: Yes.
Makes it easier to debug.
That's probably the number one reason, debugging.
If it does the same thing every time, then if you have a bug,
you can run it again.
You expect to see the bug again.
So every time you run through, hey, I get the same bug.
But if it's nondeterministic, I get a bug,
and now I go to look for it and the bug is nowhere to be found.
Makes debugging a lot harder.
There are other reasons for wanting repeatability,
so your answer is actually a broader correct answer.
But the big advantage is in the specific application
of repeatability to debugging.
So here's the golden rule of parallel programming.
Never write nondeterministic parallel programs.
They can exhibit anomalous behaviors
and it's hard to debug them.
So never ever write nondeterministic programs.
Unfortunately, this is one of these things that is
kind of hard in practice to do.
So why might you want to write a nondeterministic program
even though--
even when famous masters in the area of performance
engineering, with highly credentialed--
numerous awards and so forth, tell you
you shouldn't write nondeterministic programs?
Why might you want to do it anyway?
Yes.
AUDIENCE: You get better performance.
CHARLES LEISERSON: Yes.
You might get better performance.
That's one of the big ones.
And sometimes you can't.
The nature of the problem is maybe
that it's not deterministic.
You may have asynchronous inputs coming in and so forth.
So this is the golden rule.
We also have a silver rule.
Silver rule says never write nondeterministic parallel
programs, but if you must always devise a test strategy
to manage the nondeterminism.
So this gets into you better have
some way of handling how you're going to tell what's
going on if you have a bug.
So what are some of the typical test strategies
that you could use that would manage the nondeterminism?
So imagine you've got a parallel program
and it's got races in it and so forth,
and it's operating nondeterministically.
What-- and that's OK if everything's going right.
How would you-- you find a bug in the program.
How are you-- what are you going to do?
What kinds of ideas do you have?
Yes.
AUDIENCE: You could temporarily remove the nondeterminism.
CHARLES LEISERSON: Yes.
You could turn off the nondeterminism.
You put a switch in there that says, well,
I know the source of this nondeterministic behavior.
Let me do that.
Let me give you an example of that.
For security reasons these days, when you allocate memory,
it's allocated to different locations
on different runs of the program.
It's allocated in random places.
They want to randomize the addresses when you call malloc.
That means that you can end up with different behaviors
from run to run, and that can compromise your performance.
But it turns out that there is a compiler switch,
and if you run it in debug mode it
will always deliver the results of malloc
in deterministic locations, where
the locations of the things you're mallocing
are repeatable.
So that's good because they're supported.
They said, yes, we have to randomize for security reasons
so that people can't deterministically
exploit buffer overflow errors, for example,
but I don't want to have to do that every time.
So I don't want to randomize every time I run.
I want to have the option of making it
so that that randomization is turned off.
So that's a good one.
What's another one that can be done?
You're full of good ideas.
Let's try somebody else for now.
But I like that, I like that.
What are some other ideas?
What else can you do to handle nondeterminism?
You got a program and it's--
yes, yes, yes.
AUDIENCE: If you use random numbers, use the same seed.
CHARLES LEISERSON: Yes.
If you have random numbers, you use the same seed.
In some sense that's kind of the same thing
if you're turning off nondeterminism.
But that's a great one.
There are other places.
For example, if you read--
if you do get time of day for something in your program
for something, you could have an option where it will put
in a particular fixed value there so you can make sure that
it doesn't--
even a serial program isn't nondeterministic.
So that's good, but I also consider that to be-- it's
another great example of turning off and on determinism.
What other things can you do?
Yes.
AUDIENCE: You could record the randomized outputs or inputs
to determine correctness.
CHARLES LEISERSON: Yes.
You can do record-replay for some things.
Is that what you're saying?
Is that what you mean?
Or am I--
AUDIENCE: Maybe.
[INAUDIBLE]
CHARLES LEISERSON: So record-replay says you run it
through-- you can run it through with random numbers,
but it's recording those things, so that when you run it again,
instead of using the random numbers--
new random numbers, it uses the ones that you used to use.
So that's the record-replay thing.
Is that what you're saying, or are you saying something else?
Yes, OK, good.
So that's using some tools.
There are actually a lot of strategies.
Let me just move on and answer.
So another thing you can do is encapsulate the nondeterminism.
So that's actually done in a Cilk runtime system already.
The runtime system is using a random scheduling strategy,
but you don't see that it's random in the execution
of your code if you don't-- if you have no race conditions
in your code.
It's encapsulated.
So that the-- in the platform.
So the platform is going to guarantee
you deterministic results even though underneath the covers
it's doing nondeterministic things.
You can also substitute a deterministic alternative.
Sometimes there's a way of computing something
that is nondeterministic, but in debug mode,
ah, let me not use the nondeterministic one.
And you can also use analysis tools,
which can tell you things about your program
and which you can control things.
So there's a lot of things.
So whenever you have a nondeterministic program,
you want to find some way of controlling it.
Often, the nondeterminism is over in this corner
but your bug is over in this corner.
So if you can turn this thing off in some way,
or encapsulate it, or otherwise control
the nondeterminism over there, now you
have a better chance of catching the stuff over here.
That's going to be particularly important in project 4
when we get to it, because that's
going to be actually going to be doing
nondeterministic programming for a game playing program.
And one of the things is that the processors
are, in this case, keeping the game positions together.
And so if one processor stores something
into what's called a transposition table, which
is essentially a big hash table of positions it's seen,
another one can see that value and change its behavior.
And so one of the things you want to be do
is turn off transposition table so that you
don't take advantage of that performance advantage,
but now you can debug the search code,
or you can debug the evaluation code, and so forth.
You can also do things like unit testing
so you know whether or not a particular piece is correct
that might have--
so that you can test this thing separately
from the rest of your system which may have nondeterminism.
Anyway, this is a major thing.
So never write them.
But if you have to, you always want
to have some test strategy.
And so for people who are not watching this video
and who are not in class today, they
are going to be sorely hampered by not
knowing this lesson when they go into the fourth project.
So what we're going to do is now we're
going to talk about how to do nondeterministic programming.
So this is-- there's always some part of your code
which has a skull and crossbones.
Like you have this abstraction.
It's beautiful, and you can design, et cetera.
And then somewhere there's this really ugly thing
that nobody should know, and you put the skull and crossbones
on that, and only experts go in.
Well, anyway, that's the barrier we're crossing here.
And we're going to start out by talking about something
that you've probably seen in some
of the other classes, mutual exclusion and atomicity.
So I'm going to use the example of a hash table.
So here's a typical hash table.
It's got collisions resolved by chaining.
So you have a bunch of linked lists.
You hash to a particular slot in the table,
and then you chase down the linked list to find the value.
And so, for example, if I'm going
to insert x which has a key value of 81,
what I do is figure out which slot
I go to by hashing the key.
And then in this case I made it be the last one
so that the animations could be easier
than if it were in the middle.
So now what do I do is I make the pointer of x
go to the first element of that list,
and then I make the slot value now point to x.
And that effectively, with a constant number of operations,
inserts x into the hash table, and in particular
into the linked list in the slot that it's supposed to be.
This is all familiar, right?
So now what happens when you have
multiple parallel instructions that are
accessing the same locations?
So here we have two threads, one inserting
x and one inserting y.
And x goes, it does its thing.
It hashes to there, and it then sets the next pointer
to be the--
to add itself into the list.
And then there's this other thing
going on in parallel which effectively says, oh, I'm
going to hash.
Oh, we're going to the same slot.
It doesn't know that somebody is already there.
And so then it decides it's going
to put itself in as the first element of the list.
And then it sets the value of y--
it sets the value of the slot to point to y.
And then along comes x, finishing off what it's doing,
and it points the value to x.
And you can see that we have a race bug here, a really nasty
one because we've just destroyed the integrity of our system.
We now have-- in particular, y is sort of floating,
not in the list when it's supposed to be in the list.
So the standard solution to this is
to make some of these instructions be atomic.
And what that means is the rest of the system
can never view them as being partially executed.
So they either all have been executed or none of them
have been executed at any point in time
as far as the rest of the system is concerned.
And the part of code that is within the atomic region
is called the critical section.
And, typically, a critical section of code
is some place that should not be being executed
by two things at the same time.
So the standard solution to atomicity
is to use what's called a mutex lock, or a mutual exclusion
lock.
And it's basically an object with a lock and unlock member
functions.
And an attempt by a thread to lock an already locked mutex
causes the thread to block--
that is, wait-- until the mutex is unlocked.
So if somebody grabs the lock, somebody else grabs the lock
and it's already taken, then they have to wait.
And they sit there waiting until this guy says,
yes, I'm going to release it.
So what we'll do now is we'll make
each slot be a struct with a mutex L, and a pointer, head,
to the slot context.
So it's going to be the same data
structure we had before but now I'm
going to have not just the pointer from the slot
but I'll also have a--
also have a lock in that position.
And so the idea of--
in the code now is that before I access the lock--
before I access the list, I'm going
to lock that list in the table by locking slot.
Then I'll do the things that I need to do,
and then I'll unlock it, and now anything else can go on.
Because what's happening is-- the reason
we're getting into trouble is because we've
got some sort of interleaving of operations.
And our goal is to make sure that it's
either doing this or doing this, and never
this, to make sure that--
so that each thing, each piece of code,
is restoring the invariant of correctness after it executes
the pointer swaps.
The invariance in this case is that the elements
are in a list.
And so you want to restore that with each one.
So mutexes-- this is one way you can use
mutexes to implement atomicity.
So now let's just go back.
Who has seen mutexes before?
Is that pretty much everybody?
Yes.
OK, good.
I hope that this is not brand new for too many of you.
If it is brand new, that's great.
But what I'm trying to do is make it-- so
let's go back a little bit and recall in this class
our discussion of determinacy races.
So, remember, a determinacy race occurs
when you have two logically parallel instructions
that access the same memory location and at least one
of them performs a write.
So mutex locks can guarantee that critical sections behave
atomically, but the resulting code is
inherently nondeterministic because you've got a--
we had a race bug there.
We had two things trying to access the same slot.
But that may be what I want.
I want to have a shared hash table maybe for these things.
So I want something where there is a race,
but I just don't want to have the anomalies that arise.
In this case, the race bug caused things,
and I can solve that with atomicity.
If you have no determinacy races,
it means that the program is deterministic on that input
and that it always behaves the same.
And remember also that if a deterministic race exists
in an ostensibly deterministic program, then
it guarantees to find a race.
Now, if you put in mutexes, you still
have a nondeterministic program.
You still have a race.
Because you have two things that are logically
parallel that are both accessing the lock.
That's a race.
That's a determinacy race.
If you have two things, they're in parallel,
they're both accessing the lock, that's a determinacy race.
It may be a safe, correct one, but it is a determinacy race.
And so any codes that use locks are
nondeterministic by intention, and they're
going to invalidate the Cilksan guarantee of finding those race
bugs.
So you will end up with races in your code
if you're not careful.
And so this is one reason it's important to have
some way of turning off nondeterminism to detect stuff.
Because what you don't want is a whole rash
of false positives saying, oh, you
raced on gathering this lock.
Nor do you want to ignore that and then discover
that a race has popped up somewhere else.
Now, some people feel that--
so this is basically talking about having a data race.
And a data race is similar to the definition
of determinacy race, but it says that you
have two logically parallel instructions
and they don't hold locks in common.
And then it's the same definition.
If they access the same memory location and one of them
performs a write, then you have a--
then you have a data race bug.
But if they have the locks in common,
if they both have acquired at least one lock that's the same,
then you don't have a data race, because that
means that you've now successfully protected
the atomicity.
But it is still nondeterministic and there
is a determinacy race, just no data race.
And that's the big distinction between data races
and determinacy races.
And on quiz 2, you better know the difference
between data races and determinacy races,
because they are different.
So a program may have no determine--
may have no data races.
That doesn't mean that it doesn't
have a determinacy race.
In fact, if it's got any locks, it probably
has a determinacy race.
So one of the things is, if I have no data races,
does that mean I have no bugs?
Suppose I have no data races in my code.
Does that mean I have no bugs?
This is like an obvious answer just by quizmanship, right?
So what might happen?
Think about it a little bit.
What might happen?
How could I have no data races and yet there still
be a bug, even though--
I'm assuming it's a correct piece of code otherwise.
In other words, when it runs serially or whatever,
it's correct.
How could I end up having a code-- no data races but still
have a bug?
AUDIENCE: It's still nondeterministic [INAUDIBLE]..
CHARLES LEISERSON: Yes, but that doesn't mean it's bad, right?
AUDIENCE: Well, you said that it runs correctly serially.
CHARLES LEISERSON: Yes.
AUDIENCE: So the order that things are put in or generated
might still be--
CHARLES LEISERSON: Might still be different, yes.
AUDIENCE: [INAUDIBLE].
CHARLES LEISERSON: OK.
Yes.
Let me give you an example which is more to the point.
Here is a way of making sure that I
have no data race, which is I lock before I follow the table
slot value.
Then I unlock, and I lock again and then I set the value.
So I haven't prevented the atomicity.
Right now I've got an atomicity violation,
but I have no data races, because I never
have two things-- any two things that
are going to access things at the same time
is protected by the lock.
But it didn't solve my atomicity, so there's a--
you can definitely have no data races,
but that doesn't mean you have no bugs.
But, usually, what happens is, if you have no data races,
then usually the programmer actually got this code right.
It's one of these things where demonstrating no data races
is in fact a very positive thing in your code.
It doesn't mean the programmer did right.
But most of the time, the reason they're putting in the locks
is to provide atomicity for something,
and they usually get it right.
They don't always get it right.
In fact, Java, for example, had a very famous bug
early on in the way that it specified
locking such that the--
you could look at the length of a string and then modify it,
and then you would end up with a race bug
because somebody else could swoop in in between.
So they thought they were providing atomicity and they
didn't.
So there's another set of issues here
having to do with benign races.
Now, there's some people who argue that no races are--
no determinacy races are benign.
And they make academic statements
that I find quite compelling, actually,
what they say, about races and whether races are benign.
But, nevertheless, the literature
also continues to use the term benign race
for this kind of example.
So suppose we want to identify what is the set of digits
that occurred in some array.
So here's an array with a bunch of values in it,
each one being a digit from 0 to 9.
So I could write a little piece of code
that runs through a digits array of length 10
and sets the number of digits I've seen so far of each value
to be 0.
And now I go through--
and I'm going to do this in parallel--
and I'm going to set, every time I see a value A of i--
suppose A of i is 3--
I set the location of A3 to be 1.
And, otherwise, and now-- otherwise,
it's 0 because that's what I had it before.
So here's the kind of thing I have.
So, for example, I can have both of those 6's--
or in parallel, we're going to access the location
6 to set it to 1.
But they're both setting it to 1.
It doesn't really matter what order they do it in.
You're going to get the same value there, 1.
And so there's a race.
Maybe we don't too much care about that race,
because they're both setting the same value.
We're not going to get an incorrect value.
Well, not exactly.
We might get it on some architecture.
On the Intel architectures, you won't get an incorrect value,
on x86.
But there are codes where the elements--
the array values are not set atomically.
So, for example, on the MIPS architecture,
in order to set a bite to be a particular value,
you have to fetch the word, mask out, set the word,
and then store it back in.
Set the byte and then store it back into the word.
And so if there are two guys who are basically
operating on that same word location,
they will have a race, even though in the code
it looks like they're just setting bytes.
Does that make sense?
So nasty.
Nasty bugs.
That's why you should never do nondeterministic programming
unless you have to.
So Cilksan allows you to turn off race detection
for intentional races.
So if you really meant there to be a race, as in this case,
you can turn it off.
This is dangerous but practical, it turns out.
Usually you're not turning it off
for-- because here's what can happen.
You can turn it off and yet--
then there's something else which is using that same stuff,
and now you're running Cilksan without having turned it off
for exactly what your race might be.
There are better solutions.
So in Intel's Cilk Screen, there's
the notion of fake locks.
We just have not yet implemented it in the open Cilk compiler
and in Cilksan.
We'll eventually get to doing that.
And then people who take this class in the future
will have an easier time with that, because we'll be
able to check for that as well.
So any questions about these notions?
So you can see the notions of races can get quite hairy
and make it quite difficult to do your debugging,
and in fact even can confound your tools that
are supposed to be helping you to get correct code.
All in the name of performance.
But we like performance.
Any questions?
Yes.
AUDIENCE: So I don't really understand
how some architectures can cause some error in race conditions.
CHARLES LEISERSON: Yes.
So how can some architectures cause some error?
So here's the thing, is that if I
have a, let's say, a byte array, it
may be that this is stored as a set of let's say
four-byte words.
And so although you may write that A of 0
gets 1, what it does is it says, let me fetch these four values,
because there is no byte set instruction
on some architectures.
It can only set, in this case, 32-bit words.
So it fetches the values.
It then-- into a register.
It then sets the value in the register by masking.
So it doesn't set the other things here.
And then it stores it back so that it has a 1 here.
But what if somebody, at the same time,
is storing into this location?
They will fetch it into their own register,
set their byte, mask it, et cetera.
And now my writeback is going to--
we're going to have a lost update in the writebacks.
Does that make sense?
AUDIENCE: [INAUDIBLE].
CHARLES LEISERSON: OK.
Good.
Very good question.
Yes, I know.
I went through that orally a little bit quicker
than maybe I should have.
Great.
So let's talk a little bit about implementation.
I always like to take things down one level
below what you necessarily need to know in order to do things.
But it's helpful to sort of see how these things are
implemented, because then that gives you a better
sense at a higher level what your capabilities are
and how things are actually working underneath.
So let's talk about mutexes.
So here, first of all, understand there
are lots of different mutexes.
If you look at an operating system,
they may have a half a dozen or more different mutexes,
different locks that can provide mutual exclusion,
or parameters that can be set for what kind of mutexes.
So the first basic difference in most things
is whether the mutex is yielding or spinning.
So a yielding mutex returns control to the operating system
when it blocks.
When a program tries to get--
when it tries to get access, when
a thread tries to get access to a given lock, if it is blocked,
it doesn't just sit there and keep--
and spinning, where you're basically-- spinning
means I just sit there checking it and checking it and checking
it and checking it.
Instead what it does is it says, oh, I'm
doing useless work here.
Let me go and return control to the operating system.
Maybe there's another thread that can run at the same time,
and therefore I'll give--
by switching myself out, by yielding my scheduling quantum,
I will get better efficiency overall,
because somebody-- some other thread that
is capable of running can run at that point.
So is that a clear distinction between spinning and yielding?
Another one is whether the mutex is reentrant or nonreentrant.
A reentrant mutex allows a thread
that is already holding a lock to acquire it again.
A nonreentrant one deadlocks if the thread
attempts to require a mutex it already holds.
So I grab a lock, and now I go to a piece of code
that says grab that lock.
So very simple.
I can check to see whether I have--
if I want to be reentrant, I can check,
do I have that lock already?
And if I do, then I don't actually have to acquire it.
I just keep going.
But that's extra overhead.
It's faster for me to have a nonreentrant lock,
where I just simply grab the lock,
and if somebody has got it, including me,
then it's a deadlock.
But now if there's the possibility
that I could reacquire a lock, then that might not be safe.
You have to worry about-- the program has
to worry about that now.
Is that clear, that one?
And then a final basic property of mutexes
is whether they're fair or unfair.
So here's the thing.
It's the easiest to think about it in the context of spinning.
I have several threads that basically
came to the same lock, and we decided they're going to spin.
They're just going to sit there continually checking, waiting
for that lock to be free.
So when finally the guy who has it unlocks it,
maybe I've got a half a dozen threads sitting there.
One of them wins.
And which one wins?
Well, they're spinning.
It could be any one of them.
Then it has one.
And so the issue that can go on is
you could have what's called a starvation problem, where
some guy is sitting there for a really long time waiting
while everybody else is continually grabbing locks
out from under his or her nose.
So with a fair mutex, basically what you do
is you go for the one that's been waiting the longest,
essentially.
And so, therefore, you never have
to wait more than for however many things were there
when you got there before you're able to go.
Question.
AUDIENCE: Why is that better?
CHARLES LEISERSON: It can be better because you may freeze
out our service if there's something that's-- you may
never get to do the thing that you want to do
because there's something else always interfering with
the ability for that part of the program to make progress.
This tends to be more of an issue
in concurrent programming, where you
have different programs that are trying
to accomplish different tasks and you
want to accomplish both tasks.
It does not come across--
in parallel programming, mostly we deal with unfair--
often unfair spinning locks because they're the cheapest.
And we just trust that, a, we're not
going to have any critical regions-- we write
our code so we don't have critical regions that
are really long, so nobody ever has to wait a very long time.
But, indeed, dealing with a contention issue,
as we talked about last week, can make a difference.
good.
So here's an implementation of a simple spinning mutex
an assembly language.
So the first thing it does is it checks
to see if the-- the mutex is free if its value is 0.
So it compares the value of the mutex to 0.
And if it is 0, it says, oh, it's free.
Let me go get it.
It then-- to get the mutex, what it does is it moves a 1
into the--
it basically moves 1 into a register,
and then it exchanges the mutex with that register eax.
And then it compares to see whether or not
it actually got the mutex.
And if it didn't, then it goes back up to the top
and starts again.
And then the other branch is at the top there.
It does this pause, and this apparently
is due to a bug in x86 that they end up
having to put this pause instruction in there.
And then, otherwise, you jump to where
the Spin_Mutex is and go again.
And then, once you've done the Critical_Section,
when you're done you free it by just setting it to 0.
So the question here is-- so the exchange instruction
is an atomic exchange.
So it takes the register and the memory value and it swaps them,
and you can't have anything come in.
So one of the things that might have you
confused a little bit here is, wait a second.
I checked to see if the mutex is free,
and then I tried to get it to test if I was successful.
Why?
Why can't I just start out by essentially going to get mutex?
I mean, why do I need any of the code between Spin_Mutex
and Get_Mutex?
So if I just started with Get_Mutex, I would move a 1 in.
I would exchange, check to see if I could get it.
If I had it, fine.
Then I execute the end.
If not, I would go back and try again.
So why-- because if somebody has it, by the way,
the value that I'm going to get is going to be 1.
And that's what I swapped in, so I haven't changed anything.
I go back and I check again.
So why do I need that first part?
Yes.
AUDIENCE: Maybe it's faster to just get [INAUDIBLE]..
CHARLES LEISERSON: Yes.
Maybe it's faster.
So, indeed, it's because it's faster.
Even though you're executing extra code, it's faster.
Tell me why it's faster.
And this will take you-- you have
to think a little bit about the cache protocols
and the invalidation issue.
So why is it going to be faster?
Yes.
AUDIENCE: Because I do the atomic exchange.
CHARLES LEISERSON: OK, good.
Say more.
AUDIENCE: Basically, just to exchange atomically,
you have to have [INAUDIBLE].
And you bring it in only just to do a swap.
CHARLES LEISERSON: Yes.
So it turns out the exchange operation is like a write.
And so in order to do a write, what do I
need to do for the cache line that it's on?
AUDIENCE: To bring it in.
CHARLES LEISERSON: To bring it in.
But how does it have to be brought in?
Remember, the cache lines have--
let's ima--
AUDIENCE: [INAUDIBLE].
CHARLES LEISERSON: You have to invalidate on the other ones,
and you have to hold it in what state?
Remember, the cache lines have--
if we take a look at just a simplified protocol where--
the MSI's protocol.
AUDIENCE: [INAUDIBLE].
CHARLES LEISERSON: Yes.
You have to have it--
in MSI or MESI, you have to bring it in in modified
or at least exclusive state.
So exclusive is for the MESI protocol.
We mentioned that but we didn't really do it.
Mostly we just went--
but I have to bring it in and modify it,
where I guarantee there are no other copies.
So if I've got two guys that are polling on this location,
they're both continually invalidating each other,
and you create a whole bunch of traffic on the memory network.
That's going to slow everything down.
Whereas if I do the first one, what state do I get it in?
AUDIENCE: [INAUDIBLE].
CHARLES LEISERSON: Then you get it in shared state.
What does the other guy get it in?
AUDIENCE: Shared.
CHARLES LEISERSON: Shared state.
And now I keep going, just having
it spinning in my own local cache,
not generating any local traffic until the--
until somebody releases the lock, in which case
it invalidates all those.
And now you can actually get a little bit of a storm
after the fact.
There are in fact locks where you don't even
get a storm after the fact called MCS locks.
But this kind of lock is, for most practical purposes,
just fine.
So everybody follow that description
of what's going on there?
So that first code, for correctness purpose,
is not important.
For performance, it is important.
Isn't it great that you guys can read assembly language?
Now suppose that-- this is a spinning mutex.
Suppose that I want to do a yielding mutex.
How does this code have to change?
So this is a spinning one.
It just keeps checking.
Instead, I want to return control
to the operating system.
So how does this code change if I do that?
Yes.
AUDIENCE: Instead of the pause, [INAUDIBLE]..
CHARLES LEISERSON: Like that.
Yes, exactly.
So instead of doing that pause instruction, which--
the documentation on this is not very clear.
I'd love to have the inside scoop on why they really
had to do the pause there.
But in any case, you take that no op
that they want to have in there and you replace it
with just a call to the yield, which allows the operating
system to schedule something else.
And then when it's your turn again,
it resumes from that point.
So that's the yield.
So that's the difference in implementation,
essentially, between a spinning mutex and a yielding mutex.
Now, there's another kind of mutex
that is kind of cool which is called a competitive mutex.
So think about it this way.
I have competing goals.
One is I want to get the mutex as quickly as possible
after it's released.
I don't want-- if it's unlocked, I
don't want to sit there for a really long time
before I actually acquire it.
And, two, yes, but I don't want to sit there spinning
for a really long time.
And then-- because as long as I'm doing that,
I'm taking up cycles and not accomplishing anything.
Let me turn it over to some other thread that can use
the cycles more effectively.
So there are those two goals.
How can I get the best of both worlds here?
Something that's close to the best of both worlds.
It's not absolutely the best of both worlds,
but it's close to the best of both worlds.
What strategy could I do?
So I want to claim it very soon.
So the point is that the spinning mutex
achieves goal 1, and the yielding mutex achieved goal 2.
So how can I-- what can I do to get both goals?
Yes.
AUDIENCE: [INAUDIBLE] you could use some sort of message
passing to [INAUDIBLE].
CHARLES LEISERSON: So you're saying
use message passing to inform--
AUDIENCE: The waiting threads.
CHARLES LEISERSON: --the waiting threads.
I'm think of something a lot simpler in this context.
Because the message passing, you're
going to have to go through--
to do message passing properly, you actually
need to use mutexes that are to implement it.
So you want to be a little bit careful about that.
But interesting idea.
Yes.
AUDIENCE: Could you try using an interrupt?
CHARLES LEISERSON: Using an interrupt.
How would you do that?
AUDIENCE: Like once the [INAUDIBLE]..
CHARLES LEISERSON: Yes.
So, typically, if you implement interrupt
you also need to have some mutual exclusions to do it
properly, but--
I mean, hardware will support that.
That's pretty heavy-handed as well.
There's actually a very simple solution.
I'm seeing familiar hands.
I want to see some unfamiliar hands.
Who's got an unfamiliar hand?
I see.
You raised your left hand that time
instead of your right hand.
Yes.
AUDIENCE: You try to have whichever
one is closest to being back to the beginning of the cycle
take the lock.
CHARLES LEISERSON: Hard to measure that, right?
How would you write code to measure that?
Yes.
Hmm.
Yes.
Go ahead.
AUDIENCE: I have a question, actually.
CHARLES LEISERSON: OK, good.
AUDIENCE: Why does it [INAUDIBLE]??
CHARLES LEISERSON: Why doesn't it have a?
AUDIENCE: [INAUDIBLE].
Why does yielding mutex [INAUDIBLE]??
CHARLES LEISERSON: Because if I yield--
so what's the-- how often does--
if I context switch, how often is it going to be that I--
how long am I going to have to wait, typically,
before I am scheduled again?
When a code yields to the operating system,
how often does the operating system normally
do context switching?
What's the rate at which it context switches
for the different multiplexing of threads
that it does onto the available processors?
What's the rate at which it shifts?
Oh, this is--
OK, that's going to be on the quiz.
This is a numeracy thing.
Yes.
Do you know how frequently?
AUDIENCE: [INAUDIBLE] sub-millisecond [INAUDIBLE]..
CHARLES LEISERSON: Not quite, but you're
not off by more than an order of magnitude.
So what are the typical rates that the system
does context switching?
So in human time, it's the blink of an eye.
So it's actually around 10 milliseconds.
So it does a hundred times a second.
Some of them do.
Some do 60 times a second.
That's how often it switches.
Now, let's say it's a hundred times a second, 10
milliseconds.
So you're pretty close.
10 milliseconds.
How many orders of magnitude is that from the execution
of a simple instruction?
So we're going at more than a gigahertz.
And so a gigahertz is 10 to the ninth,
and we're talking 10 to the minus 9,
and we're talking 10 to the minus 2.
So that's 10 million instruction opportunities
that we miss if we switch out.
And, of course, we'd probably only switch out for half our--
where are you along the thing.
So you're only switching out maybe for half,
assuming nothing else is going on there.
But that means you're not grabbing the lock quickly
after it's released, because you've
got 10 million instructions that are going to execute
before you're going to have a chance to come back in and grab
it.
So that's why a yielding one does not grab it quickly.
Whereas spinning is like we're executing this stuff
at the rate of gigahertz, checking again, checking again,
checking again.
So why-- so what's the strategy here?
What can I do?
Yes.
AUDIENCE: Maybe we could spin for a little bit
and then yield.
CHARLES LEISERSON: Hey, what a good idea.
Spin for a while and then yield.
So the idea being, hey, if the lock is released soon,
then I will be able to grab it immediately
because I'm spinning.
If it takes a long time for the lock to yield,
well, I will yield eventually.
So yes, but how long to spin?
How long shall I spin?
Sure.
AUDIENCE: Somewhere close to the amount of time
it takes to yield and come back.
CHARLES LEISERSON: Yes.
Basically as long as a context switch takes, as long
as it takes to go out and come back.
And if you do that, then you never
wait more than twice the optimal time.
This is competitive analysis, which the theoreticians have
gone off-- there's brilliant work in competitive analysis.
So the idea here is that if the mutex is
released while you're spinning, then this strategy is optimal.
Because you just sat there spinning,
and as soon as it was there you got it on the next cycle.
If the mutex is released after the yield,
you've already spun for the equal to that.
So you'll come back and get it within at most a factor of 2.
This is-- by the way, this shows up
in the theory literature, if you're interested,
is it's called the ski rental problem.
And here's the idea.
You're going to go--
your friends have persuaded you to go try skiing.
Snow skiing, right?
Pu-chu, pu-chu, pu-chu.
Right?
And so you say, gee, should I buy the equipment
or should I rent?
After all, you may discover that you rent and then--
you buy it, and then you break your leg
and never want to go back.
Well, then, if you've bought it's been very expensive.
And if you've rented, well, then you're probably better off.
On the other hand, if it turns out you like it,
you're now accumulating the costs going forward.
And so the question is, well, what's your strategy?
And the idea is, well, let's look at what renting costs
and what buying costs.
Let me rent until it's equal to the cost of buying
and then buy.
And then I'm within a factor of 2
of having spent the optimal amount of money for--
because then if I break my leg after that, well, at least I--
I got-- I didn't spend more than a factor of 2.
And if I get it before, then I've spent optimally.
Yes.
AUDIENCE: So when you say how long a context switch takes,
is that in milliseconds or--
CHARLES LEISERSON: Yes.
10 milliseconds.
Yes.
So spin for 10 milliseconds, and then switch.
So now the point is that when you come back in,
the other job's going to run for 10 milliseconds or whatever.
So if you get switched out, then if the lock is released,
you're going to be done in 20 milliseconds.
And so you'll be within a factor of 2.
And if it happened if the lockout released before then,
you're right there to grab it.
Now, it turns out that there's a really clever
randomized algorithm--
I love this algorithm--
from 1994 that achieves a competitive ratio
of e over e minus 1 using a randomized strategy.
And I'll encourage you, those of you
have a theoretical bent, to go take a look at that.
It's very clever.
So, basically, you have some probability of,
at every step, of whether you, at that point,
decide to yield or continue spinning.
And by using a randomized strategy,
you can actually get this to e over e minus 1.
Questions about this?
So this is sort of some of the basics.
I'm glad we went over some of that,
because everybody should know these basic numbers about what
things cost.
Because, otherwise, you don't know where to spend it.
So context switching time is on the order of 10 milliseconds.
How long is a disk access compared to--
yes.
What's a disk access?
AUDIENCE: 150 cycles?
CHARLES LEISERSON: 150 cycles?
Hmm, that's a--
AUDIENCE: Or is that the cache?
CHARLES LEISERSON: That would be accessing DRAM.
Accessing DRAM, if it wasn't in cache, might be 150 cycles.
So two orders of magnitude or so.
So what about a disk access?
How long does that take?
Yes.
AUDIENCE: Milliseconds?
CHARLES LEISERSON: Yes.
Several milliseconds.
So 10 milliseconds or 5 milliseconds depending
upon how fast your disk is.
But, once again, it's on the order of milliseconds.
So it's helpful to know some of these numbers,
because, otherwise, where are you spending your time?
Especially, we're sort of doing performance engineering
in the small, basically looking within the pro--
within a multicore processor.
Most performance engineering is on all the stuff
on the outside, dealing with networking, and file systems,
and stuff where things are really costly,
and where, if you actually have a lot of time,
you can write a fast piece of code that can figure out
how you should best deal with these slow parts
of your system.
So those are all sort of good numbers to know.
You'll probably see some of them on quiz 2.
Deadlock.
I mentioned deadlock earlier.
Let's talk about what deadlock is and understand this.
Once again, I expect some of you have seen this,
but I still want to go through it because it's
hugely important material.
And this is the issue, that holding more than one lock
at a time can be dangerous.
So imagine that thread 1 says, I'm going to lock A, lock B,
execute the critical section, unlock B, unlock A, were A
and B are mutexes.
And thread 2 does something very similar.
It locks B and locks A. Then it does the critical section,
then it unlocks A and then unlocks
B. So what can happen here?
So thread 1 locks A, thread 2 locks
B. Thread 1 can't go and lock B because thread 2 has it.
Thread 2 can't go and lock A because thread 1 has it.
So they sit there, blocked.
I don't care if they're spinning or yielding.
They're not going anywhere.
So this is the ultimate loss of performance.
It's like-- it's incorrect.
It's like you're stuck, you've deadlocked.
Now, there's three basic conditions for deadlock.
Everybody understands this, right?
Is there anybody who has a question, because just--
OK.
There's three conditions you need for deadlock.
The first one is mutual exclusion,
that you're going to have exclusive control
over the resources.
The second is nonpreemption.
You don't release your resources.
You hold until you finish using them.
And three is circular waiting.
You have a cycle of threads, in which each thread is
blocked waiting for resources held by the next one.
In this case, the resource is the lock.
And so if you remove any one of these constraints,
you can come up with solutions that won't deadlock.
So, for example, it could be that when
I try to acquire a lock, if somebody else has them,
I take it away.
That could be one thing.
Now, they may get into other issues, which is like, well,
but what if he's actually doing real work or whatever?
So all of these things have things.
Or I don't insist that it be mutual exclusion, except that's
the kind of problem that we're trying to solve.
So these are generally the three things
that are necessary in order to have a deadlock situation.
Now, in any discussion of deadlock,
you have to talk about dining philosophers.
When I was an undergraduate--
and I graduated in 1975 from Yale, a humanities school--
I was taught the dining philosophers,
because, after all, philosophy is what
you find at humanities schools.
I mean, we have a philosophy department too.
Don't get me wrong.
But at Yale the humanities is huge.
And so philosophy, I guess they thought
this would appeal to the people who were not
real techies in the background.
I sort of like--
I was a techie in the midst of all these non-technical people.
Dining philosophers is a story of deadlock
told by Tony Hoare based on an examination question
by Edsger Dijkstra.
And it's been embellished over the years
by many, many, many retellers.
And I like the Chinese version of this.
There's versions where they use forks, but I'm going to--
this is going to be-- they're dining--
I'm going to say that they are eating noodles with chopsticks.
And there are n philosophers seated around the table,
and between every plate of noodles there's a chopstick.
And so in order to eat the noodles
they need two chopsticks, which to me sounds very natural.
And so here's the code for philosopher i.
So he's a philosopher, so he starts by thinking for a while.
And then he gets hungry, he or she gets hungry.
So the philosopher grabs the chopstick on the right--
on the left, sorry.
And then he grabs the one on the right, which is i plus 1.
But he has to do that mod n, because if it's the last one,
you've got to go around and grab the first one.
Then eats, and then it unlocks the two chopsticks.
And now they can be used by the other dining philosophers
because they don't think much about sanitation and so forth.
Because they're too busy thinking, right?
But what happens?
What's wrong with this solution?
What happens?
What can happen for this?
It's very simple.
I need two chopsticks.
I grab one, I grab the other, I eat.
One day, what happens?
Yes.
AUDIENCE: Everyone grabs the chopstick to the left
and they're all stuck with one chopstick.
CHARLES LEISERSON: Yes.
They grab one to the left, and now they go to the right.
It's not there, and they starve.
One day they grab all the things,
so we have the starving philosophers problem.
So motivated by this problem-- yes, question.
AUDIENCE: Is there any way to temporarily unlock it?
Like the philosopher could just hand the chopstick [INAUDIBLE]..
CHARLES LEISERSON: Yes.
So if you're willing to preempt, then that would be preemption.
As I say, it's got to be nonpreemptive in order
for deadlock to occur.
In this case, yes.
But you also have to worry in those cases.
Could be, oh, well if I couldn't get both,
let me put them both down.
But then you can have a thing that's called livelock.
So they all pick up their left.
They see the right one's busy, so they put it down
so somebody else can have it.
They look around.
Oh, OK.
Let me pick up one.
Oh, no.
OK.
And so they still starve even though they've done that.
So in that kind of situation, you could put in a time delay.
You could say-- let everybody pick a random number to have
a randomized scheme so that we're not--
so there are other solutions if you
don't insist on nonpreemption.
I'm going to give you one where we have nonpreemption
but we still avoid deadlock, and it's
to go for that cyclic problem.
So here's the idea.
Suppose that we can linearly order the mutexes.
So I pick some order of the mutexes,
so that whenever a thread holds a mutex L sub i
and attempts to lock another mutex L sub j,
we have that in this linear order--
L sub i comes before L sub j.
Then you can't have a deadlock.
So in this case, for the dining philosophers,
it would, for example, number the chopsticks from 1 to n,
or 0 to n minus 1, whatever.
And then grab the smaller one and then grab the larger one.
And then it says then you would never have a deadlock.
And so here's the proof.
You know I like proofs.
Proofs are really important.
So I'm going to show you that if you do that, you couldn't
have a cycle of waiting.
So suppose you had a cycle of waiting.
We're in a situation where everybody
is holding chopsticks, and one of them
is waiting for another one, which is waiting for--
all the way around to the first one.
That's what we need for deadlock to occur.
So let me just look at what's the largest mutex on the cycle.
Let's call that L max.
And suppose that it's waiting on mutex L held by the next thread
in the cycle.
Well, then, we have something that's
bigger than the maximum one.
And so that contradicts the fact that I grab them-- whenever
I grab them, I do it in order.
So very simple-- very simple proof that you can't have
deadlock if you grab them according to a linear order.
And so for this particular problem,
what I do is, instead of grabbing
the one on the left and one the right, as I say,
you grab the smaller of the two and then grab
the larger of the two.
And then you're guaranteed to have no deadlock.
Does that make sense?
Now, if you're going to use locks in Cilk,
you have to realize that in the operating--
in the runtime system of Cilk, they're doing--
they're using locks.
You can't see them.
They're encapsulated, as we talked about.
The nondeterminism in Cilk is encapsulated.
It's still going on underneath the covers.
And if you start introducing your own nondeterminism
through the use of locks you can run into trouble
if you're not careful.
And let me give you an example.
This is a situation-- you can deadlock your program in Cilk
with just one lock.
So here's an example of a code that does that.
So main spawns off foo.
And foo basically locks the lock L and then unlocks it.
And, meanwhile, after it spawns off foo,
the continuation goes and it locks L itself,
and then does a sync, and then it unlocks it.
So what happens here?
We sort of have a situation like this,
where the locking I've done with an open bracket,
and an unlock, a release, I'm doing with a closed bracket.
So I'm spawning off foo, which is the lower part there,
and locking and unlocking.
And up above unlocking then unlocking.
So what can happen here?
I can go and I basically spawn off the child, but then I lock.
And now the child goes and it says, whoops, can't--
foo is going to wait here because it can't grab the lock
because it's owned by main.
And now we get to the point where
main has to wait for the sync, and the child
is never going to complete because I
hold the resource that the child needs to complete.
So don't hold mutexes across Cilk syncs.
That's the lesson there.
There are actually places you can,
but if you don't hold them across that,
then you won't run into this particular problem.
A good strategy is only holding mutexes within strands.
So there's no parallelism.
So you have it bounded.
And also, that's a good idea generally
because you want to hold mutexes as short amount of time
as you possibly can.
So, for example, if you have a big calculation
and then you want to assign something atomically,
don't put the big calculation inside the critical region.
Move the calculation outside the critical region,
do the calculation you need to do,
and then acquire the locks just to do the interaction
you need to set a value.
And then you'll have a lot faster code
because you're not holding up other threads for a long time.
And always try to avoid nondeterministic programming.
But that's not always possible.
So any questions about that?
Then I want to go on a really interesting topic
because it's a really recent research level topic,
and that's to talk about transactional memory.
Who's heard this term before?
Anybody?
So the idea is to have database transactions,
that you have things like database transactions
where the atomicity is happening implicitly.
You don't specify locks.
You just say this is a critical region.
Don't interrupt me while I do this critical region.
The system works everything out.
Here's a good example of where it might be useful.
Suppose we want to do a concurrent graph computation.
And so you take people involved in parallel
and distributed computing at MIT and you say,
OK, I want to do Gaussian elimination on this graph.
Now, you guys, I'm sure most of you
know Gaussian elimination from the matrix context.
Do you know what it means in a graph context?
So if you have a sparse matrix, you actually have a graph.
And Gaussian elimination is a way of manipulating the graph,
and you get exactly the same behavior
as you get in the dense one.
So I'll show you what it is.
You basically pick somebody to eliminate.
[STUDENTS LAUGH]
And now what you do is look at all this vertex's neighbors.
Those guys.
And what you do is you eliminate that vertex--
bye bye-- and you interconnect all the neighbors
with all the edges that don't already exist.
And that's Gaussian elimination.
And if you think of it in terms of matrix fashion,
the question is, if you have a sparse matrix,
where are you going to get fill in?
What are the places that you need
to update when you do a pivot in Gaussian
elimination in a matrix?
So that's the basic notion of graph--
of doing Gaussian elimination.
But now we want to deal with the concurrency.
And the problem occurs if I want to eliminate
two nodes at the same time.
Because now they're adjacent to each other,
and if I just do what I expressed,
there's going to be all kinds of atomicity violations,
et cetera.
By the way, the reason I'm picking these two folks
is because they're going to a better place.
So how do you deal with this?
And so in transactional memory, what I want to be able to do
is just simply say, OK, here's the thing
that I need to be atomic.
And so if I look at this code, it's
basically saying who are my neighbors,
and then let me identify all of the edges that
need to be removed, the ones that I just showed you
that we removed.
Now let me get rid of the element v.
And now, for all of the neighbors of u,
let us add in the edge between the neighbor and--
between the pairs of neighbors.
So that's basically what it's doing.
And I'd like to just say that's atomic.
And so the idea is that if I express
that as a transaction, then the idea
is that, on the transaction commit,
all the memory updates in the critical region
appear to take it happen at once.
However, in transaction, remember
the idea is, rather than forcing it to go forward,
I can have the transactions abort.
So if I get a conflict, I'll abort one and restart it.
And then the restarted transaction
may take a different code path, because, after all, I
may have restructured the graph underneath.
And so it may do something different the second time
through than the first.
It may also abort again and so forth.
So when you study transaction, transactional memory--
let me just do a couple of definitions.
One is a conflict.
That's when you have two transactions that are--
they can't both complete.
One of them has to be aborted.
And aborting, by the way, is once again
violating the nonpreemptive nature.
Here we're going to preempt one of them
by keeping all the states so I can roll a state back
and restart it from scratch.
So contention resolution is deciding
which of the two conflicting transactions
to wait or to abort and restart, and under what conditions
you do that.
So the resolution manager has to figure out
what happens in the case of contention.
And then forward progress is avoiding deadlock of course,
but also livelock and starvation.
You want to make sure that you're going to make--
because what you don't want to have happen,
for example, is that two transactions
keep aborting each other and you never make forward progress.
And throughput, well, you'd like to run as many transactions
as concurrently as possible.
So I'm going to show you an algorithm for doing this.
It's a really simple algorithm.
It happens to be one that I discovered
just a couple of years ago.
And I was surprised that it did not appear in the literature,
and so I wrote a very short paper on it.
Because what happens for a lot of people is they--
if they discover there's a lot of aborting,
they say, oh, well let's grab a global lock.
And then if everybody grabs a global lock,
you can do this sort of thing.
You can't deadlock with a single lock
if you're not also doing things like Cilk sync or whatever.
But, in any case, if you have just a single lock,
everybody falls back to the single lock,
and then you have no concurrency in your program,
no performance, until everybody gets
through the difficult time.
So this is an algorithm that doesn't require a global lock.
So it assumes the transactional memory system
will log the reads and writes.
That's typically true of any transaction, where
you log what reads and writes you're
doing so that you can either abort and roll back,
or you can--
when you abort-- or else you sandbox things and then
atomically commit them.
And so we have all the mechanisms
for aborting and rolling back.
These are all very interesting in their own right,
and restarting.
And this is going to basically use a lock-based approach that
uses two ideas.
One is the notion of what's called a finite ownership
array, and another is a thing called release-sort-reacquire.
And let me explain those two things,
and I'll show you really quickly how this beautiful algorithm
works.
So you have an array of anti-starvation mutual
exclusion locks.
So these are ones that are going to be fair, so that you're
always going to the oldest one.
And you can do an acquire, but we're also
going to add in a try acquire.
Tell me whether, if I tried to acquire, I would get it.
That is, if I get it, give it to me.
If I don't get it, don't wait.
Just tell me that I didn't get it, and then release.
And there's an owner function that maps all of the--
function h that maps my universe of memory locations
to the indexes in this finite ownership
array, this lock array.
So the lock has length--
array has length n, has n slots in it.
To lock a location x in the set of all possible memory
locations, you actually acquire lock of h of x.
So you can think of h as a hash function,
but it doesn't have to be a fair hash function or whatever.
Any function will do.
And then, yes, there will be some advantages to picking
some functions or another one.
So rather than actually locking the location
or locking the object, I lock a location
that essentially I hash to from that object.
So if two guys are trying to grab the same location,
they will both grab the same lock
because they've got the same hash function.
But I may have inadvertent locks where
if I were locking the objects themselves,
I wouldn't have them both trying to acquire the same lock.
That might happen in this algorithm.
So here's the idea.
The first idea is called release, sort, and reacquire.
So that's the ownership array part that I just explained.
Now here's the release, sort, reacquire.
Before you access a memory location x,
simply try to grab lock of x greedily.
And if you have a conflict--
so if you don't have a conflict, you get it.
You just simply try to get it.
And if you can, that's great.
If not, then what I'm going to do is roll back the transaction
but don't release the locks I hold,
and then release all the locks with indexes
greater than h of x.
And then I'm going to acquire the lock that I want.
And now, at that point, I've released all the bigger locks,
so I'm acquiring the next lock.
And then I reacquire the released locks in sorted order.
So I go through all the locks I released and I reacquire them
in sorted order.
And then I start my transaction over again.
I try again.
So what happens each time through this process,
I'm always--
whenever I'm trying to acquire a lock,
I'm only holding locks that are smaller.
But each time that I restart, I have one more lock
that I didn't used to have before I restart
my transaction, which I've acquired in the order,
in the linear order, in that ownership array from 0
to n minus 1.
And so here's the algorithm.
I'll let you guys look at it in more detail,
because I see our time is up.
And it's actually fun to take a look at,
and we'll put the paper online.
There's one other topic that I wanted to go through here
which you should know about, is this locking anomaly
called convoying.
And this was actually a bug that we had-- a performance bug
that we had in our original and MIT-Cilk.
So it's kind of a neat one to see and how we resolved it.
And that's it.