add atomic element level

[willow-ahrens]

Co-authored-by: teocollin1995 [email protected], sortof fixes #531

[willow-ahrens]

We should add a lower_assign function here.

[willow-ahrens]

thus, instantiate would be an "early expand" function, that expands levels immediate after unfurling. "unfurl" and "lower_access" would be "late expand" functions, that don't add dimensions and wait until the next call to unfurl to expand themselves.

[wraith1995]

I forget how this works, but shouldn't a false be being added somewhere here?

[wraith1995]

and in many of these.

[willow-ahrens]

This is how it was before we started. Perhaps I need to make an issue for this. I'll look into it. One related question I had for you: This PR delays the lowering of atomics until just before their child is unfurled or accessed. That doesn't change this analysis right?

[wraith1995]

That is going to require some thought for me to answer. Hmm.

[wraith1995]

Minor comments + one concern.

[willow-ahrens]

I'm worried that mutex levels can't handle multiple writers to the same level from the same thread. Not sure how we would modify them to support that. I guess we could check and set the lock to the thread id, and wait for the tid to be the same.

[wraith1995]

I'm worried that mutex levels can't handle multiple writers to the same level from the same thread. Not sure how we would modify them to support that. I guess we could check and set the lock to the thread id, and wait for the tid to be the same.

I think we don't want the various atomic levels to have knowledge of the thread id. That strikes me as something we should avoid.

I think the concern here is a deadlock, right? I think the main thing there is just this is inefficient because we need to release the lock after each write. The key concern is where the compiler places the acquire/releases. IIRC we wanted atomic to bind as tightly as possible - if that is the case, I think this should be fine.

[willow-ahrens]

if we write

for i = _
   A[i] += 1
   A[i] += 2
end

and A is Tensor(Mutex(Dense(Element(0.0)))), it seems like this is a case where we would be okay if we knew that it was the same thread doing both modifications.

[willow-ahrens]

I guess what I'm saying is that we want that case to be okay if only one thread is running this computation, but it would currently deadlock.

[wraith1995]

if we write

for i = _
   A[i] += 1
   A[i] += 2
end

and A is Tensor(Mutex(Dense(Element(0.0)))), it seems like this is a case where we would be okay if we knew that it was the same thread doing both modifications.

I have a test meant to stress atomics - let's adapt that to this? Then I think the next thing to do is ask how we can either ensure that releases come before acquires or else ensure both are grouped under one lock. We need one of those, correct?

[willow-ahrens]

The compiler unwraps both mutex levels before lowering the loop. This means it lowers to

aquire
aquire
for i
  ...
end
release
release