[RFC] fixpoint iteration support #603

carljm · 2024-10-23T21:23:12Z

This PR removes the existing unwind-based cycle fallback support (a plus for WASM compatibility), and replaces it with support for fixpoint iteration of cycles.

To opt in to fixpoint iteration, provide two additional arguments to salsa::tracked on the definition of a tracked function: cycle_initial and cycle_fn. The former is a function which should provide a provisional starting value for fixpoint iteration on this query, and the latter is a function which has the opportunity, after each iteration that failed to converge, to decide whether to continue iterating or fallback to some fixed value. See the added test in cycle_fixpoint.rs for details.

Usability points that should be covered in the documentation:

With the old cycle fallback, it was sufficient to avoid panic for at least one query in a cycle to define a cycle fallback. With fixpoint iteration, to avoid cycle panics you must define cycle_fn and cycle_initial on every query that might end up as the "head" of a cycle (that is, queried for its value while it is already executing.)
It is entirely possible to define cycle_fn and cycle_initial so as to cause iteration to diverge and never terminate; it's up to the user to avoid this. Techniques to avoid this include a) ensuring that cycles will converge, by defining the initial value and the queries themselves monotonically (for example, in a type-inference scenario, the initial value is the bottom, or empty, type, and types will only widen, never narrow, as the cycle iterates -- thus the cycle must eventually converge to the top type, if nowhere else), and/or b) with a larger hammer, by ensuring that cycle_fn respects the iteration count it is given, and always halts iteration with a fallback value if the count reaches some "too large" value.
It's also entirely possible to define cycle_fn and cycle_initial such that memoized results can vary depending only on the order in which queries occur. Avoid this by minimizing the number of tracked functions that support fixpoint iteration and ensuring initial values and fallback values are consistent among tracked functions that may occur in a cycle together.

This is an RFC pull request to get initial reviewer feedback on the design and implementation. Remaining TODO items:

add tests for more complex cycles:
- nested (multiple head) cycles
- cycles with multiple paths back to the same cycle head
add tests for cross-thread cycles
add tests that use inputs in cycle recovery functions
test in red-knot and validate it works there
performance improvements
- lazy creation of initial-value memo?
documentation

netlify · 2024-10-23T21:23:37Z

✅ Deploy Preview for salsa-rs canceled.

Name	Link
🔨 Latest commit	`c1bbdcf`
🔍 Latest deploy log	https://app.netlify.com/sites/salsa-rs/deploys/67241d4f9e563f00086be6e9

codspeed-hq · 2024-10-23T21:25:09Z

CodSpeed Performance Report

Merging #603 will not alter performance

_{Comparing carljm:fixpoint (c1bbdcf) with master (dfbe6d6)}

Summary

✅ 8 untouched benchmarks

nikomatsakis · 2024-10-24T01:50:42Z

This is very cool! (Admittedly, I say this pre-review.)

MichaReiser

This looks great. I left a few comments where I struggled understanding the implementation or had smaller suggestions.

MichaReiser · 2024-10-24T07:44:23Z

components/salsa-macro-rules/src/setup_tracked_fn.rs

@@ -179,12 +182,16 @@ macro_rules! setup_tracked_fn {
                    $inner($db, $($input_id),*)
                }

+                fn cycle_initial<$db_lt>(db: &$db_lt dyn $Db) -> Self::Output<$db_lt> {
+                    $($cycle_recovery_initial)*(db)


I suspect that it's possible that the initial value function or the recover functions itself could create new cycles. Is this indeed the case and if so, what's salsa's behavior?

Hmm. In the use cases I have in mind (e.g. for red-knot) there should not be any reason to call a query from within one of these functions. And it seems like this could be quite difficult to deal with. So unless we have clear use cases, I would rather consider this out of scope. We could just document "don't do that," or we could add some kind of explicit prevention.

Yeah. I'm not suggesting that users should do that. I only wonder about what happens if a user does it regardless.

Ideally, we wouldn't provide a Db but we can't do that because a user might want to create a tracked struct.

I think the simplest approach here would be to add some state on Runtime that causes an error if you try to push a new active query. I'll wait for Niko review before doing that, though.

src/function/execute.rs

src/zalsa_local.rs

MichaReiser · 2024-10-24T08:03:34Z

tests/cycle_fixpoint.rs

+    }
+}
+
+#[salsa::tracked(cycle_fn=cycle_recover, cycle_initial=cycle_initial)]


Should we define a custom CycleRecovery trait that defines recover and initial methods. It would give us a good point to put cycle handling documentation and avoids the risk for incorrect-macro use when only specifiying one but not boht values.

Yes, I thought about this as well. It's not clear to me what is actually the better UX. Implementing a trait is somewhat more boilerplate, and in practice there's not much difference in the experience if you fail to implement one of the methods. Either way the compiler catches the error for you, because its an error in macro expansion if you don't provide both cycle_fn and cycle_initial, with a trait it would be a compiler error that you didn't fully implement the trait.

I guess with a trait your IDE might give you the right signatures of the methods for free, which is kind of nice...

I normally don't like traits, but I feel like a trait would be preferable because the current annotation seems a little too wordy for my tastes.

Ok, going to still wait for Niko's feedback on this point before updating, but the trait idea makes sense to me.

Actually, I'm okay with a non-trait based approach. rust-analyzer uses a #[salsa::cycle(path::to::function)]-style annotation, so I think to ease the transition, avoiding a trait would be nice.

rust-analyzer uses a #[salsa::cycle(path::to::function)]-style annotation, so I think to ease the transition, avoiding a trait would be nice.

Is cycle handling common in ra? I do see how it reduces the diff size because it isn't necessary to make the function a trait-method but you would still have to change every use because you now have to specify both the cycle and cycle initial functions.

carljm · 2024-10-29T18:15:08Z

In writing more comprehensive tests for this, I realized that it needs some changes to correctly handle multi-revision scenarios; taking it to Draft mode until I get that fixed.

carljm · 2024-10-30T00:37:43Z

Ok, multiple-revision cases are now fixed, and we now populate the initial provisional value only lazily, in case a cycle is actually encountered, which should reduce the number of memos created by quite a lot.

Also added a bunch of tests, including multiple-revision cases and one test involving durability. Still need to add cross-thread cycle tests.

carljm · 2024-10-30T00:42:47Z

tests/cycle/main.rs

+// Diagram nomenclature for nodes: Each node is represented as a:xx(ii), where `a` is a sequential
+// identifier from `a`, `b`, `c`..., xx is one of the four query kinds:
+// - `Ni` for `min_iterate`
+// - `Xi` for `max_iterate`
+// - `Np` for `min_panic`
+// - `Xp` for `max_panic`
+//
+// and `ii` is the inputs for that query, represented as a comma-separated list, with each
+// component representing an input:
+// - `a`, `b`, `c`... where the input is another node,
+// - `uXX` for `UntrackedRead(XX)`
+// - `vXX` for `Value(XX)`
+// - `sY` for `Successor(Y)`
+//


These are admittedly obscure-looking chicken scratches. I'm not claiming they are super readable, but they are concise enough to put into an ASCII graph diagram, and (once you get familiar with them) give a lot of information about the behavior of the test. They were really helpful to me in writing and debugging the tests.

Open to feedback that I should do this differently for better readability by future maintainers...

src/function/execute.rs

MichaReiser

The lazy creation of the initial value is a neat improvement. Nice for taking the time to work on it !

src/function/execute.rs

MichaReiser · 2024-10-30T07:35:48Z

The benchmarks show a 4-5% regression. It seems that we're now resizing some hash maps more often. Are we reporting more tracked reads than before? Could you take a look what's causing it?

carljm · 2024-11-01T00:42:07Z

Initial experiments using this in the red-knot type checker are promising: astral-sh/ruff#14029

Not yet using it for loopy control flow in that PR, but there are cycles in the core type definitions of Python builtins and standard library, which we previously had a hacky fallback in place for using Salsa's previous cycle fallback support. Moving over to fixpoint iteration just worked, and fixed the type of a builtin impacted by the cycle.

On the downside, it is a performance regression. Need to do more work there.

carljm added 17 commits October 21, 2024 13:10

add example test for fixpoint iteration

207656a

add a multi-symbol test

6d065ca

simplify test case

915b245

WIP: remove existing cycle handling tests for now

fd454f7

WIP: remove all existing cycle handling, add fixpoint options

fe1de3c

WIP: added provisional value and cycle fields

08901b2

rename to CycleRecoveryStrategy::Fixpoint

abb6fdd

WIP: rip out ProvisionalValue

8dc4a0b

WIP: working single-iteration with provisional memo

ff455fc

WIP: add count arg to cycle_recovery_fn

cd01235

WIP: move insert-initial out into fetch_cold

79a1ada

WIP: cycle-head iteration

7b6b1a7

WIP: move loop into execute

eb0325b

WIP: delay storing memo

c6c972c

WIP: remove ourself from cycle heads when done iterating

7a425f7

WIP: working convergence and fallback

f84ef80

WIP: clippy and cleanup

05a7d5b

carljm requested review from nikomatsakis, MichaReiser and davidbarsky October 23, 2024 21:23

MichaReiser reviewed Oct 24, 2024

View reviewed changes

carljm added 3 commits October 24, 2024 12:39

WIP: improve comments and add a type annotation

b816aa0

WIP: don't allow cycle_fn with no_eq

7fcfd7f

WIP: add tracing for cycle iteration

f411766

carljm force-pushed the fixpoint branch from f700c39 to f411766 Compare October 24, 2024 20:06

salsa-rs deleted a comment from MichaReiser Oct 24, 2024

WIP: fail fast if we get an evicted provisional value

e16055d

WIP: use FxHashSet::from_iter

446d655

carljm marked this pull request as draft October 29, 2024 18:15

add tests, fix multiple-revision, lazy provisional value

3325861

carljm marked this pull request as ready for review October 30, 2024 00:37

carljm commented Oct 30, 2024

View reviewed changes

MichaReiser reviewed Oct 30, 2024

View reviewed changes

src/function/execute.rs Outdated Show resolved Hide resolved

MichaReiser reviewed Oct 30, 2024

View reviewed changes

src/function/execute.rs Outdated Show resolved Hide resolved

src/function/execute.rs Outdated Show resolved Hide resolved

src/function/execute.rs Outdated Show resolved Hide resolved

carljm added 5 commits October 31, 2024 14:52

review feedback, more tracing

ce1e57b

fix multi-revision bug

b114580

better fix for multi-revision bug

0b59209

test fixes

706d0ad

pass inputs to cycle recovery functions

c1bbdcf

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] fixpoint iteration support #603

[RFC] fixpoint iteration support #603

carljm commented Oct 23, 2024 •

edited

Loading

netlify bot commented Oct 23, 2024 •

edited

Loading

codspeed-hq bot commented Oct 23, 2024 •

edited

Loading

nikomatsakis commented Oct 24, 2024

MichaReiser left a comment

MichaReiser Oct 24, 2024

carljm Oct 24, 2024

MichaReiser Oct 24, 2024

carljm Oct 24, 2024

MichaReiser Oct 24, 2024

carljm Oct 24, 2024

carljm Oct 24, 2024

davidbarsky Oct 24, 2024 •

edited

Loading

carljm Oct 26, 2024

davidbarsky Oct 29, 2024

MichaReiser Oct 29, 2024

carljm commented Oct 29, 2024

carljm commented Oct 30, 2024

carljm Oct 30, 2024

MichaReiser left a comment

MichaReiser commented Oct 30, 2024 •

edited

Loading

carljm commented Nov 1, 2024

[RFC] fixpoint iteration support #603

Are you sure you want to change the base?

[RFC] fixpoint iteration support #603

Conversation

carljm commented Oct 23, 2024 • edited Loading

netlify bot commented Oct 23, 2024 • edited Loading

✅ Deploy Preview for salsa-rs canceled.

codspeed-hq bot commented Oct 23, 2024 • edited Loading

Merging #603 will not alter performance

Summary

nikomatsakis commented Oct 24, 2024

MichaReiser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidbarsky Oct 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carljm commented Oct 29, 2024

carljm commented Oct 30, 2024

Choose a reason for hiding this comment

MichaReiser left a comment

Choose a reason for hiding this comment

MichaReiser commented Oct 30, 2024 • edited Loading

carljm commented Nov 1, 2024

carljm commented Oct 23, 2024 •

edited

Loading

netlify bot commented Oct 23, 2024 •

edited

Loading

codspeed-hq bot commented Oct 23, 2024 •

edited

Loading

davidbarsky Oct 24, 2024 •

edited

Loading

MichaReiser commented Oct 30, 2024 •

edited

Loading