Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trickle-server: Don't auto-create channels on first publish. #3251

Merged
merged 2 commits into from
Nov 15, 2024

Conversation

j0sh
Copy link
Collaborator

@j0sh j0sh commented Nov 14, 2024

Channels can still be created locally on the server.

This is needed in case a payment fails then we need to close the channel but we don't want the publisher to re-open them.

We introduce a flag instead of just hard-coding the behavior because we depend on auto-creating in some other tools including the server on the demo box.

Channels can still be created locally on the server.

This is needed in case a payment fails then we need to close
the channel but we don't want the publisher to re-open them.
Copy link

codecov bot commented Nov 14, 2024

Codecov Report

Attention: Patch coverage is 0% with 14 lines in your changes missing coverage. Please review.

Project coverage is 34.70842%. Comparing base (abdd534) to head (b864d76).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
trickle/trickle_server.go 0.00000% 12 Missing ⚠️
trickle/local_publisher.go 0.00000% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@                 Coverage Diff                 @@
##              master       #3251         +/-   ##
===================================================
- Coverage   34.71608%   34.70842%   -0.00766%     
===================================================
  Files            136         136                 
  Lines          36208       36216          +8     
===================================================
  Hits           12570       12570                 
- Misses         22926       22934          +8     
  Partials         712         712                 
Files with missing lines Coverage Δ
trickle/local_publisher.go 0.00000% <0.00000%> (ø)
trickle/trickle_server.go 0.00000% <0.00000%> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update abdd534...b864d76. Read the comment docs.

Files with missing lines Coverage Δ
trickle/local_publisher.go 0.00000% <0.00000%> (ø)
trickle/trickle_server.go 0.00000% <0.00000%> (ø)

Copy link
Contributor

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this PR @j0sh Looks good, but I have two questions.

  1. I believe we should delay the first payment check and allow all to start and check the first payment after 10, maybe 30s. In other words we take the optimistic approach that everything is fine not to delay the rest. What do you think? The alternative would be to wait with setting up everything until the first payment comes.

  2. When the payment check fails, then I close the stream and it prevents the streaming from happening. All good. But I wonder about cleaning up other parts.

  • Tear down runner => I guess @victorges implements some timeout, so it should happen automatically
  • Any other objects to clean up on either Orchestrator or Gateway?

@j0sh
Copy link
Collaborator Author

j0sh commented Nov 15, 2024

Thanks for the review @leszko

I believe we should delay the first payment check and allow all to start and check the first payment after 10, maybe 30s.

Possibly. We can send the first payment in along with the initial request like the other pipelines do it (eg via prepareAIPayment). Since the initial request is going to be reserving (scarce) capacity for the pipeline, it might be worth having a "down payment" of sorts, to ensure the gateway has the ability to pay for the job. But if this risk seems manageable then we can start checking after 10-30s.

Tear down runner => I guess @victorges implements some timeout, so it should happen automatically

The runner will receive an EOS (end of stream) flag and stop subscribing (PR). I think the rest of the pipeline is also terminated after that, but I haven't tested recently.

Any other objects to clean up on either Orchestrator or Gateway?

Right now, the bookkeeping on the orchestrator is minimal (just publish and subscribe streams IIRC) but I am sure we will have more to clean up later. The gateway also cleans itself up after a moment or so, but I do have some improvements to make on the publisher side so tear-down is a little more controllable, rather than the current backpressure-led error chain.

@j0sh j0sh merged commit d14d2cf into master Nov 15, 2024
18 checks passed
@j0sh j0sh deleted the ja/trickle-autopublish branch November 15, 2024 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants