Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gcp_cloud_storage never gets ready #3001

Open
AndreasBergmeier6176 opened this issue Nov 12, 2024 · 2 comments
Open

gcp_cloud_storage never gets ready #3001

AndreasBergmeier6176 opened this issue Nov 12, 2024 · 2 comments
Labels
gcp Issues relating to GCP needs investigation It looks as though have all the information needed but investigation is required

Comments

@AndreasBergmeier6176
Copy link
Contributor

AndreasBergmeier6176 commented Nov 12, 2024

I have a config, where I connect gcp_cloud_storage to gcp_pubsub.
The respective Kubernetes Deployment never becomes ready.

Looking into the implementation of /ready, it checks for v.IsReady() && v.IsRunning() in line.

Which seems odd to me since the endpoint is named ready. Anyway - while I naively assume that IsReady should be true here, I wonder whether IsRunning does actually happen for gcp_cloud_storage ever/often?

Increasing the log level I see at the end:

Waiting for pending acks to resolve before shutting down
Pending acks resolved.

It might be the problem of returning End of Input.

@mihaitodor
Copy link
Collaborator

Hey @AndreasBergmeier6176 👋 It looks like the code here doesn't use any timeout for the context that's passed to pubsub.NewClient() and we should probably set one and expose it as a config parameter. I'll take this task as a TODO.

@mihaitodor mihaitodor added the needs investigation It looks as though have all the information needed but investigation is required label Nov 13, 2024
@mihaitodor mihaitodor transferred this issue from redpanda-data/benthos Nov 13, 2024
@mihaitodor mihaitodor added the gcp Issues relating to GCP label Nov 13, 2024
@AndreasBergmeier6176
Copy link
Contributor Author

The problem was actually that gcp_cloud_storage was closing the input once the iteration was done. Which in hindsight may be okish - but it may be good to mention in the docs to maybe rather use storage notifications for streaming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gcp Issues relating to GCP needs investigation It looks as though have all the information needed but investigation is required
Projects
None yet
Development

No branches or pull requests

2 participants