-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Broken or too many connection to database when workers > 1 #3313
Comments
Hi team, following up on my investigation, I increased the number of max connection to my PGSQL database and the above error about too many connections does not happen anymore but the following one arise: The full traceback is the following:
As you can see, the error happens during a custom
It did not work, the same error happened from the same place. Hence, it would be really great to have a proper way to call that engine disposal at worker startup, rather than using the task events, is there anyway to achieve it? Many thanks. |
Hi, I am facing the same issue. Could anyone from Luigi/Spotify have a look at it? |
Hi team, many thanks for the great work.
I am using Luigi to schedule a workflow, where some tasks can be run in parallel.
Hence, I would like to have multiple workers to parallelise computation, resulting in faster completion.
However, I am experiencing issue with the connection to my database when enabling multiple workers, whereas there is no issues with a single workers.
For reference, I am using SQLAlchemy 2.0.25 with a PostgreSQL database.
The typical error arising is:
(psycopg2.OperationalError) SSL SYSCALL error: EOF detected
which typically happens when the connection to the database was closed unexpectedly, according to ref.As I understood that enabling multiple workers leads to creating/forking threads, I read that I must dispose the database connection as per the documentation.
Hence, I used Luigi events callback to dispose the database connection at the start of each of the task:
Unfortunately, such approach leads to
(psycopg2.OperationalError) FATAL: sorry, too many clients already
error.As per my understanding, I should execute this database engine disposal a single time at the creation of worker, not at the start of each task.
Given the above context, my questions are:
Many thanks for the help.
Similar question but for Django: #2782
The text was updated successfully, but these errors were encountered: