Asynchronous pipelines

[jklimcak]

Great project, thanks for this.

One thing I'm surprised by however is the synchronous nature of the pipelines. I registered a bunch of pipelines which are unrelated, hoping they would get simultanously kicked off and run in parallel. I tried adding them both as registered pipelines, but also as individual triggers in the same pipeline, but it all results in sequential runs. Are there any plans to make pipelines run in parallel?

[larsblumberg]

@lucafaggianelli Ciao Luca, is it still the case for plombery that async jobs can't run in parallel?

[lucafaggianelli]

Hey @BlackForgeOne @larsblumberg sorry for the mega delay, I didn't have the bandwidth to follow-up on Plombery, but now I'm getting back to it.

Normally, Plombery supports parallel running of different pipelines or of the same pipeline with different triggers.

The only limitation is that you can't run twice (or more times) a pipeline with the same trigger, that is, you can't hit the run button manually and hit it again while the previous run is ongoing, or create a trigger with an interval shorter than the duration of the pipeline that would start multiple parallel runs. This limitation is by design and it's for historical reasons, but now is useless so I'll take it out in the next release!

Said so, based on @BlackForgeOne issue, I don't think you're in this situation, that is, you're not running the same pipeline + trigger twice?! So probably it's a bug. Could you share some of your code (the pipelines and the start command) and info on your platform (OS, Py version and plombery version)?