Add min_size configuration option

[Niedzwiedzw]

I wrote a pool for proxy connections using deadpool, I just want to know if I could make sure that no individual connection is used more often than - say - every 5 seconds. in other words - each object gets X amount of rest before being reused. I'm trying to do it but I'm failing miserably :D

[bikeshedder]

I'm planning to add a min_size configuration option to the pool. I changed the issue title accordingly.

If size is lower than min_size the pool will prefer creating new objects over recycling existing ones. This does not meant hat objects will be created ahead of time. That's what another issue is for:

• Ahead of time object creation/recycling #167

With that option you could just set min_size = max_size and every time you call Pool::get you're guaranteed to either get back a new object or the least recently used object. Now you could just add a pre_recycle hook that checks the metrics and sleeps for MIN_DELAY - metrics.last_used(). That way you would be guaranteed to either get a new objects or an object that hasn't been used for MIN_DELAY duration.

[JosiahParry]

Just want to add my vote for this.

For context, my use case is slightly different than the typical database connection. The "pool" is a connection of R processes which may or may not have a large cost for spin up. So if one knows that the cost of start up is expensive (e.g 10 - 20 second load time and 200mb in memory) it would be great to say "i want to have at least 3 connections waiting at all time."

[JosiahParry]

Is there a way to effectively build a min_size into a pool.retain() call? I took your example and am using it but run into an edge case error where all connections are closed https://github.com/JosiahParry/valve/blob/d4c0e9cdbbb3e3aa308eaca5a4859a85017ad854/src/rust/src/start.rs#L59

[bikeshedder]

Retain is only for removing objects from the pool. There is currently no way to add new objects to the pool manually. As of now objects are always created on demand.

btw. regarding your code I'd rename too_old into keep or something similar. It took me a bit to figure out you named the variable the exact opposite of what it actually does. 🙈

[JosiahParry]

Fair point! Lol, thanks!

Is there a way to use the add() method from unmanaged in a managed pool? This could be called n_min times on startup and the prune checking thread could check to see if the pool's size exceeds n_min and only then do that check.

[bikeshedder]

As of now. No. There is no add() method, yet. Also adding an object to an existing pool from the outside is not necessarily sound as some pools (e.g. deadpool-postgres) do keep track of the created objects. Therefore adding a add() method would requires a change of the Manager interface as well. For simplicity sake adding a create() method should be much simpler. It could work similar to the get() method but skip the object recycling.