The management of thread pools in sofarpc under high concurrency requests.

[dajitui]

Your question

sofastack/sofa-bolt#322
As shown in the above issues, the Bolt framework handles request callback response results through a thread pool. If it encounters high concurrent requests, the thread pool may run out of resources. In the code com.alipay.sofa.rpc.server.bolt.BoltServer#initThreadPool, I saw that the thread pool uses the default rejection policy, which will inevitably lead to some lost responses.

My question is whether there is a corresponding solution to this problem. I know that customizing the thread pool can solve this problem, but for beginners, the framework should be more intelligent. For example, there should be a monitoring mechanism to monitor the situation of the corresponding thread pool, so as to avoid this situation by increasing the client nodes or dynamically adjusting the thread pool parameters.

[OrezzerO]

I think it is business lay which is responsible for the performance of task in threadpool.

Monitoring of the thread pool is important. You can find related code in sofa-boot/RpcThreadPoolMonitor.java. You can see that monitoring is not written in bolt repo, it is more suitable to do monitoring in upper lay.

Howerver , it is not enough to only have monitoring. We still need to do some things to ensure smooth and efficient operation of the system:

1. Adjusting thread pool parameters to fully utilize machine resources.
2. Using asynchronous solutions wisely to reduce synchronous waiting and improve concurrency.
3. Implementing effective monitoring and alerting, and performing horizontal scaling when necessary.

[dajitui]

I think it is business lay which is responsible for the performance of task in threadpool.

Monitoring of the thread pool is important. You can find related code in sofa-boot/RpcThreadPoolMonitor.java. You can see that monitoring is not written in bolt repo, it is more suitable to do monitoring in upper lay.

Howerver , it is not enough to only have monitoring. We still need to do some things to ensure smooth and efficient operation of the system:

1. Adjusting thread pool parameters to fully utilize machine resources.
2. Using asynchronous solutions wisely to reduce synchronous waiting and improve concurrency.
3. Implementing effective monitoring and alerting, and performing horizontal scaling when necessary.

@OrezzerO OK, I got it wrong. The thread pool is used by the server when processing requests.

But if you don't rely on Sofa-boot, this monitoring won't automatically start, right?

[OrezzerO]

I think it is business lay which is responsible for the performance of task in threadpool.
Monitoring of the thread pool is important. You can find related code in sofa-boot/RpcThreadPoolMonitor.java. You can see that monitoring is not written in bolt repo, it is more suitable to do monitoring in upper lay.
Howerver , it is not enough to only have monitoring. We still need to do some things to ensure smooth and efficient operation of the system:

1. Adjusting thread pool parameters to fully utilize machine resources.
2. Using asynchronous solutions wisely to reduce synchronous waiting and improve concurrency.
3. Implementing effective monitoring and alerting, and performing horizontal scaling when necessary.

@OrezzerO OK, I got it wrong. The thread pool is used by the server when processing requests.

But if you don't rely on Sofa-boot, this monitoring won't automatically start, right?

Yes, if you don't relay on sofa-boot. you need to monitor thread pool yourself. You can do some easy task to monitor your thread pool from your APM.

[dajitui]

@OrezzerO OrezzerO, by the way, with the maturity of cloud native, what do you think of the prospects for the development of middleware or Sofa components?

Because cloud native is essential for basic middleware, it is convenient for the popularization of middleware and a challenge for self-developed middleware.

For the secondary development of unique enterprise features or performance tuning?

[OrezzerO]

In my opinion, easy to use is the most important thing. @dajitui