Algorithm Descriptions

Meant to record descriptions of algorithms in JEDI modules

---

Analysis Job Throttle

Also known as Analysis Job Stabilizer

Implemented in analysis brokerage.

Algorithm

• First, we decide whether the PQ can be throttled. The PQ will NOT be throttled if it does not have enough queuing jobs; that is, when any condition of the following is satisfied:

  • nQ(PQ) < BASE_QUEUE_LENGTH_PER_PQ (say 100)
  • nQ(PQ) < BASE_EXPECTED_WAIT_HOUR_ON_PQ * trr(PQ) (say 8 hours of queue length)

• If the PQ does not meet any condition above, it can be throttled. Then we compute the nQ_max(PQ) (i.e. the affordable max queue length of each PQ), which is the max among the following values:

  • BASE_QUEUE_LENGTH_PER_PQ (say 100)
  • STATIC_MAX_QUEUE_RUNNING_RATIO * nR(PQ) (say 2 * nR(PQ))
  • MAX_EXPECTED_WAIT_HOUR * trr(PQ) (say 12 hours of queue length)

• Next, we compute what percentage of nQ_max(PQ) is for the user:

  • percentage(PQ, user) = max( nR(PQ, user)/nR(PQ) , 1/nUsers(PQ) )

• Finally, the max value among the following will be used to throttle nQ(PQ, user) (i.e. make the task pending if nQ(PQ, user) > the max of the following values)

  • BASE_DEFAULT_QUEUE_LENGTH_PER_PQ_USER (say 5)
  • BASE_QUEUE_RATIO_ON_PQ * nR(PQ) (say 5% * nR(PQ))
  • nQ_max(PQ) * percentage(PQ, user)