Skip to content

Commit

Permalink
[fix](memory) Process available memory to increase the Jemalloc cache (
Browse files Browse the repository at this point in the history
…#45621)

### What problem does this PR solve?

Currently, when the Doris BE process exceed memory limit, Jemalloc cache
will be manually released.
Add the Jemalloc cache to the available memory of the BE process is
expected to have little impact on the risk of the process OOM killer.
the process memory used has already subtracted the Jemalloc cache.

Not merge to 2.1 because 2.1 is stable now
  • Loading branch information
xinyiZzz authored Dec 19, 2024
1 parent b1ccd36 commit 6cce408
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions be/src/runtime/memory/global_memory_arbitrator.h
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ class GlobalMemoryArbitrator {
static inline int64_t sys_mem_available() {
return MemInfo::_s_sys_mem_available.load(std::memory_order_relaxed) -
refresh_interval_memory_growth.load(std::memory_order_relaxed) -
process_reserved_memory();
process_reserved_memory() + static_cast<int64_t>(MemInfo::allocator_cache_mem());
}

static inline std::string sys_mem_available_str() {
Expand All @@ -91,12 +91,14 @@ class GlobalMemoryArbitrator {
static inline std::string sys_mem_available_details_str() {
auto msg = fmt::format(
"sys available memory {}(= {}[proc/available] - {}[reserved] - "
"{}B[waiting_refresh])",
"{}B[waiting_refresh] + {}[tc/jemalloc_cache])",
PrettyPrinter::print(sys_mem_available(), TUnit::BYTES),
PrettyPrinter::print(MemInfo::_s_sys_mem_available.load(std::memory_order_relaxed),
TUnit::BYTES),
PrettyPrinter::print(process_reserved_memory(), TUnit::BYTES),
refresh_interval_memory_growth);
refresh_interval_memory_growth,
PrettyPrinter::print(static_cast<uint64_t>(MemInfo::allocator_cache_mem()),
TUnit::BYTES));
#ifdef ADDRESS_SANITIZER
msg = "[ASAN]" + msg;
#endif
Expand Down

0 comments on commit 6cce408

Please sign in to comment.