Resource Usage of clickhouse-backup

[honobi]

How much CPU, memory, and disk resources does clickhouse-backup consume?

My clickhouse-backup is running in a container within a pod, and I have set resource limits for the container.

CPU: If CPU usage is too high, it will affect the query performance of ClickHouse.
Memory: If memory usage is too high, it may cause the container to be killed.
Disk: Will temporary files be generated? If so, what is the size of these temporary files?

[Slach]

How much CPU, memory, and disk resources does clickhouse-backup consume?

too common question
common answer is
run it in your environment and monitor

resources depends on how many tables / parts / data file size you have

most of resoures is allocated during UPLOAD / DOWNLOAD operation
Memory used for buffer to increase upload/download speed
compression_format: tar allows less CPU usage
disk space used only if you have many background mutations / merges in clickhouse server

CPU: If CPU usage is too high, it will affect the query performance of ClickHouse.

Usually queries performance affected more depends on disk bandwidth usage than depends on CPU usage

Memory: If memory usage is too high, it may cause the container to be killed.

control it with
upload_concurrency (decrease)
max_part_count (for s3 increase from 5000 to 7000) if you have lagre data parts size (look to system.parts)
settings

Disk: Will temporary files be generated? If so, what is the size of these temporary files?

No temporary files generated, we use hard links for exists data files

[honobi]

When clickhouse-backup uploads data to S3, does it package the data, resulting in the creation of temporary files

[Slach]

When clickhouse-backup uploads data to S3, does it package the data, resulting in the creation of temporary files

no, this is in memory streaming, files read from disk, pack into tar, depends on compression_format stream also could be compressed and upload and write into multipart upload buffer manager and upload into s3 finally

[honobi]

Could you please provide some recommended values for CPU and memory limits for the container running clickhouse-backup? (The data size in ClickHouse may range from a few hundred GB to 10 TB. )I would like to set some initial limit values and then adjust them based on observation later.

[Slach]

just run clickhouse-backup with default settings in your data
and monitor cpu/memory usage after that try to tune resources and config parameters