aws s3 sync local -> s3bucket eats up swap while enough physical free memory is left #7936
Unanswered
KinWaiCheung
asked this question in
Q&A
Replies: 3 comments
-
Linking related issue: #7937. We probably want to close either the issue or this discussion in order to consolidate discussion in one place. |
Beta Was this translation helpful? Give feedback.
0 replies
-
closed and check the related issue |
Beta Was this translation helpful? Give feedback.
0 replies
-
Hello! Reopening this discussion to make it searchable. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
has anyone face issued with aws s3 sync from local to s3bucket and eats up swap memory?
Even though there is enough physical memory left?
aws-cli/2.11.23
every week I schedule a job to sync backup files from local to s3 bucket and the swap memory is decreased a couple hundred of MB each week.
s3 is configured with 20 concurrent requests
file size ranges from 10GB to 70GB each and total amount are in the 4TB
free -m
total used free shared buff/cache available
Mem: 515215 424992 75334 3634 14888 82955
Swap: 3999 3482 517
Beta Was this translation helpful? Give feedback.
All reactions