Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GC feature #108

Merged
merged 7 commits into from
Mar 19, 2024
Merged

GC feature #108

merged 7 commits into from
Mar 19, 2024

Conversation

tarassh
Copy link
Contributor

@tarassh tarassh commented Mar 11, 2024

Periodic Garbage Collection

To enable the feature user has to pass a cmd parameter --enable-gc. Flag activates GC configuration settings stored ~/.ipfs/config

cat ~/.ipfs/config
...
"Datastore": {
    "StorageMax": "10GB",
    "GCPeriod": "1h",
    ...
}
...

StorageMax and GCPeriod are defaulted to 10GB and 1h respectively.

@tarassh tarassh changed the title initial commit GS for file upload Mar 11, 2024
@rogarcia
Copy link
Contributor

pls use the develop branch as base for new features so the PR gets merged there first

@sudeepdino008 sudeepdino008 changed the base branch from main to develop March 12, 2024 06:25
@sudeepdino008
Copy link
Contributor

@tarassh given that we're removing the segments from the segments directory in bsp-agent, I think we should keep the ipfs maintained segments for a little longer, rather than just removing it after upload is done. The way to do this would be:

in ipfs-pinner:

in bsp-agent:

  • since ipfs-pinner is maintaining the segments for some time/storage space, we can cleanup the segments directory (or remove it entirely).

@tarassh tarassh changed the title GS for file upload GC feature Mar 16, 2024
@tarassh tarassh marked this pull request as ready for review March 16, 2024 18:15
@sudeepdino008
Copy link
Contributor

sudeepdino008 commented Mar 18, 2024

@tarassh

  • document the config parameters needed in the README.
  • Also change the dockerfile to have --enable-gc.
  • what happens when gc config parameters are not there? GC doesn't happen? If it doesn't, then it might be useful to have a warning message, something like: "gc is enabled; but parameters not available in config; GC won't run"

@tarassh
Copy link
Contributor Author

tarassh commented Mar 18, 2024

@sudeepdino008

what happens when gc config parameters are not there? GC doesn't happen? If it doesn't, then it might be useful to have a warning message, something like: "gc is enabled; but parameters not available in config; GC won't run"

Default values will be used even if config file does not exist.

@rogarcia rogarcia merged commit e260cf2 into develop Mar 19, 2024
3 checks passed
@noslav noslav mentioned this pull request Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants