Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What gets stored in Database and artifact store? #1590

Open
adhikari23 opened this issue Oct 20, 2020 · 1 comment
Open

What gets stored in Database and artifact store? #1590

adhikari23 opened this issue Oct 20, 2020 · 1 comment
Assignees
Labels

Comments

@adhikari23
Copy link

adhikari23 commented Oct 20, 2020

As it is mentioned in your documentation that
Database ModelDB Backend stores the information from the requests it receive into a Relational database.
Out of the box ModelDB is configured and verified to work against PostgreSQL.
Volumes : The relational database and the artifact store in backend need volumes attached to enable persistent storage.

What gets stored in Database and what exactly goes into Artifact store?
Also, why do we need to two storages ( Database & Artifact store )? Is it possible that we have two logical instance of single Database to serve the purpose?

@ravishetye
Copy link

Hey @adhikari23 , one rough way to think of things are things which ModelDB interprets are stored in the database and the things it does/should not are stored in the artifact store.

Most of the metadata related to Projects, Experiment, ExperimentRun, Dataset, DatasetVersion, Repository, Commits are stored in the Database. This metadata includes but is not limited to the name, description, creation and updating time, tags, metrics, hyperparameters, observations, etc.

The Artifactstore is meant to store large BLOB/ CLOB. ArtifactStore can store the actual dataset which the user logs, any artifacts which need to be associated with the experiment run (for example an image of a chart which the user wants to associate with the experiment run), the actual model, any files that. may be required to be tracked to get the model running (for example : requirements.txt).

The client APIs which would lead the backend to interact with the artifact store can be found
here

@ravishetye ravishetye self-assigned this Oct 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants