Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC - Update documentation #831

Merged
merged 5 commits into from
Jun 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
234 changes: 173 additions & 61 deletions docusaurus-docs/community/contribute/contribute-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,50 @@ description: Contribute to conda-store's codebase
# Contribute code

:::warning
This page is in active development, content may be inaccurate and incomplete.
This page is in active development, content may be inaccurate and incomplete. If you encounter any content that needs improvement, please [create an issue](https://github.com/conda-incubator/conda-store/issues/new/choose) in our issue tracker.
:::

## Set up development environment - conda-store-core
## `conda-store-core`

### Docker (recommended)
Before setting up your `conda-store-core` development environment you will need to have a local copy of the `conda-store` repository.

Install the following dependencies before developing on conda-store.
If you are a first-time contributor:

1. Go to [https://github.com/conda-incubator/conda-store/](https://github.com/conda-incubator/conda-store/) and click on the **Fork** button in the upper right corner to create your copy of the repository.
2. Clone the project to your local computer:

```bash
git clone https://github.com/<your-GH-username>/conda-store/
```

Once you have a local copy of the `conda-store` repository, you can set up your development environment.
There are two main ways to set up your local environment for development:

- Using [Docker and docker-compose(recommended)](#docker-based-development---conda-store-core)
- Local development [without Docker](#local-development-without-docker---conda-store-core)

### Docker-based development - conda-store-core

Install the following dependencies before developing on `conda-store`.

- [Docker](https://docs.docker.com/engine/install/)
- [docker-compose](https://docs.docker.com/compose/install/)

To deploy `conda-store` run the following command
To deploy `conda-store` run the following command:

```shell
```bash
docker-compose up --build -d
```

:::important
Many of the conda-store docker images are built/tested for amd64(x86-64)
Many of the conda-store Docker images are built/tested for amd64(x86-64)
there will be a performance impact when building and running on
arm architectures. Otherwise, this workflow has been shown to run and build on OSX.
Notice the `architecture: amd64` within the `docker-compose.yaml` files.
arm architectures.
Otherwise, this workflow has been shown to run and build on OSX.
**Notice** the `architecture: amd64` within the `docker-compose.yaml` files.
:::

The following resources will be available:
After running the `docker-compose` command, the following resources will be available:

| Resource | Localhost port | username | password |
|----------|----------------|----------|----------|
Expand All @@ -40,8 +58,8 @@ The following resources will be available:
| [PostgreSQL](https://www.postgresql.org/) (database: `conda-store`)| [localhost:5432](http://localhost:5432) | `admin` | `password` |
| [Redis](https://www.redis.com/) | [localhost:6379](http://localhost:6379) | - | password |

On a fast machine this deployment should only take 10 or so seconds
assuming the docker images have been partially built before.
On a fast machine, this deployment should only take 10 or so seconds
assuming the Docker images have been partially built before.

If you are making any changes to `conda-store-server` and would like to see
those changes in the deployment, run:
Expand All @@ -51,20 +69,35 @@ docker-compose down -v # not always necessary
docker-compose up --build
```

### Linux
To stop the deployment, run:

1. Install the following dependencies before developing on conda-store:
```bash
docker-compose stop

- [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html)
# optional to remove the containers
docker-compose rm -f
```

2. Install the development dependencies and activate the environment:
### Local development without Docker - conda-store-core

```shell
# replace this with environment-macos-dev.yaml or environment-windows-dev.yaml
# if you are on Mac or Windows
conda env create -f conda-store-server/environment-dev.yaml
conda activate conda-store-server-dev
```
You will need to install the following dependencies before developing on `conda-store`:

- [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html)

1. Install the development dependencies and activate the environment:

```bash
# from the root of the repository
conda env create -f conda-store-server/environment-dev.yaml
conda activate conda-store-server-dev
```

2. Install the package in editable mode:

```bash
# from the root of the repository
python -m pip install -e ./conda-store-server
```

3. Running `conda-store` in `--standalone` mode launches celery as a
subprocess of the web server.
Expand All @@ -73,50 +106,106 @@ subprocess of the web server.
conda-store-server --standalone
```

1. Visit [localhost:8080](http://localhost:8080/)
4. You should now be able to access the `conda-store` server at [localhost:8080](http://localhost:8080/) from your web browser.

## `conda-store-ui`

Before setting up your `conda-store-ui` development environment you will need to have a local copy of the
`conda-store-ui` repository.

## Set up development environment -- conda-store-ui
If you are a first-time contributor:

To get started with conda-store-ui development, there are a couple of options. This guide will help you to set up your local development environment.
1. Go to [https://github.com/conda-incubator/conda-store-ui/](https://github.com/conda-incubator/conda-store-ui/) and click on the **Fork** button in the upper right corner to create your copy of the repository.
2. Clone the project to your local computer:

### Prerequisites
```bash
git clone https://github.com/<your-GH-username>/conda-store-ui/
```

Before setting up conda-store-ui, you must prepare your environment.
Once you have a local copy of the `conda-store` repository, you can set up your development environment.
There are two main ways to set up your local environment for development:

We use [Docker Compose](https://docs.docker.com/compose/) to set up the infrastructure before starting ensure that you have docker-compose installed. If you need to install docker-compose, please see their [installation documentation](https://docs.docker.com/compose/install/)
- Using [Docker and docker-compose(recommended)](#docker-based-development---conda-store-ui)
- Local development [without Docker](#local-development-without-docker---conda-store-ui)

1. Clone the [conda-store-ui](https://github.com/conda-incubator/conda-store-ui.git) repository.
2. Copy `.env.example` to `.env`. All default settings should work, but if you want to test against a different version of conda-store-server, you can specify it in the `.env` file by setting the `CONDA_STORE_SERVER_VERSION` variable to the desired version
### Pre-requisites

### Local Development with conda-store-ui running in Docker
- [Node.js](https://nodejs.org/en/download/) >= 18
- [Yarn](https://classic.yarnpkg.com/lang/en/docs/)

### Docker-based development - conda-store-ui

Running conda-store-ui in Docker is the simplest way to set up your local development environment.

1. Run `yarn run start:docker` to start the entire development stack.
Open your local browser and go to [http://localhost:8000](http://localhost:8000) so see conda-store-ui.
3. You can then log in using the default username of `username` and default password of `password`.
We use [docker-compose](https://docs.docker.com/compose/) to set up the infrastructure before starting,
you must ensure you have docker-compose installed.
If you need to install docker-compose, please see their [installation documentation](https://docs.docker.com/compose/install/).

1. After cloning the repository change to the project directory:

```bash
# from your command line or terminal
cd conda-store-ui
```

**Note:** Hot reloading is enabled, so when you make changes to source files, your browser will reload and reflect the changes.
2. Copy `.env.example` to `.env`. All default settings should work, but if you want to test against a different version
of `conda-store-server`, you can specify it in the `.env` file by setting the `CONDA_STORE_SERVER_VERSION` variable
to the desired version.
3. Run `yarn run start:docker` to start the entire development stack.
4. Open your local browser and go to [http://localhost:8000](http://localhost:8000) so see conda-store-ui.
5. You can then log in using the default username of `username` and default password of `password`.

### Local Development without running conda-store-ui in Docker
:::note

Hot reloading is enabled, so when you make changes to source files, your browser will automatically reload
and reflect the changes.

:::

### Local development without Docker - conda-store-ui

:::note

This setup still uses Docker for supporting services but runs conda-store-ui locally.

:::

#### Set up your environment

This project uses [Conda](https://conda.io) for package management. To set up Conda, please see their [installation documentation](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html).
1. Change to the project root ` cd conda-store-ui`
2. From the project root create the conda environment `conda env create -f environment_dev.yml`
3. Activate the development environment `conda activate cs-ui-dev-env`
4. Install yarn dependencies `yarn install`
This project uses [conda](https://conda.io) for package management.
To set up conda, please see their [installation documentation](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html).

1. After cloning the repository change to the project directory:

```bash
# from your command line or terminal
cd conda-store-ui
```

2. From the project root create and activate a new conda environment:

```bash
conda env create -f environment_dev.yml
conda activate cs-ui-dev-env
```

3. Install node dependencies:

```bash
yarn install
```

#### Run the application

1. Run `yarn run start` and wait for the application to finish starting up
Open your local browser and go to [http://localhost:8000](http://localhost:8000) so see conda-store-ui.
3. You can then log in using the default username of `username` and default password of `password`.
1. Run `yarn run start` and wait for the application to spin up.
Open your local browser and go to [http://localhost:8000](http://localhost:8000).
2. You can then log in using the default username of `username` and default password of `password`.

:::note

Hot reloading is enabled, so when you make changes to source files, your browser will reload and reflect the changes.
:::

**Note:** Hot reloading is enabled, so when you make changes to source files, your browser will reload and reflect the changes.

<!-- TODO

Expand All @@ -126,23 +215,46 @@ Open your local browser and go to [http://localhost:8000](http://localhost:8000)

## Workflows

### Changes to API
### Changes to the API

The REST API is considered somewhat stable. If any changes are made to
the API make sure the update the OpenAPI/Swagger specification in
`docs/_static/openapi.json`. This may be downloaded from the `/docs`
endpoint when running conda-store. Ensure that the
`c.CondaStoreServer.url_prefix` is set to `/` when generating the
endpoints.
<!-- TODO -->
The REST API is considered somewhat stable. If any changes are made to the API make sure the update the OpenAPI/Swagger
specification in `docs/_static/openapi.json`.
This may be downloaded from the `/docs` endpoint when running conda-store.
Ensure that the `c.CondaStoreServer.url_prefix` is set to `/` when generating the endpoints.

## Workflows
### Adding new dependencies to the libraries

### `conda-store-core`

Runtime-required dependencies should **only** be added to the corresponding `pyproject.toml` files:

### Changes to API
- `conda-store-server/pyproject.toml`
- `conda-store/pyproject.toml`

Development dependencies should be added to both the `environment-dev.yaml` and `pyproject.toml` files.
Within the `pyproject.toml` file these should be added under the `[tool.hatch.envs.dev]` section.

This will ensure that conda-store distributions are properly built and tested with the correct dependencies.

:::important

The only exceptions to this runtime dependencies rules are `conda` and `constructor` which should be added to the
`environment-dev.yaml` file as they are only conda installable.

:::

### `conda-store-ui`

Dependencies should be added to the [`package.json`](https://github.com/conda-incubator/conda-store-ui/blob/main/package.json) file.

### Linting and formatting

We use pre-commit hooks to ensure that code is formatted and linted before being committed.
To install the pre-commit hooks, run:

```bash
pre-commit install --install-hooks
```

The REST API is considered somewhat stable. If any changes are made to
the API make sure the update the OpenAPI/Swagger specification in
`docs/_static/openapi.json`. This may be downloaded from the `/docs`
endpoint when running conda-store. Ensure that the
`c.CondaStoreServer.url_prefix` is set to `/` when generating the
endpoints.
Now every time you commit, the pre-commit hooks will run and check your code.
We also use [pre-commit.ci](https://pre-commit.com/) to automatically run the pre-commit hooks on every Pull Request.
7 changes: 4 additions & 3 deletions docusaurus-docs/conda-store/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,12 @@ description: Introduction to conda-store (core) documentation
The [`conda-store` repository on GitHub][conda-store-repo] consists of two separate, yet related, packages:

- **`conda-store-server`**: web server and workers that together provide the `conda-store` "service" through a REST API
- **`conda-store` (client)**: a client which interacts with the service to offer user-facing command line interface (CLI)
- **`conda-store` (client)**: a client that interacts with the service to offer a user-facing command line interface (CLI)

## Get started ✨

The fastest way to get started with conda-store is with a [**local standalone installation**][standalone-install] and the [**conda-store UI**][conda-store-ui-tutorials].
The fastest way to get started with conda-store is with a [**local standalone installation**][standalone-install]
and the [**conda-store UI**][conda-store-ui-tutorials].

Alternatively, you can use some features of conda-store through the [CLI commands][cli-ref] or as a [shebang][shebang].

Expand All @@ -39,7 +40,7 @@ Make the most of conda-store by learning about:

* [Performance impact of conda-store components][performance]
* [Configuration options for customization][configuration]
* Internal architecture: [overview][ref-arch], [auth][ref-auth], and [database][ref-database]
* Internal architecture: [overview][ref-arch], [auth][ref-auth], and [database][ref-database]

## Community and contributing 🌱

Expand Down
12 changes: 10 additions & 2 deletions docusaurus-docs/docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ const config = {
// Set /<baseUrl>/ pathname under which your site is served
baseUrl: "/",

organizationName: "conda-incubator",
projectName: "conda-store",

onBrokenLinks: "throw",
onBrokenMarkdownLinks: "warn",

Expand Down Expand Up @@ -92,10 +95,15 @@ const config = {
themeConfig:
/** @type {import('@docusaurus/preset-classic').ThemeConfig} */
({
// og:image
image: "img/horizontal-logo.png",
metadata: [
{ name: 'keywords', content: 'conda, python, conda environments, dependencies, reproducibility' },
],
navbar: {
title: "Home",
logo: {
alt: "conda-store logo",
alt: "conda-store logo - conda-store Homepage",
src: "img/logo.svg",
},
items: [
Expand Down Expand Up @@ -171,7 +179,7 @@ const config = {
],
},
],
copyright: `Copyright © ${new Date().getFullYear()} | Made with 💚 by conda-store development team`,
copyright: `Copyright © ${new Date().getFullYear()} | Made with 💚 by conda - store development team`,
},
prism: {
theme: lightCodeTheme,
Expand Down
Loading
Loading