-
Notifications
You must be signed in to change notification settings - Fork 47
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #99 from lucafaggianelli/docs
Docs
- Loading branch information
Showing
19 changed files
with
523 additions
and
48 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| name: docs | ||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| - docs | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| jobs: | ||
| deploy-docs: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| - uses: actions/setup-python@v4 | ||
| with: | ||
| python-version: 3.x | ||
| - uses: actions/cache@v3 | ||
| with: | ||
| key: ${{ github.ref }} | ||
| path: .cache | ||
| - run: pip install mkdocs-material | ||
| - run: mkdocs gh-deploy --force |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -160,3 +160,5 @@ cython_debug/ | |
| #.idea/ | ||
|
|
||
| node_modules | ||
|
|
||
| .DS_Store | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| [data-md-toggle="search"]:not(:checked) ~ .md-header .md-search__form::after { | ||
| position: absolute; | ||
| top: .3rem; | ||
| right: .3rem; | ||
| display: block; | ||
| padding: .1rem .4rem; | ||
| color: #ffffffb3; | ||
| font-weight: bold; | ||
| font-size: .8rem; | ||
| border: .05rem solid #ffffff3b; | ||
| border-radius: .1rem; | ||
| content: "/"; | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,187 @@ | ||
| MarioPype is configurable via environmental variables, a YAML file | ||
| or even better via a combination of the 2. | ||
|
|
||
| !!! info "Why a hybrid configuration?" | ||
|
|
||
| An entire configuration can be quite large so storing it as environmental | ||
| variables can be quite hard to maintain, moreover some parts of the | ||
| configuration should be stored together with the code as they are part | ||
| of the system and some parts of it are secret so you need env vars | ||
|
|
||
| Create a configuration file in the root of your project named `mario.config.yaml` | ||
| (or `mario.config.yml` if you prefer) and set the values you need, you should | ||
| commit this file to the git repo: | ||
|
|
||
| ```yaml title="mario.config.yaml" | ||
| frontend_url: https://pipelines.example.com | ||
|
|
||
| auth: | ||
| client_id: $GOOGLE_CLIENT_ID | ||
| client_secret: $GOOGLE_CLIENT_SECRET | ||
| server_metadata_url: https://accounts.google.com/.well-known/openid-configuration | ||
|
|
||
| notifications: | ||
| - pipeline_status: | ||
| - failed | ||
| channels: | ||
| - $GMAIL_ACCOUNT | ||
| - $MSTEAMS_WEBHOOK | ||
| ``` | ||
| Now define the secrets as environmental variables in a `.env` file, | ||
| in your shell or in your hosting environment. | ||
|
|
||
|
|
||
| !!! tip | ||
|
|
||
| By default, MarioPype will load any `.env` found in your project root. | ||
|
|
||
| !!! Warning | ||
|
|
||
| You shouldn't commit the `.env` file as it contains secrets! | ||
|
|
||
| ```shell title=".env" | ||
| # Auth | ||
| GOOGLE_CLIENT_ID="ABC123" | ||
| GOOGLE_CLIENT_SECRET="DEF456" | ||
| # Notifications | ||
| GMAIL_ACCOUNT=mailto://myuser:[email protected] | ||
| MSTEAMS_WEBHOOK=msteams://TokenA/TokenB/TokenC/ | ||
| ``` | ||
|
|
||
| ## System | ||
|
|
||
| !!! tip | ||
|
|
||
| If you're running MarioPype locally, in most cases you don't need to change | ||
| these settings | ||
|
|
||
| ### `database_url` | ||
|
|
||
| The Sqlite DB URI, by default `sqlite:///./mario.db` | ||
|
|
||
| ### `server_url` | ||
|
|
||
| The URL of the backend, be default `http://localhost:8000`. | ||
|
|
||
| Change it if running in production. | ||
|
|
||
| ### `frontend_url` | ||
|
|
||
| The URL of the frontend, by default is the same as the backend, | ||
| change it if the frontend is served at a different URL, for example | ||
| during the frontend development. | ||
|
|
||
| ## Notifications | ||
|
|
||
| MarioPype can send notifications after a pipeline has run based on the status | ||
| of the run itself (success, failure, etc.). | ||
|
|
||
| The notifications configuration can be defined in the YAML | ||
| file as a list of [`NotificationRule`](#notificationrule)s: | ||
|
|
||
| ```yaml title="mario.config.yaml" | ||
| notifications: | ||
| # Send notifications only if the pipelines failed | ||
| - pipeline_status: | ||
| - failed | ||
| channels: | ||
| # Send them to my gmail address (from my address itself) | ||
| # Better to use an env var here | ||
| - mailto://myuser:[email protected] | ||
| # Send notifications only if the pipelines succeeded or was cancelled | ||
| - pipeline_status: | ||
| - completed | ||
| - cancelled | ||
| channels: | ||
| # Send them to a MS Teams channel | ||
| # Better to use an env var here | ||
| - msteams://mychanneltoken | ||
| ``` | ||
|
|
||
| ### `NotificationRule` | ||
|
|
||
| A notification rule defines when to send notifications and to whom. | ||
|
|
||
| #### `pipeline_status` | ||
|
|
||
| A list of 1 or more pipeline run status among: | ||
|
|
||
| * `completed` | ||
| * `failed` | ||
| * `cancelled` | ||
|
|
||
| #### `channels` | ||
|
|
||
| A list of 1 or more recipients where to send the notifications. | ||
|
|
||
| A channel is an *Apprise* URI string that defines an email address or a MS Teams | ||
| channel, for example: | ||
|
|
||
| * **Email** mailto://myuser:[email protected] | ||
| * **MS Teams** msteams://TokenA/TokenB/TokenC/ | ||
| * **AWS SES** ses://user@domain/AccessKeyID/AccessSecretKey/RegionName/email1/ | ||
|
|
||
| Behind the scene MarioPype uses [Apprise](https://github.com/caronc/apprise), | ||
| a library to send notifications to many notification providers, so check their | ||
| docs for a full list of the available channels. | ||
|
|
||
| ## Authentication | ||
|
|
||
| MarioPype has a buil-in and ready-to-use authentication system | ||
| based on OAuth providers, so you can use your corporate auth system | ||
| or Google, Github, etc. | ||
|
|
||
| To enable the auth system you just need to configure it. | ||
|
|
||
| !!! info "Good to know" | ||
|
|
||
| The auth system is based on [Authlib](https://authlib.org/) | ||
|
|
||
| ### `AuthSettings` | ||
|
|
||
| Options available | ||
|
|
||
| #### `client_id` | ||
|
|
||
| An OAuth app client ID | ||
|
|
||
| #### `client_secret` | ||
|
|
||
| An OAuth app client secret | ||
|
|
||
| #### `server_metadata_url` | ||
|
|
||
| This a special URL that contains information about the OAuth provider | ||
| specific endpoints. If your provider doesn't have this URL or you don't | ||
| know it, you need to fill up the values for the other URLs: `access_token_url`, | ||
| `authorize_url` and `jwks_uri`. | ||
|
|
||
| Here a table of well known Metadata URLs: | ||
|
|
||
| | Provider | URL | | ||
| | -------- | --- | | ||
| | Google | https://accounts.google.com/.well-known/openid-configuration | | ||
|
|
||
| #### `access_token_url` | ||
|
|
||
| #### `authorize_url` | ||
|
|
||
| #### `jwks_uri` | ||
|
|
||
| #### `client_kwargs` | ||
|
|
||
| Additional values to pass to the OAuth client during the auth | ||
| process, for example the scope: | ||
|
|
||
| ```yaml | ||
| auth: | ||
| client_kwargs: | ||
| scope: openid email profile | ||
| ``` | ||
|
|
||
| #### `secret_key` | ||
|
|
||
| Secret key used in the backend middleware, this has a dummy default value, | ||
| but in production you should define a decent value. |
Oops, something went wrong.