[PULP-265] Add docs about STORAGES settings

docs/admin/guides/_SUMMARY.md

@@ -3,5 +3,9 @@
     * [Using external service](auth/external.md)
     * [Using Keycloak](auth/keycloak.md)
     * [Using JSON Header](auth/json_header.md)
+    * *.md
+* Configuration
+    * [Introduction](configure-pulp/index.md)
+    * [Configure Storage backends](configure-pulp/configure-storages.md)
 * *.md
 

docs/admin/guides/configure-pulp/configure-storages.md

@@ -0,0 +1,137 @@
+# Configure Storage Backend
+
+Pulp uses [django-storages](https://django-storages.readthedocs.io) to support multiple storage backends.
+See the reference for the [`STORAGES` settings](site:pulpcore/docs/admin/reference/settings/#storages).
+
+## Modifying the settings
+
+The settings can be updated via the `settings.py` file or through environment variables
+and have support to dynaconf merge features (learn more on the [Settings Introduction](site:pulpcore/docs/admin/guides/configure-pulp/introduction/)).
+
+To learn where and how to modify the files and environment variables, refer to the appropriate installation method documentation:
+
+* [Container (single-process) quickstart](site:pulp-oci-images/docs/admin/tutorials/quickstart/#single-container)
+* [Container (multi-process) quickstart](site:pulp-oci-images/docs/admin/tutorials/quickstart/#podman-or-docker-compose)
+* [Pulp Operator quickstart](site:pulp-operator/docs/admin/tutorials/quickstart-kubernetes/)
+
+## Local Filesystem (default)
+
+### Example
+
+In this example, the storage file permission and `MEDIA_ROOT` are overridden:
+
+```python
+STORAGES = {
+    "default": {
+        "BACKEND": "pulpcore.app.models.storage.FileSystem",
+        "OPTIONS": {
+            "file_permissions_mode": 0o600,
+            "directory_permissions_mode": 0o600,
+        },
+    },
+}
+MEDIA_ROOT="/custom/media/root"  # default: /var/lib/pulp/media
+```
+
+Notes:
+
+* The [`MEDIA_ROOT`](site:pulpcore/docs/admin/reference/settings/#media_root) setting specifies where Pulp
+will save the files.
+* Pulp customizes Django's default class, `django.core.files.storage.FileSystemStorage`. The original can't be used.
+* There are some other related global definitions provided by django:
+    * `MEDIA_URL`
+    * `FILE_UPLOAD_PERMISSIONS`
+    * `FILE_UPLOAD_DIRECTORY_PERMISSIONS`
+
+Comprehensive options for Local Filesystem can be found in
+[Django docs](https://docs.djangoproject.com/en/4.2/ref/files/storage/#django.core.files.storage.FileSystemStorage).
+
+## Amazon S3
+
+### Setup
+
+Before you can configure Amazon S3 storage to use with Pulp, ensure that you complete the following steps
+(consult the official Amazon S3 docs for precise steps).
+
+1. Set up an AWS account.
+2. Create an S3 bucket for Pulp to use.
+3. In AWS Identity and Access Management (IAM), create a user that Pulp can use to access your S3 bucket.
+4. Save the access key id and secret access key.
+
+### Example
+
+In this example, the storage uses a bucket called `pulp3` that is hosted in region `eu-central-1`:
+
+```python
+MEDIA_ROOT = ""
+STORAGES = {
+    "default": {
+        "BACKEND": "storages.backends.s3.S3Storage",
+        "OPTIONS": {
+            "access_key": 'AKIAIT2Z5TDYPX3ARJBA',
+            "secret_key": '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',
+            "bucket_name": 'pulp3',
+            "signature_version": "s3v4",
+            "addressing_style": "path",
+            "region_name": "eu-central-1",
+        },
+    },
+}
+```
+
+Notes:
+
+* `MEDIA_ROOT` must be set to an empty string.
+* You can omit `access_key` and `secret_key` if:
+    1. The system that hosts Pulp is running on AWS
+    2. And the [`instance_profile`](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) provides access to the S3 bucket
+
+Comprehensive options for Amazon S3 can be found in
+[`django-storages` docs](https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#configuration-settings).
+
+## S3 Compatible
+
+The same `storages.backends.s3.S3Storage` backend can be used for S3 Compatible API services, such as [Minio](https://min.io/), [Ceph/RADOS](https://docs.ceph.com/en/reef/man/8/rados/) and [Backblaze B2](https://www.backblaze.com/cloud-storage).
+
+The [django-storages](https://django-storages.readthedocs.io/en/latest/backends/s3_compatible/index.html) documentation
+provides a reference for setting up some of these services.
+
+## Azure Blob storage
+
+### Setup
+
+Before you can configure Azure Blob storage to use with Pulp, ensure that you complete the following steps
+(consult the official Azure documentation for precise steps).
+
+1. Set up an Azure account and create a storage account.
+2. In your storage account, create a container under `Blob` service.
+3. Obtain the access credentials so that you can later configure Pulp to access your Azure Blob storage. You can find the access credentials
+   at the storage account level, at Access keys (these are automatically generated).
+
+### Example
+
+In this example, the storage uses a container called `pulp3` with the `pulp-account` username.
+
+```python
+MEDIA_ROOT = ""
+STORAGES = {
+    "default": {
+        "BACKEND": "storages.backends.azure_storage.AzureStorage",
+        "OPTIONS": {
+            "account_name": 'pulp-account',
+            "azure_container": 'pulp3',
+            "account_key": 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==',
+            "expiration_secs": 60,
+            "overwrite_files": 'True',
+            "location": 'pulp3'
+        },
+    },
+}
+```
+
+Notes:
+
+* `MEDIA_ROOT` must be set to an empty string.
+
+Comprehensive options for Azure Blob can be found in
+[`django-storages` docs](https://django-storages.readthedocs.io/en/latest/backends/azure.html#configuration-settings).

docs/admin/guides/configure-pulp.md → docs/admin/guides/configure-pulp/index.md

@@ -1,4 +1,4 @@
-# Configure Pulp
+# Introduction
 
 Pulp uses [dynaconf](https://www.dynaconf.com/) for its settings which allows you
 to configure Pulp settings using various ways:

docs/admin/reference/settings.md

@@ -73,16 +73,55 @@ instructions on how to configure the database, refer to `database installation <
 
 ### DEFAULT_FILE_STORAGE
 
-By default, Pulp uses the local filesystem to store files. The default option which
-uses the local filesystem is `pulpcore.app.models.storage.FileSystem`.
+!!! warning "Deprecated in `3.70`"
+    The `DEFAULT_FILE_STORAGE` setting was deprecated in
+    [django `4.2`](https://docs.djangoproject.com/en/4.2/ref/settings/#default-file-storage)
+    and will be removed from pulpcore on `3.85`.
+    Between `3.70` and `3.85`, replace it with [`STORAGES`](#storages).
+
+### STORAGES
+
+!!! note "Added in `3.70`"
+    Starting in `3.70`, this should be used in the place of [`DEFAULT_FILE_STORAGE`](#default-file-storage).
+
+Pulp uses [django-storages](https://django-storages.readthedocs.io/en/latest/index.html) to support multiple storage backends.
+If no backend is configured, Pulp will by default use the local filesystem (`pulpcore.app.models.storage.FileSystem`).
+
+The storage setting has the form:
+
+```python
+STORAGES = {
+    "default": {
+        "BACKEND": "{storage-class}",
+        "OPTIONS": {
+            "{option-key}": "{option-value}",
+            ...
+        },
+    },
+}
+```
+
+To use another backend storage, you'll need to:
+
+1. Setup your storage and gather required credentials and specific configs.
+2. Ensure the `django-storage[s3|google|azure]` python package is installed. This depends on the installation method.
+3. Configure the default `BACKEND` storage class.
+4. Configure available `OPTIONS` for that backend.
+
+Learn more on the
+[Configure Storages](site:pulpcore/docs/admin/guides/configure-pulp/configure-storages/) guide.
+
+Overview of the integration with Pulp:
+
+- **Supported**: We support (test) Amazon S3 and Azure.
+- **Untested**: Other backends provided by `django-storages` may work as well, but we provide no guarantee.
+- **Known caveat**: Using SFTP Storage is not recommended in Pulp's current state, and doing so can lead to file corruption. This is because Pulp currently uses coroutines that seem to be incompatible with Django's SFTPStorage implementation.
 
-For more information about different Pulp storage options, see the
-`storage documentation `.
 
 ### REDIRECT_TO_OBJECT_STORAGE
 
 When set to `True` access to artifacts is redirected to the corresponding Cloud storage
-configured in `DEFAULT_FILE_STORAGE` using pre-authenticated URLs. When set to `False`
+configured in `STORAGES['default']['BACKEND']` using pre-authenticated URLs. When set to `False`
 artifacts are always served by the content app instead.
 
 Defaults to `True`; ignored for local file storage.
@@ -91,9 +130,10 @@ Defaults to `True`; ignored for local file storage.
 
 The location where Pulp will store files. By default this is `/var/lib/pulp/media`.
 
-This only affects storage location when `DEFAULT_FILE_STORAGE` is set to
-`pulpcore.app.models.storage.FileSystem`. See the `storage documentation ` for
-more info.
+This only affects storage location when `STORAGES['default']['BACKEND']` is set to
+`pulpcore.app.models.storage.FileSystem`.
+
+See the [storage documentation](site:pulpcore/docs/admin/guides/configure-pulp/configure-storages/) for more info.
 
 It should have permissions of:
 
@@ -188,7 +228,7 @@ It should have permissions of:
 ### CHUNKED_UPLOAD_DIR
 
 A relative path inside the DEPLOY_ROOT directory used exclusively for uploaded chunks. The
-uploaded chunks are stored in the default storage specified by `DEFAULT_FILE_STORAGE`. This
+uploaded chunks are stored in the default storage specified by `STORAGES['default']['BACKEND']`. This
 option allows users to customize the actual place where chunked uploads should be stored within
 the declared storage. The default, `upload`, is sufficient for most use cases. A change to
 this setting only applies to uploads created after the change.