Document backup and restore procedures

closes pulp#765

CHANGES/765.doc

@@ -0,0 +1 @@
+Added steps to configure and run backup/restore procedure.

api/v1alpha1/repo_manager_backup_types.go

@@ -23,6 +23,9 @@ import (
 
 // PulpBackupSpec defines the desired state of PulpBackup
 type PulpBackupSpec struct {
+
+	// Name of the deployment type. Can be one of {galaxy,pulp}.
+	// +kubebuilder:validation:Enum:=pulp;galaxy
 	// +operator-sdk:csv:customresourcedefinitions:type=spec
 	DeploymentType string `json:"deployment_type"`
 
@@ -32,6 +35,7 @@ type PulpBackupSpec struct {
 	DeploymentName string `json:"deployment_name"`
 
 	// +kubebuilder:default:="pulp"
+	// +kubebuilder:validation:Optional
 	// +operator-sdk:csv:customresourcedefinitions:type=spec
 	InstanceName string `json:"instance_name"`
 

api/v1alpha1/repo_manager_restore_types.go

@@ -22,6 +22,8 @@ import (
 
 // PulpRestoreSpec defines the desired state of PulpRestore
 type PulpRestoreSpec struct {
+
+	// Name of the deployment type. Can be one of {galaxy,pulp}.
 	// +kubebuilder:validation:Optional
 	// +kubebuilder:validation:Enum:=galaxy;pulp
 	// +kubebuilder:default:="pulp"

api/v1alpha1/repo_manager_types.go

@@ -38,6 +38,7 @@ type PulpSpec struct {
 	// Name of the deployment type.
 	// Default: "pulp"
 	// +kubebuilder:default:="pulp"
+	// +kubebuilder:validation:Enum:=pulp;galaxy
 	// +operator-sdk:csv:customresourcedefinitions:type=spec,xDescriptors={"urn:alm:descriptor:com.tectonic.ui:hidden"}
 	DeploymentType string `json:"deployment_type,omitempty"`
 

config/crd/bases/repo-manager.pulpproject.org_pulpbackups.yaml

@@ -883,6 +883,10 @@ spec:
                 description: Name of the deployment to be backed up
                 type: string
               deployment_type:
+                description: Name of the deployment type. Can be one of {galaxy,pulp}.
+                enum:
+                - pulp
+                - galaxy
                 type: string
               instance_name:
                 default: pulp
@@ -897,7 +901,6 @@ spec:
                 type: string
             required:
             - deployment_type
-            - instance_name
             - postgres_configuration_secret
             type: object
           status:

config/crd/bases/repo-manager.pulpproject.org_pulprestores.yaml

@@ -61,6 +61,7 @@ spec:
                 type: string
               deployment_type:
                 default: pulp
+                description: Name of the deployment type. Can be one of {galaxy,pulp}.
                 enum:
                 - galaxy
                 - pulp

config/crd/bases/repo-manager.pulpproject.org_pulps.yaml

@@ -5782,6 +5782,9 @@ spec:
               deployment_type:
                 default: pulp
                 description: 'Name of the deployment type. Default: "pulp"'
+                enum:
+                - pulp
+                - galaxy
                 type: string
               file_storage_access_mode:
                 description: The file storage access mode. This field should be used

controllers/backup/README.md

@@ -38,7 +38,7 @@ PulpBackupSpec defines the desired state of PulpBackup
 
 | Field | Description | Scheme | Required |
 | ----- | ----------- | ------ | -------- |
-| deployment_type |  | string | true |
+| deployment_type | Name of the deployment type. Can be one of {galaxy,pulp}. | string | true |
 | deployment_name | Name of the deployment to be backed up | string | true |
 | instance_name |  | string | true |
 | backup_pvc | Name of the PVC to be used for storing the backup | string | true |

controllers/restore/README.md

@@ -38,7 +38,7 @@ PulpRestoreSpec defines the desired state of PulpRestore
 
 | Field | Description | Scheme | Required |
 | ----- | ----------- | ------ | -------- |
-| deployment_type |  | string | true |
+| deployment_type | Name of the deployment type. Can be one of {galaxy,pulp}. | string | true |
 | backup_source | backup source | string | true |
 | deployment_name | Name of the deployment to be restored to | string | true |
 | backup_name | Name of the backup custom resource | string | true |

docs/backup_and_restore/config_running.md

@@ -0,0 +1,93 @@
+# Configuring and Running
+
+
+All the configurations needed to run the backup or restore procedures are made through `PulpBackup` or `PulpRestore` CRs.
+
+To get the list of fields available in each CR, check [`PulpBackupSpec`](/pulp_operator/backup/#pulpbackupspec) and [`PulpRestoreSpec`](/pulp_operator/restore/#pulprestorespec).
+
+## Backup
+
+To configure the backup controller, create a manifest file with the definition of [PulpBackup CR](/pulp_operator/backup/#pulpbackupspec).  
+For example:
+```
+---
+apiVersion: repo-manager.pulpproject.org/v1alpha1
+kind: PulpBackup
+metadata:
+  name: pulpbackup-sample
+spec:
+  deployment_name: pulp
+  deployment_type: pulp
+  backup_storage_class: standard
+  admin_password_secret: example-pulp-admin-password
+  postgres_configuration_secret: pulp-postgres-configuration
+```
+
+In the above sample we defined:
+
+* the name of `Pulp` instance (`deployment_name`), which can be gathered with:
+```
+$ kubectl get pulp
+NAME   AGE
+pulp   3m30s
+```
+
+* the type of deployment (`deployment_type`), in this case `pulp` (but could also be `galaxy` depending on the installation).
+* the name of `StorageClass` used to provision the `PVC` to store the backup data (`backup_storage_class`).
+* the name of the `Secret` with Pulp admin password (`admin_password_secret`), which can be get by:
+```
+$ kubectl get pulp pulp -ojsonpath='{.spec.admin_password_secret}{"\n"}'
+pulp-admin-password
+```
+
+* the name of the `Secret` with PostgreSQL credentials and connection information (`postgres_configuration_secret`).
+
+After finishing to configure the file, apply the configuration and the Operator will start the backup:
+```
+kubectl apply -f <backup_cr_file>.yaml
+```
+
+
+## Restore
+
+
+To configure the restore controller, create a manifest file with the definition of [PulpRestore CR](/pulp_operator/restore/#pulprestorespec).
+For example:
+```
+---
+apiVersion: repo-manager.pulpproject.org/v1alpha1
+kind: PulpRestore
+metadata:
+  name: pulprestore-sample
+spec:
+  backup_name: pulpbackup-sample
+  deployment_name: pulp
+```
+
+In the above sample we defined:
+
+* the name of `PulpBackup` instance (`backup_name`)
+* the name of `Pulp` instance (`deployment_name`). This should be the same defined in the `PulpBackup` CR.
+
+After finishing to configure the file, apply the configuration and the Operator will start the restore:
+```
+kubectl apply -f <restore_cr_file>.yaml
+```
+
+By default, the restore procedure will reprovision the environment with a single replica of each component. This is to make it easier to review the restore status and the environment health.  
+It is also possible to restore with the same number of replicas running when the backup was made. To do so, just set the `keep_replicas` field to true, for example:
+```
+---
+apiVersion: repo-manager.pulpproject.org/v1alpha1
+kind: PulpRestore
+metadata:
+  name: pulprestore-sample
+spec:
+  backup_name: pulpbackup-sample
+  deployment_name: pulp
+  keep_replicas: true
+```
+
+
+After finishing to restore the environment, the operator will create a `ConfigMap` called *`restore-lock`*. It is used to prevent a new controller reconciliation loop to run and override any data changed/created with the "old" data from backup.  
+To allow the restore controller to run again, delete the *restore-lock* `ConfigMap`.

docs/backup_and_restore/overview.md

@@ -0,0 +1,38 @@
+# Overview of backup and restore operations
+
+
+In addition to provisioning Pulp components, the Operator can also be used to backup and restore them.
+
+!!! note
+    Before starting a backup, make sure that the namespace has enough storage quota available.
+
+## Backup
+The backup procedure creates a *manager* `Pod` which will be used to execute all the backup tasks:
+
+* run a `pg_dump` (database dump) on Pulp's database
+* do a copy of the Pulp CR instance defined in `deployment_name`
+* do a copy of the `Secrets`
+* do a copy of `/var/lib/pulp` directory
+* delete the *manager* `Pod` to not consume resources
+
+These data will be stored in a new PVC defined in PulpBackup CR (`backup_pvc` or `backup_storage_class`).
+
+
+!!! note
+    The current version of the Operator does **not** execute backups of **external** PostgreSQL instances yet.
+
+
+!!! notes
+    Considering that files stored in Object Storage (like `AWS S3` and/or `Azure Blob`) are not kept in `/var/lib/pulp` directory, they will **not** be copied. If you still need to do a backup of the artifacts stored in Object Storage, please, contact your cloud provider to check the procedure to do so.
+
+
+## Restore
+The restore procedure also creates a *manager* `Pod` to execute all the tasks:
+
+* restore the `Secrets`
+* restore Pulp CR instance
+* restore Pulp database
+* restore `/var/lib/pulp` directory
+* delete the *manager* `Pod` to not consume resources
+
+All data restored comes from the PVC defined in PulpRestore CR (`backup_pvc`).

docs/faq.md

@@ -46,3 +46,9 @@ Some resources are provisioned by the Operator, but [**no reconciliation**](/pul
 ### **Which fields are reconciled when the Operator is set as `managed`?**
 
 All fields from `Spec` *should* be reconciled on *Deployments*, *Services*, *Routes* and *Ingresses* objects.
+
+
+### **I created a new PulpRestore CR, but the restore procedure is not running again. Checking the operator logs I found the message "*PulpRestore lock ConfigMap found. No restore procedure will be executed!*"**
+
+After the Operator finishes executing a restore procedure it creates a `ConfigMap` called *`restore-lock`*. This `ConfigMap` is used to control the restore reconciliation loop and avoid it ovewriting all files or `Secrets` with data from an old backup.  
+If you still want to run the restore, just delete the *`restore-lock`* `ConfigMap` and recreate the PulpRestore CR.

mkdocs.yml

@@ -46,6 +46,9 @@ nav:
       - Pod Disruption Budget: configuring/pdb.md
       - Secrets: configuring/secrets.md
       - Disabling Reconciliation: configuring/unmanaged.md
+  - Backup and Restore:
+      - Overview: backup_and_restore/overview.md
+      - Configuring and Running: backup_and_restore/config_running.md
   - Changelog: CHANGES.md
   - FAQ: faq.md
   - Troubleshooting: