Document backup repository password configuration method

[Lyndon-Li]

At present, Velero has a default backup repository password saved in secret velero-repo-credentials, the way to configure the password to to modify this secret.
However, this secret and its usage is not documented, many users don't know this.

[shubham-pampattiwar]

@Lyndon-Li I can work on this issue if needed.

[Lyndon-Li]

@shubham-pampattiwar
Sure, thanks for your efforts! I am assigning the issue to you.

[clouedoc]

Does this password cause the backups to S3 to be encrypted?
I'm a new user trying to ensure that my backups S3 backups are encrypted at rest

[draghuram]

@clouedoc, that is correct. Both Restic and Kopia (tools that Velero uses to do file system backups) encrypt data using the repo password. Check their documentation which has more details. But do check the limitations mentioned at https://velero.io/docs/v1.11/file-system-backup/#limitations.

[draghuram]

@shubham-pampattiwar, @Lyndon-Li, Documenting this would be very useful. Note that we should cover the case where user wants to modify the secret after few backups were done. Would this result in repo password being changed or it applies to only new repos?

[Lyndon-Li]

@draghuram
This is a very delicate case, or in another words, Velero doesn't support modifying password of an existing repo.
So this password configuration should only be used in the scenarios that after installing Velero, users want to use their own password to create the repo. The configuration should happen before any repo is created.

Configuring the secret after repos are created is possible, but users will see different manners for new and old repos:

• For new repos, they will use the new password in the secret
• For old repos, users will not be able to use the repos since the repos are using the old password but Velero is using the new one to connect them

cc @shubham-pampattiwar

[Lyndon-Li]

Changing password for existing repos is another task which involves lots of changes, yet for sure it is a useful feature, I've opened a new issue #6537 to track it.

[sseago]

This is further complicated by the fact that a BSL can be used in several clusters, so if any one cluster that uses this BSL for filesystem backups changes the password, all velero instances that use this need the same change.

[draghuram]

Agreed. I think the password configuration should be documented sooner rather than later.

[Lyndon-Li]

@sseago
Yes, once we support changing password for an existing repo, any instance of Velero that uses the BSL will need to use the new password, so we need a sophisticated design for this.
I will add this into the same issue #6537.

[bernardgut]

@clouedoc, that is correct. Both Restic and Kopia (tools that Velero uses to do file system backups) encrypt data using the repo password. Check their documentation which has more details. But do check the limitations mentioned at https://velero.io/docs/v1.11/file-system-backup/#limitations.

@draghuram the docs says

At present, Velero uses a static, common encryption key for all backup repositories it creates. This means that anyone who has access to your backup storage can decrypt your backup data. Make sure that you limit access to the backup storage appropriately.

What does this mean? This is not clear.

Does that mean that the key/password is stored en-clair in the backup storage next to the encrypted data ? Or that you are using the same keys in every Velero installs by any user ?

I am trying to understand how having a static key stored in-cluster somehow makes the encrypted Minio storage insecure for anyone who has access to the Minio instance... Can someone clarify?

Thanks
B.

[draghuram]

If you haven't changed repo password after installing Velero and did a file system or Snapshot datamover backup, Kopia/Restic repo will be created in the target storage. Now if a person has access to that storage, they can simply connect to that storage using kopia/restic CLI, and list backups and even restore them, because they can easily provide the repo password.

Hope this helps.

[bernardgut]

Wait I am even more confused.

If you haven't changed repo password after installing Velero and did a file system or Snapshot datamover backup, Kopia/Restic repo will be created in the target storage.

Yes and this data, on the target storage (S3), is encrypted correct ? The encryption key is what you call the "repo password" correct?

Now if a person has access to that storage, they can simply connect to that storage using kopia/restic CLI, and list backups and even restore them, because they can easily provide the repo password.

Wait. How can they "easily provide the repo password" if they only have access to that S3 bucket ("backup storage" in the docs) ? Isn't the encryption password (repo password) only stored in a secret in the source and destination clusters ? If that person does not have access to any of these clusters but has access to S3 bucket how can they provide the password and access your data ?

Trying to understand

thanks
B.

[draghuram]

Yes, repo password is what gets used in encrypting the data. And any one who has ever used Velero on any cluster knows the repo password because it is the same password every where (unless you changed it). So if I get access to your storage, I can easily try to read the contents using the hard-coded password and see if that works. It is similar to your home router coming with a standard password from the factory.

[bernardgut]

Ok thanks

So the answer to this question :

Or that you are using the same keys across all Velero installs by default?

is yes.

Maybe the docs should say

The password is stored in X, Y. The default password is a static password across all Velero installs. please update the password upon installing Velero before using it for the first time by following the docs at Z. Note that the password is static and cannot be upgraded after creating backups. The password secret will be copied by your Velero instance to all remote Velero instances sharing the same storage back-end. Manage access control to these resources accordingly,

Which is more factual than the cryptic information currently displayed. I think It would make it clearer for everyone.

What do you reckon ? I can do the PR if you point me to the right repo/location.

[draghuram]

Sure, you can submit PR with doc changes. See #6618 for the exact location where changes may need to go.