BackupStorageLocation is invalid, marking as unavailable

[Veeraraghavans]

What steps did you take and what happened:
Was using installation steps to deploy the Velero. Created Azure Storage resources and its permission followed workload identity method to create resource. It gave following error

time="2024-05-24T12:21:37Z" level=error msg="Error getting backup store for this location" backupLocation=velero/velero controller=backup-sync error="rpc error: code = Unknown desc = storageAccount is required in BSL" error.file="/go/pkg/mod/github.com/vmware-tanzu/[email protected]/pkg/util/azure/storage.go:70" error.function=github.com/vmware-tanzu/velero/pkg/util/azure.NewStorageClient logSource="pkg/controller/backup_sync_controller.go:101"
time="2024-05-24T12:21:42Z" level=error msg="Error getting a backup store" backup-storage-location=velero/velero controller=backup-storage-location error="rpc error: code = Unknown desc = storageAccount is required in BSL" error.file="/go/pkg/mod/github.com/vmware-tanzu/[email protected]/pkg/util/azure/storage.go:70" error.function=github.com/vmware-tanzu/velero/pkg/util/azure.NewStorageClient logSource="pkg/controller/backup_storage_location_controller.go:137"
time="2024-05-24T12:21:42Z" level=info msg="BackupStorageLocation is invalid, marking as unavailable" backup-storage-location=velero/velero controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:120"
time="2024-05-24T12:21:42Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/1/0, BackupStorageLocation \"velero\" is unavailable: rpc error: code = Unknown desc = storageAccount is required in BSL)" controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:178"
time="2024-05-24T12:21:42Z" level=warning msg="There is no existing BackupStorageLocation set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:187"
time="2024-05-24T12:22:37Z" level=error msg="Error getting backup store for this location" backupLocation=velero/velero controller=backup-sync error="rpc error: code = Unknown desc = storageAccount is required in BSL" error.file="/go/pkg/mod/github.com/vmware-tanzu/[email protected]/pkg/util/azure/storage.go:70" error.function=github.com/vmware-tanzu/velero/pkg/util/azure.NewStorageClient logSource="pkg/controller/backup_sync_controller.go:101"

I created backup resource with following command

velero backup-location create default --provider=azure --bucket=azwexxxvelero --config storageAccount=azwexxxveleroacc,resourceGroup=azweyyy-rg,subscriptionId=id,storageAccountKeyEnvVar=key --default

Post above command it gave following error:

time="2024-05-24T12:34:42Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/2/0, BackupStorageLocation \"velero\" is unavailable: rpc error: code = Unknown desc = storageAccount is required in BSL)" controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:178"
time="2024-05-24T12:35:37Z" level=error msg="Error getting backup store for this location" backupLocation=velero/velero2 controller=backup-sync error="rpc error: code = Unknown desc = no storage account access key with key <access-key> found in credential" error.file="/go/pkg/mod/github.com/vmware-tanzu/[email protected]/pkg/util/azure/storage.go:136" error.function=github.com/vmware-tanzu/velero/pkg/util/azure.GetStorageAccountCredentials logSource="pkg/controller/backup_sync_controller.go:101"
time="2024-05-24T12:35:37Z" level=error msg="Error getting backup store for this location" backupLocation=velero/velero controller=backup-sync error="rpc error: code = Unknown desc = storageAccount is required in BSL" error.file="/go/pkg/mod/github.com/vmware-tanzu/[email protected]/pkg/util/azure/storage.go:70" error.function=github.com/vmware-tanzu/velero/pkg/util/azure.NewStorageClient logSource="pkg/controller/backup_sync_controller.go:101"
time="2024-05-24T12:35:42Z" level=error msg="Error getting a backup store" backup-storage-location=velero/velero controller=backup-storage-location error="rpc error: code = Unknown desc = storageAccount is required in BSL" error.file="/go/pkg/mod/github.com/vmware-tanzu/[email protected]/pkg/util/azure/storage.go:70" error.function=github.com/vmware-tanzu/velero/pkg/util/azure.NewStorageClient logSource="pkg/controller/backup_storage_location_controller.go:137"
time="2024-05-24T12:35:42Z" level=info msg="BackupStorageLocation is invalid, marking as unavailable" backup-storage-location=velero/velero controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:120"
time="2024-05-24T12:35:42Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/2/0, BackupStorageLocation \"velero\" is unavailable: rpc error: code = Unknown desc = storageAccount is required in BSL)" controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:178"
time="2024-05-24T12:35:42Z" level=error msg="Error getting a backup store" backup-storage-location=velero/velero2 controller=backup-storage-location error="rpc error: code = Unknown desc = no storage account access key with key <access-key> found in credential" error.file="/go/pkg/mod/github.com/vmware-tanzu/[email protected]/pkg/util/azure/storage.go:136" error.function=github.com/vmware-tanzu/velero/pkg/util/azure.GetStorageAccountCredentials logSource="pkg/controller/backup_storage_location_controller.go:137"

Output of backup location

NAME      PROVIDER   BUCKET/PREFIX    PHASE         LAST VALIDATED                   ACCESS MODE   DEFAULT
default   azure      azwexxxvelero Unavailable   2024-05-24 15:47:52 +0200 CEST   ReadWrite     true

What did you expect to happen:

Expect the backup location is enabled to create backup of my K8s cluster.

Environment:

• helm version (use helm version): version.BuildInfo{Version:"v3.13.3"
• helm chart version and app version (use helm list -n <YOUR NAMESPACE>):

NAME    NAMESPACE       REVISION        UPDATED                                         STATUS          CHART           APP VERSION
velero  velero          1               2024-05-23 16:54:17.422595776 +0200 CEST        deployed        velero-6.2.0    1.13.0   

• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.13", GitCommit:"592eca05be27f7d927d0b25cbb4241d75a9574bf", GitTreeState:"clean", BuildDate:"2022-10-12T10:57:16Z", GoVersion:"go1.17.13", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.6", GitCommit:"16a4d7bedbaa618a21412ea5a8758b1382da58ff", GitTreeState:"clean", BuildDate:"2023-10-09T14:44:35Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"} 

• Cloud provider or hardware configuration: Azure

Same issue logged by me here

[blackpiglet]

After installing the Velero environment, you created an additional BSL?
Does the BSL created with the Helm installation work?

[Veeraraghavans]

Thanks @blackpiglet for answering. I created additional BSL still the error remains the same as created along with Helm installation in "Unavailable" phase

velero backup-location get
NAME      PROVIDER   BUCKET/PREFIX    PHASE         LAST VALIDATED                   ACCESS MODE   DEFAULT
default   azure      azwexxxvelero    Unavailable   2024-05-27 13:15:09 +0200 CEST   ReadWrite     true
velero    azure      azwexxxvelero    Unavailable   2024-05-27 13:14:49 +0200 CEST   ReadWrite     
velero2   azure      azwexxxvelero   Unavailable   2024-05-27 13:14:49 +0200 CEST   ReadWrite  

[ksudarsh00]

Try running this command.

velero backup-location create default --provider=azure --bucket=azwexxxvelero --config storageAccount=azwexxxveleroacc --config resourceGroup=azweyyy-rg --config subscriptionId=id --config storageAccountKeyEnvVar=key --default

[Veeraraghavans]

I fixed it myself thanks @ksudarsh00 @blackpiglet for your input. I recreated the secret it worked.

[idanme-tr]

@Veeraraghavans Sorry for jumping in.
Could you maybe elaborate what was missing in the secret you recreated?

Thanks