Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing Kopia Backup to Local Volume Provider #8358

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions design/kopia-to-local-volume-provider.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Kopia Backup to Local Volume Provider (PVC/NFS)

_Note_: The preferred style for design documents is one sentence per line.
*Do not wrap lines*.
This aids in review of the document as changes to a line are not obscured by the reflowing those changes caused and has a side effect of avoiding debate about one or two space after a period.

_Note_: The name of the file should follow the name pattern `<short meaningful words joined by '-'>_design.md`, e.g:
`listener-design.md`.

## Abstract
Backups involving kopia are currently stored to blob storage only, this proposal aims to add support for storing backups to local volumes (PVC/NFS).

## Background
Users have asked for the ability to store backups to local volumes (PVC/NFS) for various reasons including:
- Compliance requirements
- Cost
- Existing infrastructure
- Performance
- Flexibility

## Goals
<!-- - A short list of things which will be accomplished by implementing this proposal.
- Two things is ok.
- Three is pushing it.
- More than three goals suggests that the proposal's scope is too large. -->

## Non Goals
<!-- - A short list of items which are:
- a. out of scope
- b. follow on items which are deliberately excluded from this proposal. -->


## High-Level Design
if backup storage location is a local volume (PVC/NFS) then:

Add a new backend type `PVCBackend` to `pkg/repository/config/config.go`

```go
const (
AWSBackend BackendType = "velero.io/aws"
AzureBackend BackendType = "velero.io/azure"
GCPBackend BackendType = "velero.io/gcp"
FSBackend BackendType = "velero.io/fs"
PVCBackend BackendType = "replicated.com/pvc"
)
```

This plugin will be responsible for storing backups to a local volume PVC which can be NFS backed.

When `PVCBackend` is configured, velero will use kopia to store backups to the local volume instead of blob storage.

## Detailed Design
A detailed design describing how the changes to the product should be made.

The names of types, fields, interfaces, and methods should be agreed on here, not debated in code review.
The same applies to changes in CRDs, YAML examples, and so on.

Ideally the changes should be made in sequence so that the work required to implement this design can be done incrementally, possibly in parallel.

## Alternatives Considered
If there are alternative high level or detailed designs that were not pursued they should be called out here with a brief explanation of why they were not pursued.

## Security Considerations
If this proposal has an impact to the security of the product, its users, or data stored or transmitted via the product, they must be addressed here.

## Compatibility
A discussion of any compatibility issues that need to be considered

## Implementation
A description of the implementation, timelines, and any resources that have agreed to contribute.

## Open Issues
<!-- A discussion of issues relating to this proposal for which the author does not know the solution. This section may be omitted if there are none. -->

Velero Logs, Velero Downloads will still not work and will be solved in [Download server for Velero client #6167
](https://github.com/vmware-tanzu/velero/issues/6167)
3 changes: 2 additions & 1 deletion pkg/repository/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ const (
AzureBackend BackendType = "velero.io/azure"
GCPBackend BackendType = "velero.io/gcp"
FSBackend BackendType = "velero.io/fs"
PVCBackend BackendType = "replicated.com/pvc"

// CredentialsFileKey is the key within a BSL config that is checked to see if
// the BSL is using its own credentials, rather than those in the environment
Expand Down Expand Up @@ -109,7 +110,7 @@ func GetBackendType(provider string, config map[string]string) BackendType {
}

func IsBackendTypeValid(backendType BackendType) bool {
return (backendType == AWSBackend || backendType == AzureBackend || backendType == GCPBackend || backendType == FSBackend)
return (backendType == AWSBackend || backendType == AzureBackend || backendType == GCPBackend || backendType == FSBackend || backendType == PVCBackend)
}

// GetRepoIdentifier returns the string to be used as the value of the --repo flag in
Expand Down
11 changes: 11 additions & 0 deletions pkg/repository/provider/unified_repo.go
Original file line number Diff line number Diff line change
Expand Up @@ -444,6 +444,8 @@
return udmrepo.StorageTypeGcs
case repoconfig.FSBackend:
return udmrepo.StorageTypeFs
case repoconfig.PVCBackend:
return udmrepo.StorageTypeFs

Check warning on line 448 in pkg/repository/provider/unified_repo.go

View check run for this annotation

Codecov / codecov/patch

pkg/repository/provider/unified_repo.go#L447-L448

Added lines #L447 - L448 were not covered by tests
default:
return ""
}
Expand Down Expand Up @@ -567,6 +569,15 @@
}
result[udmrepo.StoreOptionOssRegion] = strings.Trim(region, "/")
result[udmrepo.StoreOptionFsPath] = config["fspath"]
// LVP override fspath if empty
if backupLocation.Spec.Provider == "replicated.com/pvc" && result[udmrepo.StoreOptionFsPath] == "" {
result[udmrepo.StoreOptionFsPath] =
"/var/velero-local-volume-provider" + //default root https://github.com/replicatedhq/local-volume-provider/blob/7c78cfd4d12b7ad9614d5270413cb286fd081cae/pkg/plugin/util.go#L14C7-L14C18
backupLocation.Spec.ObjectStorage.Bucket
// prefix (if available, not supported by replicated/pvc) will be handled by pkg/repository/udmrepo/kopialib/backend/file_system.go
// on backuprepo create, this path will be created by
// /go/pkg/mod/github.com/project-velero/[email protected]/repo/blob/filesystem/filesystem_storage.go L348
}

Check warning on line 580 in pkg/repository/provider/unified_repo.go

View check run for this annotation

Codecov / codecov/patch

pkg/repository/provider/unified_repo.go#L574-L580

Added lines #L574 - L580 were not covered by tests

if backupRepoConfig != nil {
if v, found := backupRepoConfig[udmrepo.StoreOptionCacheLimit]; found {
Expand Down
Loading