Skip to content

Commit

Permalink
initial design thoughts
Browse files Browse the repository at this point in the history
  • Loading branch information
kaovilai committed Oct 31, 2024
1 parent db470a7 commit 7448a03
Show file tree
Hide file tree
Showing 3 changed files with 89 additions and 1 deletion.
76 changes: 76 additions & 0 deletions design/kopia-to-local-volume-provider.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Kopia Backup to Local Volume Provider (PVC/NFS)

_Note_: The preferred style for design documents is one sentence per line.
*Do not wrap lines*.
This aids in review of the document as changes to a line are not obscured by the reflowing those changes caused and has a side effect of avoiding debate about one or two space after a period.

_Note_: The name of the file should follow the name pattern `<short meaningful words joined by '-'>_design.md`, e.g:
`listener-design.md`.

## Abstract
Backups involving kopia are currently stored to blob storage only, this proposal aims to add support for storing backups to local volumes (PVC/NFS).

## Background
Users have asked for the ability to store backups to local volumes (PVC/NFS) for various reasons including:
- Compliance requirements
- Cost
- Existing infrastructure
- Performance
- Flexibility

## Goals
<!-- - A short list of things which will be accomplished by implementing this proposal.
- Two things is ok.
- Three is pushing it.
- More than three goals suggests that the proposal's scope is too large. -->

## Non Goals
<!-- - A short list of items which are:
- a. out of scope
- b. follow on items which are deliberately excluded from this proposal. -->


## High-Level Design
if backup storage location is a local volume (PVC/NFS) then:

Add a new backend type `PVCBackend` to `pkg/repository/config/config.go`

```go
const (
AWSBackend BackendType = "velero.io/aws"
AzureBackend BackendType = "velero.io/azure"
GCPBackend BackendType = "velero.io/gcp"
FSBackend BackendType = "velero.io/fs"
PVCBackend BackendType = "replicated.com/pvc"
)
```

This plugin will be responsible for storing backups to a local volume PVC which can be NFS backed.

When `PVCBackend` is configured, velero will use kopia to store backups to the local volume instead of blob storage.

## Detailed Design
A detailed design describing how the changes to the product should be made.

The names of types, fields, interfaces, and methods should be agreed on here, not debated in code review.
The same applies to changes in CRDs, YAML examples, and so on.

Ideally the changes should be made in sequence so that the work required to implement this design can be done incrementally, possibly in parallel.

## Alternatives Considered
If there are alternative high level or detailed designs that were not pursued they should be called out here with a brief explanation of why they were not pursued.

## Security Considerations
If this proposal has an impact to the security of the product, its users, or data stored or transmitted via the product, they must be addressed here.

## Compatibility
A discussion of any compatibility issues that need to be considered

## Implementation
A description of the implementation, timelines, and any resources that have agreed to contribute.

## Open Issues
<!-- A discussion of issues relating to this proposal for which the author does not know the solution. This section may be omitted if there are none. -->

Velero Logs, Velero Downloads will still not work and will be solved in [Download server for Velero client #6167
](https://github.com/vmware-tanzu/velero/issues/6167)
3 changes: 2 additions & 1 deletion pkg/repository/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ const (
AzureBackend BackendType = "velero.io/azure"
GCPBackend BackendType = "velero.io/gcp"
FSBackend BackendType = "velero.io/fs"
PVCBackend BackendType = "replicated.com/pvc"

// CredentialsFileKey is the key within a BSL config that is checked to see if
// the BSL is using its own credentials, rather than those in the environment
Expand Down Expand Up @@ -109,7 +110,7 @@ func GetBackendType(provider string, config map[string]string) BackendType {
}

func IsBackendTypeValid(backendType BackendType) bool {
return (backendType == AWSBackend || backendType == AzureBackend || backendType == GCPBackend || backendType == FSBackend)
return (backendType == AWSBackend || backendType == AzureBackend || backendType == GCPBackend || backendType == FSBackend || backendType == PVCBackend)
}

// GetRepoIdentifier returns the string to be used as the value of the --repo flag in
Expand Down
11 changes: 11 additions & 0 deletions pkg/repository/provider/unified_repo.go
Original file line number Diff line number Diff line change
Expand Up @@ -444,6 +444,8 @@ func getStorageType(backupLocation *velerov1api.BackupStorageLocation) string {
return udmrepo.StorageTypeGcs
case repoconfig.FSBackend:
return udmrepo.StorageTypeFs
case repoconfig.PVCBackend:
return udmrepo.StorageTypeFs

Check warning on line 448 in pkg/repository/provider/unified_repo.go

View check run for this annotation

Codecov / codecov/patch

pkg/repository/provider/unified_repo.go#L447-L448

Added lines #L447 - L448 were not covered by tests
default:
return ""
}
Expand Down Expand Up @@ -567,6 +569,15 @@ func getStorageVariables(backupLocation *velerov1api.BackupStorageLocation, repo
}
result[udmrepo.StoreOptionOssRegion] = strings.Trim(region, "/")
result[udmrepo.StoreOptionFsPath] = config["fspath"]
// LVP override fspath if empty
if backupLocation.Spec.Provider == "replicated.com/pvc" && result[udmrepo.StoreOptionFsPath] == "" {
result[udmrepo.StoreOptionFsPath] =
"/var/velero-local-volume-provider" + //default root https://github.com/replicatedhq/local-volume-provider/blob/7c78cfd4d12b7ad9614d5270413cb286fd081cae/pkg/plugin/util.go#L14C7-L14C18
backupLocation.Spec.ObjectStorage.Bucket
// prefix will be handled by pkg/repository/udmrepo/kopialib/backend/file_system.go
// on backuprepo create, this path will be created by
// /go/pkg/mod/github.com/project-velero/[email protected]/repo/blob/filesystem/filesystem_storage.go L348
}

Check warning on line 580 in pkg/repository/provider/unified_repo.go

View check run for this annotation

Codecov / codecov/patch

pkg/repository/provider/unified_repo.go#L574-L580

Added lines #L574 - L580 were not covered by tests

if backupRepoConfig != nil {
if v, found := backupRepoConfig[udmrepo.StoreOptionCacheLimit]; found {
Expand Down

0 comments on commit 7448a03

Please sign in to comment.