diff --git a/.licensei.toml b/.licensei.toml
index 2dfd636..70118ce 100644
--- a/.licensei.toml
+++ b/.licensei.toml
@@ -35,7 +35,7 @@ ignored = [
[header]
authors = ["Cisco"]
-ignorePaths = [".direnv", ".devenv", "vendor"]
+ignorePaths = [".direnv", ".devenv", "vendor", "examples"]
ignoreFiles = ["zz_generated.*.go"]
template = """// Copyright © :YEAR: :AUTHOR:
//
diff --git a/README.md b/README.md
index 45aaa0d..ea4de6b 100644
--- a/README.md
+++ b/README.md
@@ -10,8 +10,6 @@
**Secret Sync** exposes a generic way to interact with external secret storage systems like [HashiCorp Vault](https://www.vaultproject.io/) and provides a set of API models to interact and orchestrate the synchronization of secrets between them.
-This name was chosen in a rush, we are open to naming suggestions 😄
-
> [!IMPORTANT]
> This is an **early alpha version** and breaking changes are expected.
> As such, it is not recommended for usage in production.
@@ -27,40 +25,32 @@ This name was chosen in a rush, we are open to naming suggestions 😄
- Advanced templating capabilities for transforming secrets
| **Supported store** | **Status** |
-|--------------------------------------------------------------------------|--------|
-| [HashiCorp Vault](https://www.vaultproject.io) | alpha |
-| [Local Provider](#secret-store) | alpha |
-| [AWS Secrets Manager](https://aws.amazon.com/secrets-manager) | _planned_ |
-| [Google Secrets Manager](https://cloud.google.com/secret-manager) | _planned_ |
-| [Azure Key Vault](https://azure.microsoft.com/en-us/services/key-vault/) | _planned_ |
-| [Kubernetes Secret](https://kubernetes.io/) | _planned_ |
+|--------------------------------------------------------------------------|------------|
+| [HashiCorp Vault](https://www.vaultproject.io) | alpha |
+| [Local Provider](#secret-store) | alpha |
+| [AWS Secrets Manager](https://aws.amazon.com/secrets-manager) | _planned_ |
+| [Google Secrets Manager](https://cloud.google.com/secret-manager) | _planned_ |
+| [Azure Key Vault](https://azure.microsoft.com/en-us/services/key-vault/) | _planned_ |
+| [Kubernetes Secret](https://kubernetes.io/) | _planned_ |
Check details about upcoming features by visiting the [project issue](https://github.com/bank-vaults/secret-sync/issues) board.
## Goals
* Provide safe and simple way to work with secrets
-* Common API for secret management regardless of the store backend
-* Explicit control over the secret synchronization process
+* Expose common API for secret management regardless of the store backend
+* Give total control of the secret synchronization process
> Consider a situation where Dev teams need access to secrets from different environments.
-> Ops teams can provide access to secrets in the form of a sandboxed environment (e.g. new Vault instance) synced only with secrets Devs require; all in GitOps way.
-
-## Getting Started
-
-To get familiarized, we will show how you can use these tools to answer two questions:
-
-- **Ops**: How do I synchronize secrets from one secret storage systems to another?
-- **Dev**: How do I consume secrets to bootstrap my application?
+> Ops teams can provide access to secrets in the form of an isolated environment (e.g. new Vault instance) synced only with secrets Devs require; all in GitOps way.
-To answer the first question, we shall create some database secrets and synchronize them into Vault.
-For the second question, we will use some secrets from Vault to create an access file for an application.
+## Getting started
-You can find complete examples and instructions in the [EXAMPLE](EXAMPLE.md) file.
+To get familiarized, check out the collection of different [examples](examples) using this tool.
## Documentation
-Check out the documentation at [API documentation](docs/API.md) page or on [pkg.go.dev](https://pkg.go.dev/mod/github.com/bank-vaults/secret-sync).
+Check out the [project documentation](docs) or [pkg.go.dev](https://pkg.go.dev/mod/github.com/bank-vaults/secret-sync).
## Development
diff --git a/docs/API.md b/docs/API.md
index 6692833..21428fd 100644
--- a/docs/API.md
+++ b/docs/API.md
@@ -13,8 +13,6 @@ secretsStore:
storeName: storeSpec
```
-You can find all the Secret Store specifications in [pkg/apis/v1alpha1/secretstore.go](pkg/apis/v1alpha1/secretstore.go)
-
Store Spec: HashiCorp Vault*
@@ -68,8 +66,6 @@ sync:
- actionSpec
```
-You can find all the Sync Plan specifications in [pkg/apis/v1alpha1/syncjob_types.go](pkg/apis/v1alpha1/syncjob_types.go)
-
Action Spec: Synchronize a secret from reference
@@ -302,5 +298,5 @@ Note that only YAML configuration files are supported.
You can also provide optional params for CRON schedule to periodically sync secrets via `--schedule` flag.
All sync actions are indexed in logs based on their order in the sync plan config file.
-You can also use [pkg/storesync](pkg/storesync) package to run secret synchronization plan natively from Golang.
+You can also use [pkg/storesync](https://pkg.go.dev/github.com/bank-vaults/secret-sync/pkg/storesync) package to run secret synchronization plan natively from Golang.
This is how the CLI works as well.
diff --git a/docs/README.md b/docs/README.md
new file mode 100644
index 0000000..7f3c633
--- /dev/null
+++ b/docs/README.md
@@ -0,0 +1,5 @@
+## Documentation
+
+Here you can find documentation about the way **Secret Sync** works. Refer to:
+
+- [API](API.md) - Documentation on how to use different secret stores, synchronization policies, and templating rules.
diff --git a/examples/README.md b/examples/README.md
new file mode 100644
index 0000000..3b43368
--- /dev/null
+++ b/examples/README.md
@@ -0,0 +1,5 @@
+## Usage Examples
+
+Here you can find various examples on how you can use **Secret Sync**. Examples include:
+
+- [single-source-advanced](single-source-advanced) - Use a central source of truth to synchronize secrets into different secret stores. Uses templating for fine-grained synchronization.
diff --git a/examples/assets/gen-assets.sh b/examples/assets/gen-assets.sh
new file mode 100644
index 0000000..50f75fb
--- /dev/null
+++ b/examples/assets/gen-assets.sh
@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+
+# Requirement: https://github.com/terrastruct/d2
+
+ASSET_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+for d2_file in "$ASSET_DIR"/*.d2
+do
+ # Get files
+ png_file=${d2_file%".d2"}.png
+
+ # Generate PNG from diagram
+ d2 $d2_file $png_file
+done
diff --git a/examples/assets/native-approach.d2 b/examples/assets/native-approach.d2
new file mode 100644
index 0000000..d7a036c
--- /dev/null
+++ b/examples/assets/native-approach.d2
@@ -0,0 +1,42 @@
+direction: down
+
+Vault: {
+ label: Vault
+ shape: image
+ icon: https://www.svgrepo.com/download/448679/vault.svg
+}
+
+AWSSM: {
+ label: AWS Secrets Manager
+ shape: image
+ icon: https://icons.terrastruct.com/aws%2FSecurity%2C%20Identity%2C%20&%20Compliance%2FAWS-Secrets-Manager.svg
+}
+
+GCPSM: {
+ label: GCP Secret Manager
+ shape: image
+ icon: https://www.svgrepo.com/download/375492/secret-manager.svg
+}
+
+AzureKeyVault: {
+ label: Azure Key Vault
+ shape: image
+ icon: https://icons.terrastruct.com/azure%2FSecurity%20Service%20Color%2FKey%20Vaults.svg
+}
+
+Cluster: {
+ label: Cluster
+ icon: https://icons.terrastruct.com/azure%2F_Companies%2FKubernetes.svg
+
+ Local: {
+ Label: Secret
+ }
+ Workload: Workload
+
+ Local -> Workload
+}
+
+Cluster.Local <- Vault
+Cluster.Local <- AWSSM
+Cluster.Local <- GCPSM
+Cluster.Local <- AzureKeyVault
diff --git a/examples/assets/native-approach.png b/examples/assets/native-approach.png
new file mode 100644
index 0000000..8e3e81d
Binary files /dev/null and b/examples/assets/native-approach.png differ
diff --git a/examples/assets/secret-sync.d2 b/examples/assets/secret-sync.d2
new file mode 100644
index 0000000..469b64d
--- /dev/null
+++ b/examples/assets/secret-sync.d2
@@ -0,0 +1,137 @@
+direction: down
+
+Vault: {
+ label: Vault
+ shape: image
+ icon: https://www.svgrepo.com/download/448679/vault.svg
+}
+
+AWSSM: {
+ label: AWS Secrets Manager
+ shape: image
+ icon: https://icons.terrastruct.com/aws%2FSecurity%2C%20Identity%2C%20&%20Compliance%2FAWS-Secrets-Manager.svg
+ style: {
+ opacity: 0.4
+ }
+}
+
+GCPSM: {
+ label: GCP Secret Manager
+ shape: image
+ icon: https://www.svgrepo.com/download/375492/secret-manager.svg
+ style: {
+ opacity: 0.4
+ }
+}
+
+AzureKeyVault: {
+ label: Azure Key Vault
+ shape: image
+ icon: https://icons.terrastruct.com/azure%2FSecurity%20Service%20Color%2FKey%20Vaults.svg
+ style: {
+ opacity: 0.4
+ }
+}
+
+BVSecretSync: {
+ label: Bank-Vaults Secret Sync
+ shape: image
+ icon: https://raw.githubusercontent.com/bank-vaults/bank-vaults.dev/main/assets/icons/logo.svg
+}
+
+Vault -> BVSecretSync
+AWSSM -> BVSecretSync {
+ style: {
+ opacity: 0.4
+ }
+}
+GCPSM -> BVSecretSync {
+ style: {
+ opacity: 0.4
+ }
+}
+AzureKeyVault -> BVSecretSync {
+ style: {
+ opacity: 0.4
+ }
+}
+
+ClusterA: {
+ label: Cluster A
+ icon: https://icons.terrastruct.com/azure%2F_Companies%2FKubernetes.svg
+
+ style: {
+ opacity: 0.4
+ }
+
+ LocalA: Secret {
+ style: {
+ opacity: 0.4
+ }
+ }
+ WorkloadA: Workload {
+ style: {
+ opacity: 0.4
+ }
+ }
+
+ LocalA -> WorkloadA {
+ style: {
+ opacity: 0.4
+ }
+ }
+}
+
+ClusterB: {
+ label: Cluster B
+ icon: https://icons.terrastruct.com/azure%2F_Companies%2FKubernetes.svg
+
+ LocalB: {
+ label: Local Vault
+ shape: image
+ width: 80
+ icon: https://www.svgrepo.com/download/448679/vault.svg
+ }
+ WorkloadB: Workload
+
+ LocalB -> WorkloadB
+}
+
+LocalC: {
+ label: Tenant Environment
+ shape: image
+ width: 80
+ icon: https://www.svgrepo.com/download/375492/secret-manager.svg
+ style: {
+ opacity: 0.4
+ }
+}
+
+ClusterC: {
+ label: Cluster C
+ icon: https://icons.terrastruct.com/azure%2F_Companies%2FKubernetes.svg
+
+ style: {
+ opacity: 0.4
+ }
+
+ WorkloadC: Workload {
+ style: {
+ opacity: 0.4
+ }
+ }
+}
+
+LocalC -> ClusterC.WorkloadC
+
+BVSecretSync -> ClusterB.LocalB: Sync
+BVSecretSync -> ClusterA.LocalA: Sync {
+ style: {
+ opacity: 0.4
+ }
+}
+BVSecretSync -> LocalC: Sync {
+ style: {
+ opacity: 0.4
+ }
+}
diff --git a/examples/assets/secret-sync.png b/examples/assets/secret-sync.png
new file mode 100644
index 0000000..ea12f62
Binary files /dev/null and b/examples/assets/secret-sync.png differ
diff --git a/EXAMPLE.md b/examples/single-source-advanced/README.md
similarity index 72%
rename from EXAMPLE.md
rename to examples/single-source-advanced/README.md
index 6f458cb..84efced 100644
--- a/EXAMPLE.md
+++ b/examples/single-source-advanced/README.md
@@ -1,22 +1,23 @@
## Getting Started
-To get familiarized, we will show how you can use these tools to answer two questions:
+To get familiarized, we will show how you can use Secret Sync tool to cover two scenarios:
-- **Ops**: How do I synchronize secrets from one secret storage systems to another?
-- **Dev**: How do I consume secrets to bootstrap my application?
+- **Case 1**: Synchronize secrets from one secret store to another.
+ - We will use our local file store as a source of truth to synchronize some database secrets into Vault.
+- **Case 2**: Consume secrets to bootstrap an application.
+ - We will use Vault instance to fetch database secrets to our local store in a form of a configuration file for an application.
-To answer the first question, we shall create some database secrets and synchronize them into Vault.
-For the second question, we will use some secrets from Vault to create an access file for an application.
+*Note:* The same logic applies to any other combination of secret stores.
-### 1. Prepare environment
+### Step 1: Prepare environment
You will need the following tools to continue:
-- Docker
- Git
- Makefile
- Golang `>= 1.21`
To set up the environment, you can build from source:
+
```bash
git clone https://github.com/bank-vaults/secret-sync.git /tmp/secret-sync
cd /tmp/secret-sync
@@ -25,16 +26,17 @@ alias secret-sync="/tmp/secret-sync/build/secret-sync"
```
Alternatively, you can also use only Docker:
+
```bash
-alias secret-sync="docker run --rm -v /tmp:/tmp ghcr.io/bank-vaults/secret-sync:latest secret-sync"
+alias secret-sync="docker run --rm -v /tmp:/tmp ghcr.io/bank-vaults/secret-sync:v0.1.0 secret-sync"
```
-### 2. Define secret stores
+### Step 2: Define secret stores
-Documentation and examples on how to use different secret stores can be found in chapter [Secret Store](README.md#secret-store).
+#### Local store
-#### 2.1. Local store
Create a directory and a config file to use as the _local secret store_.
+
```bash
# Create local store directory
mkdir -p /tmp/example/local-store
@@ -47,8 +49,10 @@ secretsStore:
EOF
```
-#### 2.2. Vault store
+#### Vault store
+
Deploy Vault and create config file to use as the _Vault secret store_.
+
```bash
# Deploy a Vault instance
docker compose -f dev/vault/docker-compose.yml up -d
@@ -64,12 +68,11 @@ secretsStore:
EOF
```
-### 3. Define sync plans
-Documentation and examples on how to create a more extensive sync plan can be found in chapter [Sync Plan](README.md#sync-plan).
+### Step 3: Define sync plans
+
+#### Database secrets
-#### 3.1. Database secrets
-Define a sync plan for `db-host`, `db-user`, `db-pass` secrets.
-These secrets will be synced from our local to Vault secret store.
+Define a sync plan for `db-host`, `db-user`, `db-pass` secrets. These secrets will be synced from our local to Vault secret store.
```bash
cat < /tmp/example/db-secrets-sync.yml
@@ -81,10 +84,9 @@ sync:
EOF
```
-#### 3.1. Application access secret
-Define a sync plan for app-specific secret `app-access-config` created from various other secrets (e.g. database).
-This secret will be synced from Vault to our local secret store (as a file).
-It can also be synced against the same store to refresh the secret.
+#### Application access secret
+
+Define a sync plan for app-specific secret `app-access-config` created from various other secrets (e.g. database). This secret will be synced from Vault to our local secret store (as a file). It can also be synced against the same store to refresh the secret.
```bash
cat < /tmp/example/app-access-config-sync.yml
@@ -109,20 +111,21 @@ sync:
EOF
```
-### 4. Create database secrets
+### Step 4: Create database secrets
Create database access secrets in our local secret store.
+
```bash
echo -n "very-secret-hostname" > /tmp/example/local-store/db-host
echo -n "very-secret-username" > /tmp/example/local-store/db-user
echo -n "very-secret-password" > /tmp/example/local-store/db-pass
```
-### 5. Perform sync
+### Step 5: Perform sync
-Secret synchronization is performed using the [CLI](README.md#running-the-synchronization) by executing the sync plan between source and target secret stores.
+Secret synchronization is performed using the CLI by executing the sync plan between source and target secret stores.
-#### 5.1. Database secrets
+#### Database secrets
To synchronize database secrets from our local to Vault secret store, run:
@@ -141,7 +144,7 @@ If successful, your output should contain something like:
You can also navigate to the local Vault instance and verify these secrets.
-#### 5.2. Application access secret
+#### Application access secret
To synchronize application access secret from Vault to our local secret store, run:
@@ -157,7 +160,7 @@ cat /tmp/example/local-store/app-access-config
```
-### 6. Cleanup
+### Step 6: Cleanup
```bash
# Destroy Vault instance