Bucket Repo

This is a lightweight artifacts repository with low memory footprint which can be used as a minimal replacement for Nexus. It is able to cache artifacts from a remote repository on a local filesystem volume and also to store them on a cloud storage bucket via go-cloud.

It can be deployed either as a side-car container to a Kubernetes build pod or as a standalone service.

Getting Started

Configuration

The default configuration enables only the local file cache:

http:
    addr: ":8080"

storage:
    enabled: false
    bucket_url: "gs://bucketrepo"

cache:
    base_dir: "/tmp/bucketrepo"

repositories:
    - url: "https://repo1.maven.org/maven2"
    - url: "http://uk.maven.org/maven2/"

The header (for example Bearer token authentication) and timeout can be modified for every remote repository:

...

repositories:
    - url: "https://repo1.maven.org/maven2"
      timeout: "30s"
    - url: "http://uk.maven.org/maven2/"
    - url: "http://my.private.maven.repo"
      timeout: "10s"
      header:
          Authorization: "Bearer <Token>"

The cloud storage can be enabled by providing a bucket URL:

storage:
    enabled: true
    bucket_url: "gs://mybucket"
    # if necessary is possible to use Path prefix
    prefix: "myfolder" 

Also the TLS and basic authentication can be configured with:

http:
    addr: ":8080"
    https: true
    crt: "/certs/domain.crt"
    key: "/certs/domain.key"
    username: "myuser"
    password: "mypassword"

Note that the basic authentication is turned off when HTTPS is disabled.

You can make artifacts not used in a while be removed from disk storage (cloud storage is not touched):

cache:
    base_dir: "/tmp/bucketrepo"
    clean_interval: "24h"
    cache_time: "720h"

The clean interval of 24 hours is the default, while the cache time doesn't have a default. This means that cleaning of the cache isn't enabled by default.

Note

For this to work the access times needs to be recorded in the file system used for caching. Typically it is.

If you do want cloud storage to be cleaned you can for example in the case of s3 add a lifecycle policy to the bucket.

Supported Artifacts

This repository has been tested with maven and helm tools, but it can also store other artifacts.

Installing

Kubernetes

The repository service can be installed in a Kubernetes cluster using helm. First, you need to add the jenkins-x chart repository to your helm repositories:

helm repo add jx3 https://jenkins-x-charts.github.io/repo
helm repo update

You can now install the chart with:

helm install jx3/bucketrepo --name bucketrepo

S3 compatible buckets

When using an S3 compatible bucket deployed locally (like Minio, Ceph...) you might need to configure bucketrepo to trust the SSL certificate for the bucket.

In order to that, you can add AWS_CA_BUNDLE to envSecrets with path of the CA file, and mount that file using extraConfig that looks like this:

extraConfig:
  ca-certificates.crt: |
    -----BEGIN CERTIFICATE-----
    ...

the certificate can be retrieved using:

kubectl -n minio get secrets minio1-tls -o yaml | ksd (public.crt)

the bucketUrl should look like this:

"s3://bucketrepo?endpoint=https://minio.minio.svc.cluster.local&s3ForcePathStyle=true&region=us-east-1"

Locally

The repository can be started in a docker container usinged the latest released image:

docker run -v $(pwd)/config:/config -p 8080:8080 gcr.io/jenkinsxio/bucketrepo:0.1.12 -config-path=/config

Or it can be built and run with:

make build
bin/bucketrepo -config-path=config -log-level=debug

Maven Configuration

bucketrepocan be configured as a mirror by adding the following in ~/.m2/settings.xml file:

<settings>
<mirrors>
    <mirror>
      <id>bucketrepo</id>
      <name>bucketrepo- mirror</name>
      <url>http://localhost:8080/bucketrepo/</url>
      <mirrorOf>*</mirrorOf>
    </mirror>
  </mirrors>
</settings>

And as a repository by adding the following in the pom.xml file:

<repositories>
    <repository>
        <id>bucketrepo</id>
        <url>http://localhost:8080/bucketrepo/</url>
    </repository>
</repositories>

<distributionManagement>
    <snapshotRepository>
        <id>snapshots</id>
        <url>http://localhost:8080/bucketrepo/deploy/maven-snapshots/</url>
    </snapshotRepository>
    <repository>
        <id>releases</id>
        <url>http://localhost:8080/bucketrepo/deploy/maven-releases/</url>
    </repository>
</distributionManagement>

Acknowledgments

This project is originally based on nexus-minimal. Thank you atsman for creating that project.

License

MIT