README.md

Introduction

Replicator for CORTX cross cluster/region replication for S3 objects.

Description

This module performs following

1. Receive "replication jobs" from replication manager.
2. Perform the replication
3. Acknowledge the replication jobs to replication manager.
4. Maintain a list of inprogress jobs and return on query.

Quickstart

Create a virtualenv.

python3 -m venv s3env

Activate virtualenv s3env.

source s3env/bin/activate

Clean earlier install.

python3 setup.py clean --all

Install dependencies in active virtualenv.

pip3 install -r requirements.txt

Install the development/test dependencies from devel-requirements.txt

pip3 install  -e .[development]

Install the package.

python3 setup.py install

Start the replicator.

python3 -m s3replicator

Run simple system tests.

py.test tests/system/test_replicator.py -v

Run load transfer tests.

python3 tests/system/load_transfer_test.py

Configurations

Number of connections should be greater than count of replications run in parallel, and ideally it should be twice. In replicator config.yaml, its recommended to have max_connections_per_s3_session = 2 * max_replications