README.md

Introduction

This section describes common content for various modules in this project. See individual module readme for more details.

Quickstart

Ensure you have latest pip installed.

python3 -m pip install --upgrade pip

Perform flake8 checks on the python code.

make check

or

make flake8

Autocorrect formatting errors for flake8.

make autopep8

Run basic tests for each module. Clean, Build and test.

make test

Build & test script

A script which facilitates users to create virtual environment, build all modules and also starts s3-in-memory server, s3replicationmanager and s3replicator as systemd proccesses and then run load transfer tests for s3replicationmanager and s3replicator. This script will also be used by jenkins to build and run tests.

sh build_and_test.sh

Design documents

For more understanding on replication process checkout these sequence diagrams.

Troubleshooting

When running make test on MacOS, timeout in Makefile may fail and you need following.

brew install coreutils
sudo ln -s /usr/local/bin/gtimeout /usr/local/bin/timeout
export PATH=$PATH:/usr/local/bin/

Additionally MacOS returns non-zero error code without timeout even with --preserve-status, so you may want to for now manually test last 2 steps in make test. Will be fixed soon.

python3 -m s3replicationmanager
# Server should be started. Press CTRL+C to quit
python3 -m s3replicator
# Server should be started. Press CTRL+C to quit