-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Impromve readme documentation (#100)
- Loading branch information
Showing
1 changed file
with
63 additions
and
33 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,25 +2,53 @@ | |
![App workflow](https://github.com/kaypee90/zebrok/actions/workflows/python-app.yml/badge.svg) | ||
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) | ||
|
||
Brokerless task queue for python based on 0Mq | ||
**Brokerless task queue for Python based on ZeroMQ** | ||
|
||
## Key Benefits of Using ZeroMQ | ||
- **Fast:** High-performance messaging library. | ||
- **Lightweight:** Minimal resource usage. | ||
- **Open Source:** Free to use and modify. | ||
- **Low Latency:** Efficient message passing. | ||
- **No Broker Required:** Direct communication between endpoints. | ||
|
||
## Running Zebrok Examples with Docker Compose | ||
1. **Clone the repository:** | ||
```sh | ||
git clone [email protected]:kaypee90/zebrok.git | ||
``` | ||
|
||
2. **Navigate to the Zebrok directory:** | ||
```sh | ||
cd zebrok | ||
``` | ||
|
||
3. **Start the worker and publisher containers:** | ||
```sh | ||
docker-compose up | ||
``` | ||
|
||
4. **Start the workers:** | ||
* Access the shell for the worker container: | ||
```sh | ||
docker exec -it <worker-container-id> /bin/sh | ||
``` | ||
* Run the start script: | ||
```sh | ||
python examples/start.py | ||
``` | ||
|
||
5. **Queue jobs from the publisher:** | ||
* Access the shell for the publisher container from a different terminal: | ||
```sh | ||
docker exec -it <publisher-container-id> /bin/sh | ||
``` | ||
* Queue jobs: | ||
```sh | ||
python examples/client.py | ||
``` | ||
|
||
6. **Expected Output:** | ||
|
||
- This benefits of libraty of 0Mq | ||
- Fast | ||
- Lightweight | ||
- Open source | ||
- Low Latency | ||
- No broker required | ||
|
||
|
||
### Running Zebrok examples with docker compose | ||
* First clone the repository, use the command `git clone [email protected]:kaypee90/zebrok.git` | ||
* Change directory into the zebrok directory and run `docker compose up` command to start the worker and publisher containers. | ||
* Access the shell for the **worker** container and `run` the command `python examples/start.py` to start the workers. | ||
* From a different terminal window, access the shell for the running **publisher** container and run the command `python examples/client.py` to queue 2 jobs to be processed. | ||
* Once these commands are executed you should see 2 tasks processed successfully in the publisher terminal. | ||
|
||
|
||
**Sample output:** | ||
``` | ||
** 2 ZEBROK TASKS DISCOVERED! | ||
===================================================== | ||
|
@@ -54,18 +82,18 @@ DONE!!! | |
DONE!!! | ||
``` | ||
### - Trying out zebrok | ||
======================== | ||
## Trying out zebrok | ||
* To `install` zebrok use the command `pip install git+https://github.com/kaypee90/zebrok.git#egg=zebrok` | ||
|
||
* Configuring Environment Variables: | ||
- WORKER_HOST         # The IP address for running workers | ||
- WORKER_PORT               # The port number workers should listen on | ||
1. **Install Zebrok** | ||
```sh | ||
pip install git+https://github.com/kaypee90/zebrok.git#egg=zebrok | ||
``` | ||
|
||
-- `If not set defaults to localhost:5690` | ||
2. **Configuring Environment Variables:** | ||
- `WORKER_HOST: The IP address for running workers (default: localhost) | ||
- `WORKER_PORT: The port number workers should listen on (default: 5690) | ||
|
||
* Creating A Task `[tasks.py]` | ||
3. **Creating a Task `(tasks.py)`** | ||
``` | ||
from zebrok import app | ||
|
@@ -74,7 +102,7 @@ def long_running_task(param): | |
do_some_time_consuming_task(param) | ||
``` | ||
|
||
* Configuring a worker and registering the task `[examples/start.py]` | ||
4. **Configuring a Worker and Registering the Task (examples/start.py):** | ||
- NB: `A task can also be discovered automatically if placed in a tasks.py file in the root folder of the project.` | ||
`- You can also set number of slave worker threads to be running by passing number_of_slaves argument` | ||
``` | ||
|
@@ -87,10 +115,13 @@ worker.register_task(long_running_task) | |
worker.start() | ||
``` | ||
|
||
* Starting the Zebrok Worker to listen for tasks -`python examples/start.py` | ||
5. **Starting the Zebrok Worker:** | ||
where **start.py** is the file in which you configured the worker | ||
```sh | ||
python examples/start.py | ||
``` | ||
|
||
* Executing a task `[examples/client.py]` | ||
6. **Executing a task (`examples/client.py`)** | ||
``` | ||
from tasks import long_running_task | ||
|
||
|
@@ -101,6 +132,5 @@ long_running_task.run(param="dowork") | |
### Using a container orchestration technology (like Kubernetes): | ||
- `number_of_slaves` must always be set to 0, then you can spin a number of replicas for the workers. | ||
- `WORKER_HOST` Environment variable for a worker must always be `*` | ||
|
||
- Set `number_of_slaves` to 0, then spin up multiple replicas for the workers. | ||
- The `WORKER_HOST` environment variable for a worker must be set `*` |