Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create real world examples #21

Open
gabfssilva opened this issue Mar 23, 2023 · 0 comments
Open

Create real world examples #21

gabfssilva opened this issue Mar 23, 2023 · 0 comments
Labels
documentation Improvements or additions to documentation feature good first issue Good for newcomers help wanted Extra attention is needed wip

Comments

@gabfssilva
Copy link
Collaborator

gabfssilva commented Mar 23, 2023

It's quite difficult to really explain the benefits of a reactive library like River. One way having great real-world examples, showing how this library could be useful in different situations.

Some scenarios as inspiration:

  • A pipeline that reads data from a CSV file in S3, split the file stream into single CSV lines, and then writes each line to a PostgreSQL database using the JDBC connector:
+-------------+   +-------+   +------+   +---------------+
| S3 CSV file |-->| Split |-->| JDBC |-->| Postgres Sink |
+-------------+   +-------+   +------+   +---------------+
  • A pipeline that listens to RabbitMQ queue for incoming data and then writes the results to a MySQL using the R2DBC connector:
+---------------+   +--------+   +-----------+
| RabbitMQ Queue|-->| R2DBC  |-->| MySQL Sink|
+---------------+   +--------+   +-----------+
  • A pipeline that listens to Debezium changes, sends the data into a S3 bucket as a CSV file for audit purposes, chunks the items into 100 items within a 1 sec time window and then sends the chunks as individual items in Azure Queue Storage:
+----------+   +-----------------+   +--------------------+
| Debezium |-->| Chunker (100/1s)|-->| Azure Queue Storage|
+----------+   +-----------------+   +--------------------+ 
        |     
        |
        |            +------------+   +------+   
        +----------> | CSV Parser |-->|  S3  |
                     +------------+   +------+ 
  • A pipeline that listens to an SQS queue for incoming data, maps the items to json objects, and then stream the objects into a HTTP endpoint using application/stream+json content-type:
+--------------+   +-------------+   +---------------+
| SQS Queue    |-->| JSON Mapper |-->| HTTP Endpoint |
+--------------+   +-------------+   +---------------+
@gabfssilva gabfssilva changed the title Create real world examples. Create real world examples Mar 23, 2023
@gabfssilva gabfssilva added documentation Improvements or additions to documentation help wanted Extra attention is needed good first issue Good for newcomers labels Mar 27, 2023
@gabfssilva gabfssilva added the wip label Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation feature good first issue Good for newcomers help wanted Extra attention is needed wip
Projects
None yet
Development

No branches or pull requests

1 participant