Skip to content

flipkart-incubator/varadhi

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Varadhi : RESTBus - A message bus implementation with a REST interface

build License

Introduction

Varadhi is the open source version of a similar FK internal project, a message bus implementation with a REST interface. Varadhi supports both P2P (point-to-point) and Publish-Subscribe messaging models. The idea behind RESTBus is to take your HTTP API stack and automatically transform them into service bus driven, queue/pub-sub, message oriented endpoints. The communication between Varadhi and endpoints will be on HTTP.

Varadhi is in active development and has been in production since last 10 years inside Flipkart as the backbone of async REST communication between various microservices sending billions of messages daily.

With Varadhi we hope others would be able to benefit from our learnings of a decade.

Full Documentation

See the Wiki for concepts, use cases, architecture details, API spec and other detailed information.

Try locally

Give it a go locally on your machine. You will require java 21, docker & python. Follow this guide: Wiki/Try Locally

Build

./gradlew build test

Integration Tests

./gradlew copyDependencies copyConfigForE2E -x test

docker build . --file setup/docker/Dockerfile --tag varadhi.docker.registry/varadhi:latest --build-arg

docker compose --profile test -f setup/docker/compose.yml up -d --wait --wait-timeout 180

./gradlew testE2E

Dependencies

To provide the required functionality, Varadhi takes dependency on various tech stack.

Run the following to start the below dependencies. docker compose --profile dev -f setup/docker/compose.yml up -d --wait --wait-timeout 180

OR

Message Broker

Varadhi needs a message broker for message persistence and delivery. Apache Pulsar is used as underlying message broker. For the development environment users can use containerised Pulsar in a standalone mode. Details can be found here.

Metadata Store

For storing metadata about various Varadhi entities, a datastore is needed. Zookeeper is used as global datastore to persist json formatted entities. For the development environment, containerised Zookeeper can be used in a standalone mode. Details can be found here.

Varadhi Server

Finally, to run the Varadhi server, provide the custom zk & pulsar endpoints at server/src/main/resources/config.overrides and then do ./gradlew run

If you are using the dev profile in our docker compose to start the zk and pulsar, then simply do ./gradlew run. No config overrides are required.

k8s Deployment

cd setup/helm/varadhi

helm install varadhi-server . -f values/common.values.yaml -f values/local.server.values.yaml

helm install varadhi-controller . -f values/common.values.yaml -f values/local.controller.values.yaml

Modules

  • entities: It has all the entities used by the spi module and varadhi apis.
  • spi: It has all the interfaces related to persistence and messaging.
  • pulsar: It contains messaging-spi implementation using Apache Pulsar.
  • core: It contains all the core logic of Varadhi and is relied upon by various other sub-modules.
  • server: It is the entry point of Varadhi. It houses all server related logic and binds together all other modules.

Releases

[ TBD ]

Roadmap

Status: Initial Review Completed.

See the Wiki page Roadmap for details.

Communications

[ TBD ]

Want to Contribute ?

Refer to Contributing.

You can also reachout to [email protected] or [email protected].

Bugs and Feedback

[ TBD ]

Blogs

LICENSE

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages