-
Notifications
You must be signed in to change notification settings - Fork 184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: more comprehensive docs #305
Changes from 15 commits
b8c3592
0cd132c
25cea9c
982e50f
8043fe1
6819e75
9ffc1b1
79e98c9
30e6a16
c28bfa1
5b2261d
1ddcec5
e2be021
f06e965
1bd034f
2aa4cab
2f6fb78
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
# `apiserver` | ||
|
||
::: llama_deploy.apiserver.deployment | ||
|
||
::: llama_deploy.apiserver.config_parser | ||
options: | ||
members: | ||
- Config | ||
|
||
::: llama_deploy.apiserver.source_managers |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
# `control_plane` | ||
|
||
::: llama_deploy.control_plane | ||
options: | ||
members: - ControlPlaneConfig - ControlPlaneServer - BaseControlPlane |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# `deploy` | ||
|
||
::: llama_deploy.deploy |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
# `message_consumers` | ||
|
||
::: llama_deploy.message_consumers | ||
options: | ||
members: - BaseMessageQueueConsumer - CallableMessageConsumer |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
# `message_publishers` | ||
|
||
::: llama_deploy.message_publishers | ||
options: | ||
members: - MessageQueuePublisherMixin |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
# message_queues | ||
|
||
::: llama_deploy.message_queues.base | ||
options: | ||
members: | ||
- BaseMessageQueue |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# apache_kafka | ||
|
||
::: llama_deploy.message_queues.apache_kafka |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# rabbitmq | ||
|
||
::: llama_deploy.message_queues.rabbitmq |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# redis | ||
|
||
::: llama_deploy.message_queues.redis |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# simple | ||
|
||
::: llama_deploy.message_queues.simple |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
# `messages` | ||
|
||
::: llama_deploy.messages | ||
options: | ||
members: - QueueMessage |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
# `orchestrators` | ||
|
||
::: llama_deploy.orchestrators | ||
options: | ||
members: - BaseOrchestrator - SimpleOrchestrator - SimpleOrchestratorConfig |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# Python SDK | ||
|
||
## `client` | ||
|
||
::: llama_deploy.client |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
# `services` | ||
|
||
::: llama_deploy.services | ||
options: | ||
members: - BaseService - WorkflowService - WorkflowServiceConfig |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,8 @@ | ||
# `types` | ||
|
||
::: llama_deploy.types | ||
options: | ||
members: - ActionTypes - NewTask - ServiceDefinition - SessionDefinition - TaskDefinition - TaskResult | ||
options: | ||
members: | ||
- TaskDefinition | ||
- TaskResult | ||
- TaskStream |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
# Getting Started | ||
|
||
Let's start with deploying a simple workflow on a local instance of Llama Deploy. After installing Llama Deploy, create | ||
a `src` folder add a `workflow.py` file to it containing the following Python code: | ||
masci marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
```python | ||
import asyncio | ||
from llama_index.core.workflow import Workflow, StartEvent, StopEvent, step | ||
|
||
|
||
class EchoWorkflow(Workflow): | ||
"""A dummy workflow with only one step sending back the input given.""" | ||
|
||
@step() | ||
async def run_step(self, ev: StartEvent) -> StopEvent: | ||
message = str(ev.get("message", "")) | ||
return StopEvent(result=f"Message received: {message}") | ||
|
||
|
||
# `echo_workflow` will be imported by Llama Deploy | ||
echo_workflow = EchoWorkflow() | ||
|
||
|
||
async def main(): | ||
print(await echo_workflow.run(message="Hello!")) | ||
|
||
|
||
# Make this script runnable from the shell so we can test the workflow execution | ||
if __name__ == "__main__": | ||
asyncio.run(main()) | ||
``` | ||
|
||
Test the workflow runs locally: | ||
|
||
``` | ||
$ python src/workflow.py | ||
Message received: Hello! | ||
``` | ||
|
||
Time to deploy that workflow! Create a file called `deployment.yml` containing the following YAML code: | ||
|
||
```yaml | ||
name: QuickStart | ||
|
||
control-plane: | ||
port: 8000 | ||
|
||
default-service: echo_workflow | ||
|
||
services: | ||
echo_workflow: | ||
name: Echo Workflow | ||
# We tell Llama Deploy where to look for our workflow | ||
source: | ||
# In this case, we instruct Llama Deploy to look in the local filesystem | ||
type: local | ||
# The path in the local filesystem where to look. This assumes there's an src folder in the | ||
# current working directory containing the file workflow.py we created previously | ||
name: ./src | ||
# This assumes the file workflow.py contains a variable called `echo_workflow` containing our workflow instance | ||
path: workflow:echo_workflow | ||
``` | ||
|
||
The YAML code above defines the deployment that Llama Deploy will create and run as a service. As you can | ||
see, this deployment has a name, some configuration for the control plane and one service to wrap our workflow. The | ||
service will look for a Python variable named `echo_workflow` in a Python module named `workflow` and run the workflow. | ||
|
||
At this point we have all we need to run this deployment. Ideally, we would have the API server already running | ||
somewhere in the cloud, but to get started let's start an instance locally. Run the following python script from a shell: | ||
|
||
``` | ||
$ python -m llama_deploy.apiserver | ||
INFO: Started server process [10842] | ||
INFO: Waiting for application startup. | ||
INFO: Application startup complete. | ||
INFO: Uvicorn running on http://0.0.0.0:4501 (Press CTRL+C to quit) | ||
``` | ||
|
||
From another shell, use `llamactl` to create the deployment: | ||
|
||
``` | ||
$ llamactl deploy deployment.yml | ||
Deployment successful: QuickStart | ||
``` | ||
|
||
Our workflow is now part of the `QuickStart` deployment and ready to serve requests! We can use `llamactl` to interact | ||
with this deployment: | ||
|
||
``` | ||
$ llamactl run --deployment QuickStart --arg message 'Hello from my shell!' | ||
Message received: Hello from my shell! | ||
``` | ||
|
||
### Run the API server with Docker | ||
|
||
Llama Deploy comes with Docker images that can be used to run the API server without effort. In the previous example, | ||
if you have Docker installed, you can replace running the API server locally with `python -m llama_deploy.apiserver` | ||
with: | ||
|
||
``` | ||
$ docker run -p 4501:4501 -v .:/opt/quickstart -w /opt/quickstart llamaindex/llama-deploy | ||
INFO: Started server process [1] | ||
INFO: Waiting for application startup. | ||
INFO: Application startup complete. | ||
INFO: Uvicorn running on http://0.0.0.0:4501 (Press CTRL+C to quit) | ||
``` | ||
|
||
The API server will be available at `http://localhost:4501` on your host, so `llamactl` will work the same as if you | ||
run `python -m llama_deploy.apiserver`. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,120 @@ | ||
# Core Components | ||
|
||
Llama Deploy consists of several core components acting as services in order to provide the environment where | ||
multi-agent applications can run and communicate with each other. This sections details each and every component and | ||
will help you navigate the rest of the documentation. | ||
|
||
## Deployment | ||
|
||
In Llama Deploy each workflow is wrapped in a [_Service_](#service) object, endlessly processing incoming requests in | ||
form of [_Task_](#task) objects. Each service pulls and publishes messages to and from a [_Message Queue_](#message-queue). | ||
An internal component called [_Control Plane_](#control-plane) handles ongoing tasks, manages the internal state, keeps | ||
track of which services are available, and decides which service should handle the next step of a task using another | ||
internal component called [_Orchestrator_](#orchestrator). | ||
|
||
A well defined set of these components is called _Deployment_. | ||
|
||
Deployments can be defined with YAML code, for example: | ||
|
||
```yaml | ||
name: QuickStart | ||
|
||
control-plane: | ||
port: 8000 | ||
|
||
default-service: dummy_workflow | ||
|
||
services: | ||
dummy_workflow: | ||
name: Dummy Workflow | ||
source: | ||
type: local | ||
name: src | ||
path: workflow:echo_workflow | ||
``` | ||
|
||
For more details, see the API reference for the deployment [`Config`](../../api_reference/llama_deploy/apiserver.md#llama_deploy.apiserver.config_parser.Config) object. | ||
|
||
## API Server | ||
|
||
The API Server is a core component of Llama Deploy responsible for serving and managing multiple deployments. It is | ||
responsible for running and managing multiple deployments at the same time, and it exposes a HTTP API that can be used | ||
for administrative purposes as well as for querying the deployed services. You can interact with the administrative | ||
API through [`llamactl`](./50_llamactl.md) or the [Python SDK](./40_python_sdk.md). | ||
|
||
For more details see [the Python API reference](../../api_reference/llama_deploy/apiserver.md), while the administrative | ||
API is documented below. | ||
|
||
!!swagger apiserver.json!! | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh nice, I didn't realize this was possible There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's a bit funky when I run it locally, sometimes it shows sometimes it doesn't, but it might be the local server. I'll keep an eye on its stability. Also we should find a way to remember (best would be automate) to update the openapi def when we change the fastapi routes 😅 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ha definitely -- there's probably some script we can write to dump this, maybe when calling the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. really cool! shows up for me when testing on local |
||
|
||
## Control Plane | ||
|
||
The control plane is responsible for managing the state of the system, including: | ||
|
||
- Registering services. | ||
- Managing sessions and tasks. | ||
- Handling service completion. | ||
- Launching the control plane server. | ||
|
||
## Service | ||
|
||
The general structure of a service is as follows: | ||
|
||
- A service has a name. | ||
- A service has a service definition. | ||
- A service uses a message queue to send/receive messages. | ||
- A service has a processing loop, for continuous processing of messages. | ||
- A service can process a message. | ||
- A service can publish a message to another service. | ||
- A service can be launched in-process. | ||
- A service can be launched as a server. | ||
- A service can be registered to the control plane. | ||
- A service can be registered to the message queue. | ||
|
||
## Message Queue | ||
|
||
In addition to `SimpleMessageQueue`, we provide integrations for various | ||
message queue providers, such as RabbitMQ, Redis, etc. The general usage pattern | ||
for any of these message queues is the same as that for `SimpleMessageQueue`, | ||
however the appropriate extra would need to be installed along with `llama-deploy`. | ||
|
||
For example, for `RabbitMQMessageQueue`, we need to install the "rabbitmq" extra: | ||
|
||
```sh | ||
# using pip install | ||
pip install llama-agents[rabbitmq] | ||
|
||
# using poetry | ||
poetry add llama-agents -E "rabbitmq" | ||
``` | ||
|
||
Using the `RabbitMQMessageQueue` is then done as follows: | ||
|
||
```python | ||
from llama_agents.message_queue.rabbitmq import ( | ||
RabbitMQMessageQueueConfig, | ||
RabbitMQMessageQueue, | ||
) | ||
|
||
message_queue_config = ( | ||
RabbitMQMessageQueueConfig() | ||
) # loads params from environment vars | ||
message_queue = RabbitMQMessageQueue(**message_queue_config) | ||
``` | ||
|
||
|
||
> [!NOTE] | ||
> `RabbitMQMessageQueueConfig` can load its params from environment variables. | ||
|
||
|
||
## Orchestrator | ||
|
||
The general idea for an orchestrator is to manage the flow of messages between services. | ||
|
||
Given some state, and task, figure out the next messages to publish. Then, once | ||
the messages are processed, update the state with the results. | ||
|
||
## Task | ||
|
||
A Task is an object representing a request for an operation sent to a Service and the response that will be sent back. | ||
For the details you can look at the [API reference](../../api_reference/llama_deploy/types.md) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This structure makes sense to me (there's possibly some debate on module_guides vs/ understanding in our docs, but eh)
Probably once this merges, we can greatly simplify the README and point to this :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, once the docs are available on the main website I would trim down the README to the minimum that makes more sense and point users there.