Skip to content

Commit

Permalink
Adds image and SDLC section in overview docs.
Browse files Browse the repository at this point in the history
  • Loading branch information
skrawcz committed Dec 27, 2024
1 parent 8e820a3 commit 84c5b0d
Show file tree
Hide file tree
Showing 4 changed files with 41 additions and 0 deletions.
Binary file added docs/_static/burr_sdlc.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 27 additions & 0 deletions docs/concepts/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,30 @@ This is a quick overview of Burr's design -- the concepts are explored in more d
Note that we did not mention LLMs above at all! That's good -- you want your LLM frameworks to be ever-so-slightly decoupled from them for the best experience (all of AI is just software, plain and simple, after all...).

And that's the basics! Let's dive into the details.

================================
SDLC with LLM based applications
================================
If you're building an LLM-based application, you'll want to follow a slightly different SDLC than you would for a traditional software project. Here's a rough outline of what that might look like:

.. image:: ../_static/burr_sdlc.png
:alt: SDLC with LLMs
:align: center

The two cycles that exist are:

1. App Dev Loop.
2. Test Driven Development Loop.

and you will use one to feed into the other, etc.

Walking through the diagram the SDLC looks like this:

1. Write code with Burr.
2. Use Burr's integrated observability, and trace all parts of your application.
3. With the data collected, you can: (1) annotate what was captured and export it, or (2) create a pytest fixture with it.
4. Create a data set from the annotated data or by running tests.
5. Evaluate the data set.
6. Analyze the results.
7. Either adjust code or prompts, or ship the code.
8. Iterate using one of the loops...
14 changes: 14 additions & 0 deletions examples/pytest/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
# A SDLC with Burr and pytest
Here we show a quick example of a software development lifecycle (SDLC) with Burr and pytest.

![Burr and pytest](burr_sdlc.png)

While we don't cover everything in the diagram, in this example we specifically show how to do most of the TDD loop:

1. Create a test case.
2. Run the test case.
3. Create a dataset.
4. Show how you might construct evaluation logic to evaluate the output of your agent / augmented LLM / application.

using Burr and pytest.

# Using pytest to evaluate your agent / augmented LLM / application

An agent / augmented LLM is a combination of LLM calls and logic. But how do we know if it's working? Well we can test & evaluate it.
Expand Down
Binary file added examples/pytest/burr_sdlc.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 84c5b0d

Please sign in to comment.