update

tamingllms/_build/html/_sources/markdown/intro.md

@@ -1,10 +1,18 @@
 # Introduction
 
+```{epigraph}
+Too often we enjoy the comfort of opinion without the discomfort of thought.
+
+-- John F. Kennedy
+```
+```{contents}
+```
+
+## Core Challenges We'll Address
 In recent years, Large Language Models (LLMs) have emerged as a transformative force in technology, promising to revolutionize how we build products and interact with computers. From ChatGPT to GitHub Copilot, Claude Artifacts, cursor.com, replit, and others, these systems have captured the public imagination and sparked a gold rush of AI-powered applications. However, beneath the surface of this technological revolution lies a complex landscape of challenges that practitioners must navigate. 
 
 As we'll explore in this book, the engineering effort required to manage these challenges - from handling non-deterministic outputs to preventing hallucinations - cannot be overstated. While the potential of LLM technology remains compelling, understanding and addressing the hidden costs and complexities of building reliable LLM-powered systems will enable us to fully harness their transformative impact.
 
-## Core Challenges We'll Address
 While the capabilities of LLMs are indeed remarkable, the prevailing narrative often glosses over fundamental problems that engineers, product managers, and organizations face when building real-world applications. This book aims to bridge that gap, offering a practical, clear-eyed examination of the pitfalls and challenges in working with LLMs.
 
 Throughout this book, we'll tackle the following (non-exhaustive) list of critical challenges:
@@ -73,9 +81,15 @@ pip install -r requirements.txt
 ```
 
 ### 2. API Keys Configuration
-Set required API keys:
-```bash
-export OPENAI_API_KEY=your-openai-key
+1. Create a `.env` file in the root directory of the project.
+2. Add your API keys and other sensitive information to the `.env` file. For example:
+
+   ```
+   OPENAI_API_KEY=your_openai_api_key_here
+   ```
+
+```{warning}
+Never share your `.env` file or commit it to version control. It contains sensitive information that should be kept private.
 ```
 
 ### 3. Code Repository
@@ -85,11 +99,11 @@ git clone https://github.com/souzatharsis/tamingllms.git
 cd tamingllms
 ```
 
+
+
 ### Troubleshooting Common Issues
 - If you encounter API rate limits, consider using smaller examples or implementing retry logic
 - For package conflicts, try creating a fresh virtual environment or use a package manager like `poetry`
 - Check the book's repository issues page for known problems and solutions
 
-Now that your environment is set up, let's begin our exploration of LLM challenges.
-
-
+Now that your environment is set up, let's begin our exploration of LLM challenges.

tamingllms/_build/html/_sources/notebooks/nondeterminism.ipynb

@@ -6,11 +6,18 @@
    "source": [
     "\n",
     "# Non-determinism & Evals\n",
+    "```{epigraph}\n",
+    "Evals are surprisingly often all you need.\n",
     "\n",
-    "One of the most fundamental challenges when building products with Large Language Models (LLMs) is their non-deterministic nature. Unlike traditional software systems where the same input reliably produces the same output, LLMs can generate different responses each time they're queried - even with identical prompts. This characteristic is both a strength and a significant engineering challenge.\n",
+    "-- Greg Brockman, OpenAI's President\n",
+    "```\n",
+    "```{contents}\n",
+    "```\n",
     "\n",
     "## Understanding the Challenge\n",
     "\n",
+    "One of the most fundamental challenges when building products with Large Language Models (LLMs) is their non-deterministic nature. Unlike traditional software systems where the same input reliably produces the same output, LLMs can generate different responses each time they're queried - even with identical prompts. This characteristic is both a strength and a significant engineering challenge.\n",
+    "\n",
     "### What is Non-determinism in LLMs?\n",
     "\n",
     "When you ask ChatGPT or any other LLM the same question multiple times, you'll likely get different responses. This isn't a bug - it's a fundamental feature of how these models work. The \"temperature\" parameter, which controls the randomness of outputs, allows models to be creative and generate diverse responses. However, this same feature makes it incredibly difficult to build reliable, testable systems.\n",

tamingllms/_build/html/_sources/notebooks/output_size_limit.ipynb

@@ -5,15 +5,21 @@
    "metadata": {},
    "source": [
     "# Output Size Limitations\n",
+    "```{epigraph}\n",
+    "Only those who will risk going too far can possibly find out how far one can go.\n",
     "\n",
+    "-- T.S. Eliot\n",
+    "```\n",
+    "```{contents}\n",
+    "```\n",
     "## What are Token Limits?\n",
     "\n",
     "Tokens are the basic units that LLMs process text with. A token can be as short as a single character or as long as a complete word. In English, a general rule of thumb is that 1 token ≈ 4 characters or ¾ of a word.\n",
     "\n",
-    "The `max_output_tokens` is parameter often available in modern LLMs that determines the maximum length of text that an LLM can generate in a single response. Contrary to what one might expect, the model does not \"summarizes the answer\" such that it does not surpass `max_output_tokens` limit. Instead, it will stop once it reaches this limit, even mid-sentence, i.e. the response may be truncated.\n",
-    "\n",
-    "**Table 1: Token Cost and Length Limitation Comparison Across Key Models**\n",
+    "The `max_output_tokens` is parameter often available in modern LLMs that determines the maximum length of text that an LLM can generate in a single response. {numref}`token-cost-table` shows the `max_output_tokens` for several key models, which typically range between 4096 and 16384 tokens. Contrary to what one might expect, the model does not \"summarizes the answer\" such that it does not surpass `max_output_tokens` limit. Instead, it will stop once it reaches this limit, even mid-sentence, i.e. the response may be truncated.\n",
     "\n",
+    "```{table} Token Cost and Length Limitation Comparison Across Key Models\n",
+    ":name: token-cost-table\n",
     "| Model                        | max_output_tokens | max_input_tokens | input_cost_per_token | output_cost_per_token |\n",
     "|------------------------------|-------------------|------------------|----------------------|-----------------------|\n",
     "| meta.llama3-2-11b-instruct-v1:0 | 4096              | 128000           | 3.5e-7               | 3.5e-7                |\n",
@@ -23,6 +29,7 @@
     "| gpt-4o-mini                  | 16384             | 128000           | 1.5e-7               | 6e-7                  |\n",
     "| gemini/gemini-1.5-flash-002  | 8192              | 1048576          | 7.5e-8               | 3e-7                  |\n",
     "| gemini/gemini-1.5-pro-002    | 8192              | 2097152          | 3.5e-6               | 1.05e-5               |\n",
+    "```\n",
     "\n",
     "## Problem Statement\n",
     "\n",

tamingllms/_build/html/_sources/notebooks/structured_output.ipynb

@@ -5,6 +5,13 @@
    "metadata": {},
    "source": [
     "# Wrestling with Structured Output\n",
+    "```{epigraph}\n",
+    "In limits, there is freedom. Creativity thrives within structure.\n",
+    "\n",
+    "-- Julia B. Cameron\n",
+    "```\n",
+    "```{contents}\n",
+    "```\n",
     "\n",
     "## The Structured Output Challenges\n",
     "\n",