ready for review: Chapter 7: Breaking Free from Cloud-Based Models

souzatharsis · Dec 23, 2024 · 06f8bdf · 06f8bdf
1 parent c16db0d
commit 06f8bdf
Show file tree

Hide file tree

Showing 74 changed files with 3,027 additions and 868 deletions.
diff --git a/README.md b/README.md
@@ -10,21 +10,23 @@ Please [open an issue](https://github.com/souzatharsis/tamingLLMs/issues) with y
 *Publication Date: February 2, 2025*
 ### *A Practical Guide to LLM Pitfalls with Open Source Software*
 
-Abstract: *The current discourse around Large Language Models (LLMs) tends to focus heavily on their capabilities while glossing over fundamental challenges. Conversely, this book takes a critical look at the key limitations and implementation pitfalls that engineers and technical product managers encounter when building LLM-powered applications. Through practical Python examples and proven open source solutions, it provides an introductory yet comprehensive guide for navigating these challenges. The focus is on concrete problems - from handling unstructured output to managing context windows - with reproducible code examples and battle-tested open source tools. By understanding these pitfalls upfront, readers will be better equipped to build products that harness the power of LLMs while sidestepping their inherent limitations.*
+Abstract: *The current discourse around Large Language Models (LLMs) tends to focus heavily on their capabilities while glossing over fundamental challenges. Conversely, this book takes a critical look at the key limitations and implementation pitfalls that engineers and technical leaders encounter when building LLM-powered applications. Through practical Python examples and proven open source solutions, it provides an introductory yet comprehensive guide for navigating these challenges. The focus is on concrete problems with reproducible code examples and battle-tested open source tools. 
+
+By understanding these pitfalls upfront, readers will be better equipped to build products that harness the power of LLMs while sidestepping their inherent limitations.*
 
 | Chapter                                   | Website      | Notebook      | Status               |
 |-------------------------------------------|--------------|---------------|----------------------|
-| Preface                   | [html](https://www.souzatharsis.com/tamingLLMs/markdown/preface.html) | N/A           | *Ready for Review*                   |
-| Chapter 1: Introduction                   | [html](https://www.souzatharsis.com/tamingLLMs/markdown/intro.html) | N/A           | *Ready for Review*                   |
-| Chapter 2: Wrestling with Structured Output| [html](https://www.souzatharsis.com/tamingLLMs/notebooks/structured_output.html) | [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/structured_output.ipynb) | *Ready for Review*     |
-| Chapter 3: The Input Data Challenge      |              |               |                  |
-| Chapter 4: Output Size Limitations       | [html](https://www.souzatharsis.com/tamingLLMs/notebooks/output_size_limit.html) | [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/output_size_limit.ipynb) | *Ready for Review*    |
-| Chapter 5: The Evals Gap                 | [html](https://www.souzatharsis.com/tamingLLMs/notebooks/evals.html) | [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/evals.ipynb) | *Ready for Review*     |
-| Chapter 6: Safety Concerns               | [html](https://www.souzatharsis.com/tamingLLMs/notebooks/safety.html)  |  [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/safety.ipynb) | *Ready for Review*     |
-| Chapter 7: Preference-Based Alignment     | [html](https://www.souzatharsis.com/tamingLLMs/notebooks/alignment.html) | [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/alignment.ipynb) | *Ready for Review*     |
-| Chapter 8: Breaking Free from Cloud Providers |   [html](https://www.souzatharsis.com/tamingLLMs/notebooks/local.html) | [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/local.ipynb) | WIP     |
-| Chapter 9: The Cost Factor                |              |               |                 |
-| Chapter 10: Frontiers                |              |               |                 |
+| Foreword                   | [html](https://www.souzatharsis.com/tamingLLMs/markdown/preface.html) | N/A           | *Ready for Review*                   |
+| Preface                   | [html](https://www.souzatharsis.com/tamingLLMs/markdown/intro.html) | N/A           | *Ready for Review*                   |
+| Chapter 1: Wrestling with Structured Output| [html](https://www.souzatharsis.com/tamingLLMs/notebooks/structured_output.html) | [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/structured_output.ipynb) | *Ready for Review*     |
+| Chapter 2: The Input Data Challenge      |              |               |                  |
+| Chapter 3: Output Size Limitations       | [html](https://www.souzatharsis.com/tamingLLMs/notebooks/output_size_limit.html) | [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/output_size_limit.ipynb) | *Ready for Review*    |
+| Chapter 4: The Evals Gap                 | [html](https://www.souzatharsis.com/tamingLLMs/notebooks/evals.html) | [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/evals.ipynb) | *Ready for Review*     |
+| Chapter 5: Safety Concerns               | [html](https://www.souzatharsis.com/tamingLLMs/notebooks/safety.html)  |  [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/safety.ipynb) | *Ready for Review*     |
+| Chapter 6: Preference-Based Alignment     | [html](https://www.souzatharsis.com/tamingLLMs/notebooks/alignment.html) | [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/alignment.ipynb) | *Ready for Review*     |
+| Chapter 7: Breaking Free from Cloud-Based Models |   [html](https://www.souzatharsis.com/tamingLLMs/notebooks/local.html) | [ipynb](https://github.com/souzatharsis/tamingLLMs/blob/master/tamingllms/notebooks/local.ipynb) | *Ready for Review*     |
+| Chapter 8: The Cost Factor                |              |               |                 |
+| Chapter 9: Frontiers                |              |               |                 |
 | Appendix A: Tools and Resources           |              |               |                |
 
 

diff --git a/tamingllms/_build/.doctrees/environment.pickle b/tamingllms/_build/.doctrees/environment.pickle
diff --git a/tamingllms/_build/.doctrees/markdown/preface.doctree b/tamingllms/_build/.doctrees/markdown/preface.doctree
diff --git a/tamingllms/_build/.doctrees/markdown/toc.doctree b/tamingllms/_build/.doctrees/markdown/toc.doctree
diff --git a/tamingllms/_build/.doctrees/notebooks/alignment.doctree b/tamingllms/_build/.doctrees/notebooks/alignment.doctree
diff --git a/tamingllms/_build/.doctrees/notebooks/evals.doctree b/tamingllms/_build/.doctrees/notebooks/evals.doctree
diff --git a/tamingllms/_build/.doctrees/notebooks/local.doctree b/tamingllms/_build/.doctrees/notebooks/local.doctree
diff --git a/tamingllms/_build/.doctrees/notebooks/output_size_limit.doctree b/tamingllms/_build/.doctrees/notebooks/output_size_limit.doctree
diff --git a/tamingllms/_build/.doctrees/notebooks/safety.doctree b/tamingllms/_build/.doctrees/notebooks/safety.doctree
diff --git a/tamingllms/_build/.doctrees/notebooks/structured_output.doctree b/tamingllms/_build/.doctrees/notebooks/structured_output.doctree
diff --git a/tamingllms/_build/html/_images/downloads.png b/tamingllms/_build/html/_images/downloads.png
diff --git a/tamingllms/_build/html/_images/latency.png b/tamingllms/_build/html/_images/latency.png
diff --git a/tamingllms/_build/html/_images/model_types.svg b/tamingllms/_build/html/_images/model_types.svg
diff --git a/tamingllms/_build/html/_images/p1.png b/tamingllms/_build/html/_images/p1.png
diff --git a/tamingllms/_build/html/_images/p2.png b/tamingllms/_build/html/_images/p2.png
diff --git a/tamingllms/_build/html/_images/perf_.png b/tamingllms/_build/html/_images/perf_.png
diff --git a/tamingllms/_build/html/_images/qwen_perf.png b/tamingllms/_build/html/_images/qwen_perf.png
diff --git a/tamingllms/_build/html/_images/task_number.png b/tamingllms/_build/html/_images/task_number.png
diff --git a/tamingllms/_build/html/_sources/markdown/toc.md b/tamingllms/_build/html/_sources/markdown/toc.md
@@ -14,33 +14,33 @@ Sign-up to receive updates on [new Chapters here](https://tamingllm.substack.com
 # [Taming LLMs](https://www.souzatharsis.com/tamingLLMs)
 ## *A Practical Guide to LLM Pitfalls with Open Source Software*
 
-Abstract: *The current discourse around Large Language Models (LLMs) tends to focus heavily on their capabilities while glossing over fundamental challenges. Conversely, this book takes a critical look at the key limitations and implementation pitfalls that engineers and technical product managers encounter when building LLM-powered applications. Through practical Python examples and proven open source solutions, it provides an introductory yet comprehensive guide for navigating these challenges. The focus is on concrete problems - from handling unstructured output to managing context windows - with reproducible code examples and battle-tested open source tools. By understanding these pitfalls upfront, readers will be better equipped to build products that harness the power of LLMs while sidestepping their inherent limitations.*
+Abstract: *The current discourse around Large Language Models (LLMs) tends to focus heavily on their capabilities while glossing over fundamental challenges. Conversely, this book takes a critical look at the key limitations and implementation pitfalls that engineers and technical leaders encounter when building LLM-powered applications. Through practical Python examples and proven open source solutions, it provides an introductory yet comprehensive guide for navigating these challenges. The focus is on concrete problems with reproducible code examples and battle-tested open source tools. By understanding these pitfalls upfront, readers will be better equipped to build products that harness the power of LLMs while sidestepping their inherent limitations.*
 
-## [Preface](https://www.souzatharsis.com/tamingLLMs/markdown/preface.html)
+## [Foreword](https://www.souzatharsis.com/tamingLLMs/markdown/preface.html)
 
-## [Chapter 1: Introduction](https://www.souzatharsis.com/tamingLLMs/markdown/intro.html)
+## [Preface](https://www.souzatharsis.com/tamingLLMs/markdown/intro.html)
 
-## [Chapter 2: Wrestling with Structured Output](https://www.souzatharsis.com/tamingLLMs/notebooks/structured_output.html)
+## [Chapter 1: Wrestling with Structured Output](https://www.souzatharsis.com/tamingLLMs/notebooks/structured_output.html)
 
-## Chapter 3: Input Data Challenge
+## Chapter 2: Input Data Challenge
 
-## [Chapter 4: Output Size and Length Limitations](https://www.souzatharsis.com/tamingLLMs/notebooks/output_size_limit.html)
+## [Chapter 3: Output Size and Length Limitations](https://www.souzatharsis.com/tamingLLMs/notebooks/output_size_limit.html)
 
-## [Chapter 5: The Evals Gap](https://www.souzatharsis.com/tamingLLMs/notebooks/evals.html)
+## [Chapter 4: The Evals Gap](https://www.souzatharsis.com/tamingLLMs/notebooks/evals.html)
 
-## [Chapter 6: Safety Concerns](https://www.souzatharsis.com/tamingLLMs/notebooks/safety.html)
+## [Chapter 5: Safety Concerns](https://www.souzatharsis.com/tamingLLMs/notebooks/safety.html)
 
-## [Chapter 7: Preference-based Alignment](https://www.souzatharsis.com/tamingLLMs/notebooks/alignment.html)
+## [Chapter 6: Preference-based Alignment](https://www.souzatharsis.com/tamingLLMs/notebooks/alignment.html)
 
-## [Chapter 8: Breaking Free from Cloud Providers](https://www.souzatharsis.com/tamingLLMs/notebooks/local.html)
+## [Chapter 7: Breaking Free from Cloud-Based Models](https://www.souzatharsis.com/tamingLLMs/notebooks/local.html)
 
-## Chapter 9: The Cost Factor
+## Chapter 8: The Cost Factor
 
-## Chapter 10: Frontiers
+## Chapter 9: Frontiers
 
 ## Appendix A: Tools and Resources
 
-## Citation
+
 [![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]
 
 [cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/

diff --git a/tamingllms/_build/html/_sources/notebooks/alignment.ipynb b/tamingllms/_build/html/_sources/notebooks/alignment.ipynb
@@ -414,6 +414,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "(alignment-case-study)=\n",
     "## Case Study: Aligning a Language Model to a Policy\n",
     "\n",
     "In this case study, we will align a language model to a policy. The policy is a set of principles and rules that we want the language model to adhere to. All methodology and code available solves this general problem of policy-based alignment. However, we will describe a specific case study to illustrate our approach.\n",
@@ -427,9 +428,7 @@
     "2. Fine-tuning a base model using Direct Preference Optimization (DPO)\n",
     "3. Evaluating the aligned model against the base model and measuring alignment with Acme Inc.'s educational policies\n",
     "\n",
-    "\n",
-    "### Introduction\n",
-    "#### Experimental Setup\n",
+    "### Experimental Setup\n",
     "\n",
     "We will use the following base model: `HuggingFaceTB/SmolLM2-360M-Instruct` {cite}`smollm2024model`, a compact open source language model that is part of the SmolLM2 family published by HuggingFace.\n",
     "\n",
@@ -448,15 +447,15 @@
     "OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>\n",
     "```\n",
     "\n",
-    "#### Deliverables\n",
+    "### Deliverables\n",
     "\n",
     "As a result, we will have:\n",
     "\n",
     "* `smolK-12`, a fine-tuned model aligned with Acme Inc.'s policy \n",
     "* A DPO-based reusable dataset capturing policy preferences\n",
     "* Evaluation metrics to measure alignment\n",
     "\n",
-    "#### A Note on smolLM2 Models\n",
+    "### A Note on smolLM2 Models\n",
     "\n",
     "Since we have decided to anchor our Case Study on HuggingFace's SmolLM2 models {cite}`smollm2024`, it is worth providing a reason for this choice.\n",
     "\n",

diff --git a/tamingllms/_build/html/_sources/notebooks/evals.ipynb b/tamingllms/_build/html/_sources/notebooks/evals.ipynb
@@ -4,6 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "(evals)=\n",
     "# The Evals Gap\n",
     "```{epigraph}\n",
     "It doesn't matter how beautiful your theory is, <br>\n",