update toc

pyproject.toml

@@ -37,6 +37,7 @@ datasets = "^3.1.0"
 text-generation = "^0.7.0"
 ollama = "^0.4.3"
 langsmith = "0.2.0"
+sphinxcontrib-mermaid = "^1.0.0"
 
 
 [build-system]

tamingllms/_build/html/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 0a894bf701cc67875c35ad0e77bfe357
+config: 0674df45a82890a4968a11e155ae506b
 tags: 645f666f9bcd5a90fca523b33c5a78b7

tamingllms/_build/html/_sources/markdown/toc.md

@@ -4,14 +4,15 @@ author: "Tharsis T. P. Souza"
 date: "2024-11-22"
 ---
 
+Sign-up to receive updates on [new Chapters here](https://tamingllm.substack.com/).
+
 # Taming LLMs
 ## *A Practical Guide to LLM Pitfalls with Open Source Software*
 
-
 Abstract: *The current discourse around Large Language Models (LLMs) tends to focus heavily on their capabilities while glossing over fundamental challenges. Conversely, this book takes a critical look at the key limitations and implementation pitfalls that engineers and technical product managers encounter when building LLM-powered applications. Through practical Python examples and proven open source solutions, it provides an introductory yet comprehensive guide for navigating these challenges. The focus is on concrete problems - from handling unstructured output to managing context windows - with reproducible code examples and battle-tested open source tools. By understanding these pitfalls upfront, readers will be better equipped to build products that harness the power of LLMs while sidestepping their inherent limitations.*
 
 
-## Chapter 1: Introduction
+## [Chapter 1: Introduction](https://www.souzatharsis.com/tamingLLMs/markdown/intro.html)
 - 1.1 Core Challenges We'll Address
 - 1.2 A Practical Approach
 - 1.3 A Note on Perspective
@@ -25,7 +26,7 @@ Abstract: *The current discourse around Large Language Models (LLMs) tends to fo
   - 1.7.4 Troubleshooting Common Issues
 - 1.8 About the Author(s)
 
-## Chapter 2: Wrestling with Structured Output
+## [Chapter 2: Wrestling with Structured Output](https://www.souzatharsis.com/tamingLLMs/notebooks/structured_output.html)
 - 2.1 The Structured Output Challenge
 - 2.2 Problem Statement
 - 2.3 Solutions
@@ -55,7 +56,7 @@ Abstract: *The current discourse around Large Language Models (LLMs) tends to fo
 - 3.5 Implementation Patterns
 - 3.6 Testing Long-form Content
 
-## Chapter 4: Output Size and Length Limitations
+## [Chapter 4: Output Size and Length Limitations](https://www.souzatharsis.com/tamingLLMs/notebooks/output_size_limit.html)
 - 4.1 What are Token Limits?
 - 4.2 Problem Statement
 - 4.3 Content Chunking with Contextual Linking
@@ -71,7 +72,7 @@ Abstract: *The current discourse around Large Language Models (LLMs) tends to fo
 - 4.7 Conclusion
 - 4.8 References
 
-## Chapter 5: The Evals Gap
+## [Chapter 5: The Evals Gap](https://www.souzatharsis.com/tamingLLMs/notebooks/evals.html)
 - 5.1 Non-Deterministic Machines
   - 5.1.1 Temperature and Sampling
   - 5.1.2 The Temperature Spectrum

tamingllms/_build/html/genindex.html

@@ -29,6 +29,14 @@
         <script src="_static/design-tabs.js"></script>
         <script>const THEBE_JS_URL = "https://unpkg.com/[email protected]/lib/index.js"; const thebe_selector = ".thebe,.cell"; const thebe_selector_input = "pre"; const thebe_selector_output = ".output, .cell_output"</script>
         <script async="async" src="_static/sphinx-thebe.js"></script>
+        <script type="module" src="https://cdn.jsdelivr.net/npm/[email protected]/dist/mermaid.esm.min.mjs"></script>
+        <script type="module" src="https://cdn.jsdelivr.net/npm/@mermaid-js/[email protected]/dist/mermaid-layout-elk.esm.min.mjs"></script>
+        <script type="module">import mermaid from "https://cdn.jsdelivr.net/npm/[email protected]/dist/mermaid.esm.min.mjs";import elkLayouts from "https://cdn.jsdelivr.net/npm/@mermaid-js/[email protected]/dist/mermaid-layout-elk.esm.min.mjs";mermaid.registerLayoutLoaders(elkLayouts);mermaid.initialize({startOnLoad:false});</script>
+        <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/d3.min.js"></script>
+        <script type="module">
+import mermaid from "https://cdn.jsdelivr.net/npm/[email protected]/dist/mermaid.esm.min.mjs";
+window.addEventListener("load", () => mermaid.run());
+</script>
 
 
       <!-- bundled in js (rollup iife) -->

tamingllms/_build/html/markdown/toc.html

@@ -170,14 +170,15 @@
 <hr>
           <div class="content" role="main" v-pre>
 
-  <section class="tex2jax_ignore mathjax_ignore" id="taming-llms">
+  <p>Sign-up to receive updates on <a class="reference external" href="https://tamingllm.substack.com/">new Chapters here</a>.</p>
+<section class="tex2jax_ignore mathjax_ignore" id="taming-llms">
 <h1>Taming LLMs<a class="headerlink" href="#taming-llms" title="Permalink to this heading">¶</a></h1>
 <section id="a-practical-guide-to-llm-pitfalls-with-open-source-software">
 <h2><em>A Practical Guide to LLM Pitfalls with Open Source Software</em><a class="headerlink" href="#a-practical-guide-to-llm-pitfalls-with-open-source-software" title="Permalink to this heading">¶</a></h2>
 <p>Abstract: <em>The current discourse around Large Language Models (LLMs) tends to focus heavily on their capabilities while glossing over fundamental challenges. Conversely, this book takes a critical look at the key limitations and implementation pitfalls that engineers and technical product managers encounter when building LLM-powered applications. Through practical Python examples and proven open source solutions, it provides an introductory yet comprehensive guide for navigating these challenges. The focus is on concrete problems - from handling unstructured output to managing context windows - with reproducible code examples and battle-tested open source tools. By understanding these pitfalls upfront, readers will be better equipped to build products that harness the power of LLMs while sidestepping their inherent limitations.</em></p>
 </section>
 <section id="chapter-1-introduction">
-<h2>Chapter 1: Introduction<a class="headerlink" href="#chapter-1-introduction" title="Permalink to this heading">¶</a></h2>
+<h2><a class="reference external" href="https://www.souzatharsis.com/tamingLLMs/markdown/intro.html">Chapter 1: Introduction</a><a class="headerlink" href="#chapter-1-introduction" title="Permalink to this heading">¶</a></h2>
 <ul class="simple">
 <li><p>1.1 Core Challenges We’ll Address</p></li>
 <li><p>1.2 A Practical Approach</p></li>
@@ -197,7 +198,7 @@ <h2>Chapter 1: Introduction<a class="headerlink" href="#chapter-1-introduction"
 </ul>
 </section>
 <section id="chapter-2-wrestling-with-structured-output">
-<h2>Chapter 2: Wrestling with Structured Output<a class="headerlink" href="#chapter-2-wrestling-with-structured-output" title="Permalink to this heading">¶</a></h2>
+<h2><a class="reference external" href="https://www.souzatharsis.com/tamingLLMs/notebooks/structured_output.html">Chapter 2: Wrestling with Structured Output</a><a class="headerlink" href="#chapter-2-wrestling-with-structured-output" title="Permalink to this heading">¶</a></h2>
 <ul class="simple">
 <li><p>2.1 The Structured Output Challenge</p></li>
 <li><p>2.2 Problem Statement</p></li>
@@ -248,7 +249,7 @@ <h2>Chapter 3: Input Size and Length Limitations<a class="headerlink" href="#cha
 </ul>
 </section>
 <section id="chapter-4-output-size-and-length-limitations">
-<h2>Chapter 4: Output Size and Length Limitations<a class="headerlink" href="#chapter-4-output-size-and-length-limitations" title="Permalink to this heading">¶</a></h2>
+<h2><a class="reference external" href="https://www.souzatharsis.com/tamingLLMs/notebooks/output_size_limit.html">Chapter 4: Output Size and Length Limitations</a><a class="headerlink" href="#chapter-4-output-size-and-length-limitations" title="Permalink to this heading">¶</a></h2>
 <ul class="simple">
 <li><p>4.1 What are Token Limits?</p></li>
 <li><p>4.2 Problem Statement</p></li>
@@ -270,7 +271,7 @@ <h2>Chapter 4: Output Size and Length Limitations<a class="headerlink" href="#ch
 </ul>
 </section>
 <section id="chapter-5-the-evals-gap">
-<h2>Chapter 5: The Evals Gap<a class="headerlink" href="#chapter-5-the-evals-gap" title="Permalink to this heading">¶</a></h2>
+<h2><a class="reference external" href="https://www.souzatharsis.com/tamingLLMs/notebooks/evals.html">Chapter 5: The Evals Gap</a><a class="headerlink" href="#chapter-5-the-evals-gap" title="Permalink to this heading">¶</a></h2>
 <ul class="simple">
 <li><p>5.1 Non-Deterministic Machines</p>
 <ul>

tamingllms/_build/html/search.html

@@ -29,6 +29,14 @@
         <script src="_static/design-tabs.js"></script>
         <script>const THEBE_JS_URL = "https://unpkg.com/[email protected]/lib/index.js"; const thebe_selector = ".thebe,.cell"; const thebe_selector_input = "pre"; const thebe_selector_output = ".output, .cell_output"</script>
         <script async="async" src="_static/sphinx-thebe.js"></script>
+        <script type="module" src="https://cdn.jsdelivr.net/npm/[email protected]/dist/mermaid.esm.min.mjs"></script>
+        <script type="module" src="https://cdn.jsdelivr.net/npm/@mermaid-js/[email protected]/dist/mermaid-layout-elk.esm.min.mjs"></script>
+        <script type="module">import mermaid from "https://cdn.jsdelivr.net/npm/[email protected]/dist/mermaid.esm.min.mjs";import elkLayouts from "https://cdn.jsdelivr.net/npm/@mermaid-js/[email protected]/dist/mermaid-layout-elk.esm.min.mjs";mermaid.registerLayoutLoaders(elkLayouts);mermaid.initialize({startOnLoad:false});</script>
+        <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/d3.min.js"></script>
+        <script type="module">
+import mermaid from "https://cdn.jsdelivr.net/npm/[email protected]/dist/mermaid.esm.min.mjs";
+window.addEventListener("load", () => mermaid.run());
+</script>
 
 
       <!-- bundled in js (rollup iife) -->

tamingllms/_build/jupyter_execute/markdown/intro.ipynb

@@ -2,7 +2,7 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "9bda1934",
+   "id": "6fed7978",
    "metadata": {},
    "source": [
     "(intro)=\n",

tamingllms/_config.yml

@@ -37,8 +37,9 @@ bibtex_bibfiles:
 
 only_build_toc_files: true
 
-
 sphinx:
+  extra_extensions:
+    - sphinxcontrib.mermaid
   config:
     bibtex_reference_style: author_year
     html_theme: 'press' #insipid

tamingllms/_toc.yml

@@ -10,6 +10,7 @@ chapters:
 - file: notebooks/output_size_limit.ipynb
 - file: notebooks/structured_output.ipynb
 - file: notebooks/evals.ipynb
+#- file: notebooks/alignment.ipynb
 #- file: markdown
 #- file: notebooks
 #- file: markdown-notebooks