build html

souzatharsis · Nov 23, 2024 · cab22b7 · cab22b7
1 parent 5efd914
commit cab22b7
Show file tree

Hide file tree

Showing 22 changed files with 137 additions and 119 deletions.
diff --git a/tamingllms/_build/.doctrees/environment.pickle b/tamingllms/_build/.doctrees/environment.pickle
diff --git a/tamingllms/_build/.doctrees/markdown/toc.doctree b/tamingllms/_build/.doctrees/markdown/toc.doctree
diff --git a/tamingllms/_build/.doctrees/notebooks/nondeterminism.doctree b/tamingllms/_build/.doctrees/notebooks/nondeterminism.doctree
diff --git a/tamingllms/_build/html/.buildinfo b/tamingllms/_build/html/.buildinfo
@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: d5b2da7e1983f3da43c754f98d2e7fad
+config: a685905c3bf9c83b1dd9f8ab3742b2b8
 tags: 645f666f9bcd5a90fca523b33c5a78b7
diff --git a/tamingllms/_build/html/_sources/notebooks/nondeterminism.ipynb b/tamingllms/_build/html/_sources/notebooks/nondeterminism.ipynb
@@ -5,7 +5,7 @@
    "metadata": {},
    "source": [
     "\n",
-    "# Non-determinism\n",
+    "# Non-determinism & Evals\n",
     "\n",
     "One of the most fundamental challenges when building products with Large Language Models (LLMs) is their non-deterministic nature. Unlike traditional software systems where the same input reliably produces the same output, LLMs can generate different responses each time they're queried - even with identical prompts. This characteristic is both a strength and a significant engineering challenge.\n",
     "\n",
@@ -132,10 +132,17 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Evaluating Non-deterministic \n",
+    "\n",
+    "Evaluating non-deterministic LLM systems can be challenging due to the inherent variability in their responses. Here are some strategies to effectively evaluate these systems.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
    "source": []
   }
  ],

diff --git a/tamingllms/_build/html/genindex.html b/tamingllms/_build/html/genindex.html
@@ -99,7 +99,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="notebooks/nondeterminism.html" class="reference internal ">Non-determinism</a>
+              <a href="notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
 
 
 
@@ -108,7 +108,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
+              <a href="notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
 
 
 
@@ -117,7 +117,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
+              <a href="notebooks/nondeterminism.html" class="reference internal ">Non-determinism & Evals</a>
 
 
 
@@ -159,7 +159,7 @@ <h1 id="index">Index</h1>
           <div class="page-nav">
             <div class="inner"><ul class="page-nav">
 </ul><div class="footer" role="contentinfo">
-      &#169; Copyright 2024.
+      &#169; Copyright Tharsis T. P. Souza, 2024.
     <br>
     Created using <a href="http://sphinx-doc.org/">Sphinx</a> 6.2.1 with <a href="https://github.com/schettino72/sphinx_press_theme">Press Theme</a> 0.9.1.
 </div>

diff --git a/tamingllms/_build/html/markdown/intro.html b/tamingllms/_build/html/markdown/intro.html
@@ -38,7 +38,7 @@
 
   <link rel="index" title="Index" href="../genindex.html" />
   <link rel="search" title="Search" href="../search.html" />
-  <link rel="next" title="Non-determinism" href="../notebooks/nondeterminism.html" />
+  <link rel="next" title="Output Size Limitations" href="../notebooks/output_size_limit.html" />
   <link rel="prev" title="Taming Large Language Models" href="toc.html" /> 
   </head>
 
@@ -117,7 +117,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism</a>
+              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
 
 
 
@@ -126,7 +126,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
+              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
 
 
 
@@ -135,7 +135,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
+              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism & Evals</a>
 
 
 
@@ -164,8 +164,8 @@
        title="previous chapter">← Taming Large Language Models</a>
   </li>
   <li class="next">
-    <a href="../notebooks/nondeterminism.html"
-       title="next chapter">Non-determinism →</a>
+    <a href="../notebooks/output_size_limit.html"
+       title="next chapter">Output Size Limitations →</a>
   </li>
 </ul>
 
@@ -298,11 +298,11 @@ <h3>Troubleshooting Common Issues<a class="headerlink" href="#troubleshooting-co
        title="previous chapter">← Taming Large Language Models</a>
   </li>
   <li class="next">
-    <a href="../notebooks/nondeterminism.html"
-       title="next chapter">Non-determinism →</a>
+    <a href="../notebooks/output_size_limit.html"
+       title="next chapter">Output Size Limitations →</a>
   </li>
 </ul><div class="footer" role="contentinfo">
-      &#169; Copyright 2024.
+      &#169; Copyright Tharsis T. P. Souza, 2024.
     <br>
     Created using <a href="http://sphinx-doc.org/">Sphinx</a> 6.2.1 with <a href="https://github.com/schettino72/sphinx_press_theme">Press Theme</a> 0.9.1.
 </div>

diff --git a/tamingllms/_build/html/markdown/markdown-notebooks.html b/tamingllms/_build/html/markdown/markdown-notebooks.html
@@ -99,7 +99,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism</a>
+              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
 
 
 
@@ -108,7 +108,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
+              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
 
 
 
@@ -117,7 +117,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
+              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism & Evals</a>
 
 
 
@@ -213,7 +213,7 @@ <h2>Quickly add YAML metadata for MyST Notebooks<a class="headerlink" href="#qui
           <div class="page-nav">
             <div class="inner"><ul class="page-nav">
 </ul><div class="footer" role="contentinfo">
-      &#169; Copyright 2024.
+      &#169; Copyright Tharsis T. P. Souza, 2024.
     <br>
     Created using <a href="http://sphinx-doc.org/">Sphinx</a> 6.2.1 with <a href="https://github.com/schettino72/sphinx_press_theme">Press Theme</a> 0.9.1.
 </div>

diff --git a/tamingllms/_build/html/markdown/markdown.html b/tamingllms/_build/html/markdown/markdown.html
@@ -99,7 +99,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism</a>
+              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
 
 
 
@@ -108,7 +108,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
+              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
 
 
 
@@ -117,7 +117,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
+              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism & Evals</a>
 
 
 
@@ -226,7 +226,7 @@ <h2>Learn more<a class="headerlink" href="#learn-more" title="Permalink to this
           <div class="page-nav">
             <div class="inner"><ul class="page-nav">
 </ul><div class="footer" role="contentinfo">
-      &#169; Copyright 2024.
+      &#169; Copyright Tharsis T. P. Souza, 2024.
     <br>
     Created using <a href="http://sphinx-doc.org/">Sphinx</a> 6.2.1 with <a href="https://github.com/schettino72/sphinx_press_theme">Press Theme</a> 0.9.1.
 </div>

diff --git a/tamingllms/_build/html/markdown/output_size_limit-NOTES.html b/tamingllms/_build/html/markdown/output_size_limit-NOTES.html
@@ -99,7 +99,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism</a>
+              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
 
 
 
@@ -108,7 +108,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
+              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
 
 
 
@@ -117,7 +117,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
+              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism & Evals</a>
 
 
 
@@ -396,7 +396,7 @@ <h2>References<a class="headerlink" href="#references" title="Permalink to this
           <div class="page-nav">
             <div class="inner"><ul class="page-nav">
 </ul><div class="footer" role="contentinfo">
-      &#169; Copyright 2024.
+      &#169; Copyright Tharsis T. P. Souza, 2024.
     <br>
     Created using <a href="http://sphinx-doc.org/">Sphinx</a> 6.2.1 with <a href="https://github.com/schettino72/sphinx_press_theme">Press Theme</a> 0.9.1.
 </div>

diff --git a/tamingllms/_build/html/markdown/structured_output.html b/tamingllms/_build/html/markdown/structured_output.html
@@ -99,7 +99,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism</a>
+              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
 
 
 
@@ -108,7 +108,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
+              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
 
 
 
@@ -117,7 +117,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
+              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism & Evals</a>
 
 
 
@@ -521,7 +521,7 @@ <h2>Conclusion<a class="headerlink" href="#conclusion" title="Permalink to this
           <div class="page-nav">
             <div class="inner"><ul class="page-nav">
 </ul><div class="footer" role="contentinfo">
-      &#169; Copyright 2024.
+      &#169; Copyright Tharsis T. P. Souza, 2024.
     <br>
     Created using <a href="http://sphinx-doc.org/">Sphinx</a> 6.2.1 with <a href="https://github.com/schettino72/sphinx_press_theme">Press Theme</a> 0.9.1.
 </div>

diff --git a/tamingllms/_build/html/markdown/toc.html b/tamingllms/_build/html/markdown/toc.html
@@ -100,7 +100,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism</a>
+              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
 
 
 
@@ -109,7 +109,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
+              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
 
 
 
@@ -118,7 +118,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="../notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
+              <a href="../notebooks/nondeterminism.html" class="reference internal ">Non-determinism & Evals</a>
 
 
 
@@ -333,7 +333,7 @@ <h2>Appendix B: Tools and Resources<a class="headerlink" href="#appendix-b-tools
        title="next chapter">Introduction →</a>
   </li>
 </ul><div class="footer" role="contentinfo">
-      &#169; Copyright 2024.
+      &#169; Copyright Tharsis T. P. Souza, 2024.
     <br>
     Created using <a href="http://sphinx-doc.org/">Sphinx</a> 6.2.1 with <a href="https://github.com/schettino72/sphinx_press_theme">Press Theme</a> 0.9.1.
 </div>

diff --git a/tamingllms/_build/html/nondeterminism-NOTES.html b/tamingllms/_build/html/nondeterminism-NOTES.html
@@ -99,7 +99,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="notebooks/nondeterminism.html" class="reference internal ">Non-determinism</a>
+              <a href="notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
 
 
 
@@ -108,7 +108,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="notebooks/output_size_limit.html" class="reference internal ">Output Size Limitations</a>
+              <a href="notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
 
 
 
@@ -117,7 +117,7 @@
 
           <li class="toctree-l1 ">
 
-              <a href="notebooks/structured_output.html" class="reference internal ">Wrestling with Structured Output</a>
+              <a href="notebooks/nondeterminism.html" class="reference internal ">Non-determinism & Evals</a>
 
 
 
@@ -378,7 +378,7 @@ <h2>References<a class="headerlink" href="#references" title="Permalink to this
           <div class="page-nav">
             <div class="inner"><ul class="page-nav">
 </ul><div class="footer" role="contentinfo">
-      &#169; Copyright 2024.
+      &#169; Copyright Tharsis T. P. Souza, 2024.
     <br>
     Created using <a href="http://sphinx-doc.org/">Sphinx</a> 6.2.1 with <a href="https://github.com/schettino72/sphinx_press_theme">Press Theme</a> 0.9.1.
 </div>