update structured

tamingllms/_build/html/_sources/notebooks/structured_output.ipynb

@@ -777,6 +777,19 @@
     "\n",
     "On the other hand, those findings are not without criticism. The .txt team challenges the findings of the paper \"Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models\" {cite}`tam2024letspeakfreelystudy`. The paper claims that structured output formats, like JSON, negatively affect the performance of LLMs, particularly when it comes to reasoning. The rebuttal argues that **structured generation, when done correctly, actually *improves* performance**.\n",
     "\n",
+    "\n",
+    "```{figure} ../_static/structured_output/rebuttal.png\n",
+    "---\n",
+    "name: structured_vs_unstructured\n",
+    "alt: Structured vs Unstructured Results by .txt team\n",
+    "scale: 60%\n",
+    "align: center\n",
+    "---\n",
+    "Structured vs Unstructured Results by .txt team.\n",
+    "```\n",
+    "The .txt team presents compelling evidence through their reproduction of the paper's experiments. While their unstructured results align with the original paper's findings, their structured results paint a dramatically different picture - demonstrating that structured generation actually improves performance (see {numref}`structured_vs_unstructured`). The team has made their experimental notebooks publicly available on GitHub for independent verification {cite}`dottxt2024demos`.\n",
+    "\n",
+    "\n",
     ".txt team identifies several flaws in the methodology of \"Let Me Speak Freely?\" that they believe led to inaccurate conclusions:\n",
     "\n",
     "*   The paper finds that structured output improves performance on classification tasks but doesn't reconcile this finding with its overall negative conclusion about structured output. \n",

tamingllms/_build/jupyter_execute/markdown/intro.ipynb

@@ -2,7 +2,7 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "04e29306",
+   "id": "af54935e",
    "metadata": {},
    "source": [
     "(intro)=\n",

tamingllms/_build/jupyter_execute/notebooks/structured_output.ipynb

@@ -777,6 +777,19 @@
     "\n",
     "On the other hand, those findings are not without criticism. The .txt team challenges the findings of the paper \"Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models\" {cite}`tam2024letspeakfreelystudy`. The paper claims that structured output formats, like JSON, negatively affect the performance of LLMs, particularly when it comes to reasoning. The rebuttal argues that **structured generation, when done correctly, actually *improves* performance**.\n",
     "\n",
+    "\n",
+    "```{figure} ../_static/structured_output/rebuttal.png\n",
+    "---\n",
+    "name: structured_vs_unstructured\n",
+    "alt: Structured vs Unstructured Results by .txt team\n",
+    "scale: 60%\n",
+    "align: center\n",
+    "---\n",
+    "Structured vs Unstructured Results by .txt team.\n",
+    "```\n",
+    "The .txt team presents compelling evidence through their reproduction of the paper's experiments. While their unstructured results align with the original paper's findings, their structured results paint a dramatically different picture - demonstrating that structured generation actually improves performance (see {numref}`structured_vs_unstructured`). The team has made their experimental notebooks publicly available on GitHub for independent verification {cite}`dottxt2024demos`.\n",
+    "\n",
+    "\n",
     ".txt team identifies several flaws in the methodology of \"Let Me Speak Freely?\" that they believe led to inaccurate conclusions:\n",
     "\n",
     "*   The paper finds that structured output improves performance on classification tasks but doesn't reconcile this finding with its overall negative conclusion about structured output. \n",

tamingllms/notebooks/structured_output.ipynb

@@ -777,6 +777,19 @@
     "\n",
     "On the other hand, those findings are not without criticism. The .txt team challenges the findings of the paper \"Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models\" {cite}`tam2024letspeakfreelystudy`. The paper claims that structured output formats, like JSON, negatively affect the performance of LLMs, particularly when it comes to reasoning. The rebuttal argues that **structured generation, when done correctly, actually *improves* performance**.\n",
     "\n",
+    "\n",
+    "```{figure} ../_static/structured_output/rebuttal.png\n",
+    "---\n",
+    "name: structured_vs_unstructured\n",
+    "alt: Structured vs Unstructured Results by .txt team\n",
+    "scale: 60%\n",
+    "align: center\n",
+    "---\n",
+    "Structured vs Unstructured Results by .txt team.\n",
+    "```\n",
+    "The .txt team presents compelling evidence through their reproduction of the paper's experiments. While their unstructured results align with the original paper's findings, their structured results paint a dramatically different picture - demonstrating that structured generation actually improves performance (see {numref}`structured_vs_unstructured`). The team has made their experimental notebooks publicly available on GitHub for independent verification {cite}`dottxt2024demos`.\n",
+    "\n",
+    "\n",
     ".txt team identifies several flaws in the methodology of \"Let Me Speak Freely?\" that they believe led to inaccurate conclusions:\n",
     "\n",
     "*   The paper finds that structured output improves performance on classification tasks but doesn't reconcile this finding with its overall negative conclusion about structured output. \n",

tamingllms/references.bib

@@ -45,3 +45,12 @@ @misc{aider2024codejson
       howpublished={\url{https://aider.chat/2024/08/14/code-in-json.html}},
       note={Accessed: 2024}
 }
+
+@misc{dottxt2024demos,
+      title={Say What You Mean: Demos},
+      author={Dottxt},
+      year={2024},
+      howpublished={\url{https://github.com/dottxt-ai/demos/tree/main/say-what-you-mean}},
+      note={Accessed: 2024}
+}
+