Skip to content

Commit

Permalink
update alignment notebook
Browse files Browse the repository at this point in the history
  • Loading branch information
souzatharsis committed Dec 14, 2024
1 parent 808a8a7 commit 86ae400
Show file tree
Hide file tree
Showing 8 changed files with 89 additions and 4 deletions.
Binary file modified tamingllms/_build/.doctrees/environment.pickle
View file
Binary file not shown.
Binary file modified tamingllms/_build/.doctrees/notebooks/alignment.doctree
View file
Binary file not shown.
23 changes: 23 additions & 0 deletions tamingllms/_build/html/_sources/notebooks/alignment.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2232,6 +2232,29 @@
"One important limitation of our current implementation is that we did not carefully split our user prompts between in-sample data for fine-tuning and out-of-sample data for evaluation. This means our evaluation metrics may be overly optimistic as the fine-tuned model could be memorizing prompts rather than learning generalizable alignment. Future work should implement proper train/test splits to better assess generalization performance while making sure out/in-sample distributions are similar and representative of real-world data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Citation\n",
"[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]\n",
"\n",
"[cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/\n",
"[cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png\n",
"[cc-by-nc-sa-shield]: https://img.shields.io/badge/License-CC-BY--NC--SA-4.0-lightgrey.svg\n",
"\n",
"```\n",
"@misc{tharsistpsouza2024tamingllms,\n",
" author = {Tharsis T. P. Souza},\n",
" title = {Taming LLMs},\n",
" year = {2024},\n",
" journal = {GitHub repository},\n",
" url = {https://github.com/souzatharsis/tamingLLMs)\n",
"}\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down
20 changes: 18 additions & 2 deletions tamingllms/_build/html/notebooks/alignment.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,8 @@

<li class="toctree-l2"><a href="#case-study-aligning-a-language-model-to-a-policy" class="reference internal">Case Study: Aligning a Language Model to a Policy</a></li>

<li class="toctree-l2"><a href="#citation" class="reference internal">Citation</a></li>

<li class="toctree-l2"><a href="#references" class="reference internal">References</a></li>

</ul>
Expand Down Expand Up @@ -235,7 +237,8 @@ <h1><a class="toc-backref" href="#id89" role="doc-backlink"><span class="section
<li><p><a class="reference internal" href="#discussion" id="id106">Discussion</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#references" id="id107">References</a></p></li>
<li><p><a class="reference internal" href="#citation" id="id107">Citation</a></p></li>
<li><p><a class="reference internal" href="#references" id="id108">References</a></p></li>
</ul>
</li>
</ul>
Expand Down Expand Up @@ -2056,8 +2059,21 @@ <h3><a class="toc-backref" href="#id106" role="doc-backlink"><span class="sectio
<p>One important limitation of our current implementation is that we did not carefully split our user prompts between in-sample data for fine-tuning and out-of-sample data for evaluation. This means our evaluation metrics may be overly optimistic as the fine-tuned model could be memorizing prompts rather than learning generalizable alignment. Future work should implement proper train/test splits to better assess generalization performance while making sure out/in-sample distributions are similar and representative of real-world data.</p>
</section>
</section>
<section id="citation">
<h2><a class="toc-backref" href="#id107" role="doc-backlink"><span class="section-number">5.2. </span>Citation</a><a class="headerlink" href="#citation" title="Permalink to this heading"></a></h2>
<p><a class="reference external" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="CC BY-NC-SA 4.0" src="https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png" /></a></p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@misc</span><span class="p">{</span><span class="n">tharsistpsouza2024tamingllms</span><span class="p">,</span>
<span class="n">author</span> <span class="o">=</span> <span class="p">{</span><span class="n">Tharsis</span> <span class="n">T</span><span class="o">.</span> <span class="n">P</span><span class="o">.</span> <span class="n">Souza</span><span class="p">},</span>
<span class="n">title</span> <span class="o">=</span> <span class="p">{</span><span class="n">Taming</span> <span class="n">LLMs</span><span class="p">},</span>
<span class="n">year</span> <span class="o">=</span> <span class="p">{</span><span class="mi">2024</span><span class="p">},</span>
<span class="n">journal</span> <span class="o">=</span> <span class="p">{</span><span class="n">GitHub</span> <span class="n">repository</span><span class="p">},</span>
<span class="n">url</span> <span class="o">=</span> <span class="p">{</span><span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">souzatharsis</span><span class="o">/</span><span class="n">tamingLLMs</span><span class="p">)</span>
<span class="p">}</span>
</pre></div>
</div>
</section>
<section id="references">
<h2><a class="toc-backref" href="#id107" role="doc-backlink"><span class="section-number">5.2. </span>References</a><a class="headerlink" href="#references" title="Permalink to this heading"></a></h2>
<h2><a class="toc-backref" href="#id108" role="doc-backlink"><span class="section-number">5.3. </span>References</a><a class="headerlink" href="#references" title="Permalink to this heading"></a></h2>
<div class="docutils container" id="id22">
<div class="citation" id="id77" role="doc-biblioentry">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id5">BKK+22</a><span class="fn-bracket">]</span></span>
Expand Down
2 changes: 1 addition & 1 deletion tamingllms/_build/html/searchindex.js
View file

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion tamingllms/_build/jupyter_execute/markdown/intro.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "9a3aac65",
"id": "47492779",
"metadata": {},
"source": [
"(intro)=\n",
Expand Down
23 changes: 23 additions & 0 deletions tamingllms/_build/jupyter_execute/notebooks/alignment.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2232,6 +2232,29 @@
"One important limitation of our current implementation is that we did not carefully split our user prompts between in-sample data for fine-tuning and out-of-sample data for evaluation. This means our evaluation metrics may be overly optimistic as the fine-tuned model could be memorizing prompts rather than learning generalizable alignment. Future work should implement proper train/test splits to better assess generalization performance while making sure out/in-sample distributions are similar and representative of real-world data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Citation\n",
"[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]\n",
"\n",
"[cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/\n",
"[cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png\n",
"[cc-by-nc-sa-shield]: https://img.shields.io/badge/License-CC-BY--NC--SA-4.0-lightgrey.svg\n",
"\n",
"```\n",
"@misc{tharsistpsouza2024tamingllms,\n",
" author = {Tharsis T. P. Souza},\n",
" title = {Taming LLMs},\n",
" year = {2024},\n",
" journal = {GitHub repository},\n",
" url = {https://github.com/souzatharsis/tamingLLMs)\n",
"}\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down
23 changes: 23 additions & 0 deletions tamingllms/notebooks/alignment.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2232,6 +2232,29 @@
"One important limitation of our current implementation is that we did not carefully split our user prompts between in-sample data for fine-tuning and out-of-sample data for evaluation. This means our evaluation metrics may be overly optimistic as the fine-tuned model could be memorizing prompts rather than learning generalizable alignment. Future work should implement proper train/test splits to better assess generalization performance while making sure out/in-sample distributions are similar and representative of real-world data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Citation\n",
"[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]\n",
"\n",
"[cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/\n",
"[cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png\n",
"[cc-by-nc-sa-shield]: https://img.shields.io/badge/License-CC-BY--NC--SA-4.0-lightgrey.svg\n",
"\n",
"```\n",
"@misc{tharsistpsouza2024tamingllms,\n",
" author = {Tharsis T. P. Souza},\n",
" title = {Taming LLMs},\n",
" year = {2024},\n",
" journal = {GitHub repository},\n",
" url = {https://github.com/souzatharsis/tamingLLMs)\n",
"}\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down

0 comments on commit 86ae400

Please sign in to comment.