Skip to content

Commit

Permalink
Lakehouse Engine 1.23.0 documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
Lakehouse Engine committed Oct 28, 2024
1 parent f5a14bd commit b7229ff
Show file tree
Hide file tree
Showing 155 changed files with 5,155 additions and 5,037 deletions.
2 changes: 1 addition & 1 deletion lakehouse_engine.html
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ <h2>Submodules</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/algorithms.html
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ <h2>Submodules</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
493 changes: 251 additions & 242 deletions lakehouse_engine/algorithms/algorithm.html

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion lakehouse_engine/algorithms/data_loader.html
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/algorithms/dq_validator.html
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
6 changes: 5 additions & 1 deletion lakehouse_engine/algorithms/exceptions.html
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down Expand Up @@ -152,6 +152,7 @@ <h5>Inherited Members</h5>
</div>
<div><dt>builtins.BaseException</dt>
<dd id="ReconciliationFailedException.with_traceback" class="function">with_traceback</dd>
<dd id="ReconciliationFailedException.add_note" class="function">add_note</dd>
<dd id="ReconciliationFailedException.args" class="variable">args</dd>

</div>
Expand Down Expand Up @@ -189,6 +190,7 @@ <h5>Inherited Members</h5>
</div>
<div><dt>builtins.BaseException</dt>
<dd id="NoNewDataException.with_traceback" class="function">with_traceback</dd>
<dd id="NoNewDataException.add_note" class="function">add_note</dd>
<dd id="NoNewDataException.args" class="variable">args</dd>

</div>
Expand Down Expand Up @@ -226,6 +228,7 @@ <h5>Inherited Members</h5>
</div>
<div><dt>builtins.BaseException</dt>
<dd id="SensorAlreadyExistsException.with_traceback" class="function">with_traceback</dd>
<dd id="SensorAlreadyExistsException.add_note" class="function">add_note</dd>
<dd id="SensorAlreadyExistsException.args" class="variable">args</dd>

</div>
Expand Down Expand Up @@ -263,6 +266,7 @@ <h5>Inherited Members</h5>
</div>
<div><dt>builtins.BaseException</dt>
<dd id="RestoreTypeNotFoundException.with_traceback" class="function">with_traceback</dd>
<dd id="RestoreTypeNotFoundException.add_note" class="function">add_note</dd>
<dd id="RestoreTypeNotFoundException.args" class="variable">args</dd>

</div>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/algorithms/gab.html
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/algorithms/reconciliator.html
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
643 changes: 326 additions & 317 deletions lakehouse_engine/algorithms/sensor.html

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion lakehouse_engine/configs.html
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@



<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/core.html
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ <h2>Submodules</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/core/dbfs_file_manager.html
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
3,981 changes: 2,005 additions & 1,976 deletions lakehouse_engine/core/definitions.html

Large diffs are not rendered by default.

420 changes: 213 additions & 207 deletions lakehouse_engine/core/exec_env.html

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion lakehouse_engine/core/executable.html
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/core/file_manager.html
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/core/gab_manager.html
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/core/gab_sql_generator.html
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/core/s3_file_manager.html
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
16 changes: 8 additions & 8 deletions lakehouse_engine/core/sensor_manager.html
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand All @@ -111,7 +111,7 @@ <h1 class="modulename">
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">delta.tables</span> <span class="kn">import</span> <span class="n">DeltaTable</span>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">pyspark.sql</span> <span class="kn">import</span> <span class="n">DataFrame</span><span class="p">,</span> <span class="n">Row</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">pyspark.sql.functions</span> <span class="kn">import</span> <span class="n">array</span><span class="p">,</span> <span class="n">lit</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">pyspark.sql.functions</span> <span class="kn">import</span> <span class="n">array</span><span class="p">,</span> <span class="n">col</span><span class="p">,</span> <span class="n">lit</span>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a>
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">from</span> <span class="nn">lakehouse_engine.core.definitions</span> <span class="kn">import</span> <span class="p">(</span>
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a> <span class="n">SENSOR_SCHEMA</span><span class="p">,</span>
Expand Down Expand Up @@ -316,9 +316,9 @@ <h1 class="modulename">
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="p">)</span><span class="o">.</span><span class="n">toDF</span><span class="p">()</span>
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a>
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="k">if</span> <span class="n">sensor_id</span><span class="p">:</span>
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">sensor_id</span> <span class="o">==</span> <span class="n">sensor_id</span><span class="p">)</span>
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;sensor_id&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="n">sensor_id</span><span class="p">)</span>
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="k">elif</span> <span class="n">assets</span><span class="p">:</span>
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">assets</span> <span class="o">==</span> <span class="n">array</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="n">lit</span><span class="p">(</span><span class="n">asset</span><span class="p">)</span> <span class="k">for</span> <span class="n">asset</span> <span class="ow">in</span> <span class="n">assets</span><span class="p">]))</span>
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;assets&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="n">array</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="n">lit</span><span class="p">(</span><span class="n">asset</span><span class="p">)</span> <span class="k">for</span> <span class="n">asset</span> <span class="ow">in</span> <span class="n">assets</span><span class="p">]))</span>
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="s2">&quot;Either sensor_id or assets need to be provided as arguments.&quot;</span>
Expand Down Expand Up @@ -715,9 +715,9 @@ <h1 class="modulename">
</span><span id="SensorControlTableManager-211"><a href="#SensorControlTableManager-211"><span class="linenos">211</span></a> <span class="p">)</span><span class="o">.</span><span class="n">toDF</span><span class="p">()</span>
</span><span id="SensorControlTableManager-212"><a href="#SensorControlTableManager-212"><span class="linenos">212</span></a>
</span><span id="SensorControlTableManager-213"><a href="#SensorControlTableManager-213"><span class="linenos">213</span></a> <span class="k">if</span> <span class="n">sensor_id</span><span class="p">:</span>
</span><span id="SensorControlTableManager-214"><a href="#SensorControlTableManager-214"><span class="linenos">214</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">sensor_id</span> <span class="o">==</span> <span class="n">sensor_id</span><span class="p">)</span>
</span><span id="SensorControlTableManager-214"><a href="#SensorControlTableManager-214"><span class="linenos">214</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;sensor_id&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="n">sensor_id</span><span class="p">)</span>
</span><span id="SensorControlTableManager-215"><a href="#SensorControlTableManager-215"><span class="linenos">215</span></a> <span class="k">elif</span> <span class="n">assets</span><span class="p">:</span>
</span><span id="SensorControlTableManager-216"><a href="#SensorControlTableManager-216"><span class="linenos">216</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">assets</span> <span class="o">==</span> <span class="n">array</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="n">lit</span><span class="p">(</span><span class="n">asset</span><span class="p">)</span> <span class="k">for</span> <span class="n">asset</span> <span class="ow">in</span> <span class="n">assets</span><span class="p">]))</span>
</span><span id="SensorControlTableManager-216"><a href="#SensorControlTableManager-216"><span class="linenos">216</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;assets&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="n">array</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="n">lit</span><span class="p">(</span><span class="n">asset</span><span class="p">)</span> <span class="k">for</span> <span class="n">asset</span> <span class="ow">in</span> <span class="n">assets</span><span class="p">]))</span>
</span><span id="SensorControlTableManager-217"><a href="#SensorControlTableManager-217"><span class="linenos">217</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="SensorControlTableManager-218"><a href="#SensorControlTableManager-218"><span class="linenos">218</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
</span><span id="SensorControlTableManager-219"><a href="#SensorControlTableManager-219"><span class="linenos">219</span></a> <span class="s2">&quot;Either sensor_id or assets need to be provided as arguments.&quot;</span>
Expand Down Expand Up @@ -905,9 +905,9 @@ <h6 id="arguments">Arguments:</h6>
</span><span id="SensorControlTableManager.read_sensor_table_data-211"><a href="#SensorControlTableManager.read_sensor_table_data-211"><span class="linenos">211</span></a> <span class="p">)</span><span class="o">.</span><span class="n">toDF</span><span class="p">()</span>
</span><span id="SensorControlTableManager.read_sensor_table_data-212"><a href="#SensorControlTableManager.read_sensor_table_data-212"><span class="linenos">212</span></a>
</span><span id="SensorControlTableManager.read_sensor_table_data-213"><a href="#SensorControlTableManager.read_sensor_table_data-213"><span class="linenos">213</span></a> <span class="k">if</span> <span class="n">sensor_id</span><span class="p">:</span>
</span><span id="SensorControlTableManager.read_sensor_table_data-214"><a href="#SensorControlTableManager.read_sensor_table_data-214"><span class="linenos">214</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">sensor_id</span> <span class="o">==</span> <span class="n">sensor_id</span><span class="p">)</span>
</span><span id="SensorControlTableManager.read_sensor_table_data-214"><a href="#SensorControlTableManager.read_sensor_table_data-214"><span class="linenos">214</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;sensor_id&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="n">sensor_id</span><span class="p">)</span>
</span><span id="SensorControlTableManager.read_sensor_table_data-215"><a href="#SensorControlTableManager.read_sensor_table_data-215"><span class="linenos">215</span></a> <span class="k">elif</span> <span class="n">assets</span><span class="p">:</span>
</span><span id="SensorControlTableManager.read_sensor_table_data-216"><a href="#SensorControlTableManager.read_sensor_table_data-216"><span class="linenos">216</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">assets</span> <span class="o">==</span> <span class="n">array</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="n">lit</span><span class="p">(</span><span class="n">asset</span><span class="p">)</span> <span class="k">for</span> <span class="n">asset</span> <span class="ow">in</span> <span class="n">assets</span><span class="p">]))</span>
</span><span id="SensorControlTableManager.read_sensor_table_data-216"><a href="#SensorControlTableManager.read_sensor_table_data-216"><span class="linenos">216</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;assets&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="n">array</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="n">lit</span><span class="p">(</span><span class="n">asset</span><span class="p">)</span> <span class="k">for</span> <span class="n">asset</span> <span class="ow">in</span> <span class="n">assets</span><span class="p">]))</span>
</span><span id="SensorControlTableManager.read_sensor_table_data-217"><a href="#SensorControlTableManager.read_sensor_table_data-217"><span class="linenos">217</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="SensorControlTableManager.read_sensor_table_data-218"><a href="#SensorControlTableManager.read_sensor_table_data-218"><span class="linenos">218</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
</span><span id="SensorControlTableManager.read_sensor_table_data-219"><a href="#SensorControlTableManager.read_sensor_table_data-219"><span class="linenos">219</span></a> <span class="s2">&quot;Either sensor_id or assets need to be provided as arguments.&quot;</span>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/core/table_manager.html
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/dq_processors.html
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ <h2>Submodules</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
2 changes: 1 addition & 1 deletion lakehouse_engine/dq_processors/custom_expectations.html
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ <h2>Submodules</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ <h2>API Documentation</h2>
</ul>


<footer>Lakehouse Engine v1.22.2</footer>
<footer>Lakehouse Engine v1.23.0</footer>

</div>
</nav>
Expand Down
Loading

0 comments on commit b7229ff

Please sign in to comment.