Await cloning process before initiating S3 upload

Check result of cloning and quit backup if there are errors
Add custom warnings and exceptions
Add an option to ignore certain repos/gists
Any repo excluded will also exclude the gist
Update requirements.txt and pyproject.toml
Check for files within subdirectories before initiating upload
Update runbook

.gitignore

@@ -168,3 +168,6 @@ cython_debug/
 !doc_gen/Makefile
 !doc_gen/static.css
 doc_gen/*
+
+# Examples
+!samples/*

README.md

@@ -51,31 +51,16 @@ git2s3 start
 <details>
 <summary><strong>Sourcing environment variables from an env file</strong></summary>
 
-Environment variables can be sourced using any `plaintext` / `JSON` / `YAML` file.
-The filepath should be provided as an argument during object instantiation.
-
-> _By default, `Git2S3` will look for a `.env` file in the current working directory._
-
-**Examples**
-
-- **CLI**
-```shell
-git2s3 start --env-file "/path/to/env/file"
-```
-
-- **IDE**
-```python
-import git2s3
-backup = git2s3.Git2S3(env_file='/path/to/env/file')
-backup.start()
-```
+> _By default, `Git2S3` will look for a `.env` file in the current working directory._<br>
+> Refer [samples] directory for examples.
 
 </details>
 
 - **GIT_API_URL** - GitHub API endpoint. Defaults to `https://api.github.com/`
 - **GIT_OWNER** - GitHub profile owner or organization name.
 - **GIT_TOKEN** - GitHub token to get ALL repos (including private).
-- **FIELDS** - Fields options to restore. Defaults to all.
+- **GIT_IGNORE** - List of repositories/gists to ignore. Defaults to `[]`
+- **SOURCE** - Source options `[repo, gist, wiki]` to back up. Defaults to all.
 - **LOG** - Log options to log to a ``file`` or ``stdout``. _Does not apply when custom logger is used_
 - **DEBUG** - Boolean flag to enable debug level logging. _Does not apply when custom logger is used_
 - **AWS_PROFILE_NAME** - AWS profile name. Uses the CLI config value ``AWS_DEFAULT_PROFILE`` by default.
@@ -162,3 +147,4 @@ Licensed under the [MIT License][license]
 [license]: https://github.com/thevickypedia/git2s3/blob/master/LICENSE
 [runbook]: https://thevickypedia.github.io/git2s3/
 [Boto3 retry configuration]: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html#available-retry-modes
+[samples]: https://github.com/thevickypedia/git2s3/tree/main/samples

doc_gen/index.rst

@@ -14,6 +14,7 @@ Welcome to Git2S3's documentation!
 
 Git2S3 - Main
 =============
+
 .. automodule:: git2s3.main
 
 S3
@@ -22,19 +23,18 @@ S3
 
 Squire
 ======
+
 .. automodule:: git2s3.squire
 
 Configuration
 =============
 
-.. autoclass:: git2s3.config.Field(BaseModel)
-   :members: EnvConfig
-   :exclude-members: _abc_impl, model_config, model_fields
+.. autoclass:: git2s3.config.DataStore(BaseModel)
+   :exclude-members: _abc_impl, model_config, model_fields, model_computed_fields
 
 ====
 
 .. autoclass:: git2s3.config.EnvConfig(BaseSettings)
-   :members: EnvConfig
    :exclude-members: _abc_impl, model_config, model_fields, model_computed_fields
 
 ====
@@ -43,7 +43,12 @@ Configuration
 
 ====
 
-.. autoclass:: git2s3.config.Fields(StrEnum)
+.. autoclass:: git2s3.config.SourceControl(StrEnum)
+
+Exceptions
+==========
+
+.. automodule:: git2s3.exc
 
 Indices and tables
 ==================

docs/README.html

@@ -87,31 +87,16 @@ <h2>Kick off<a class="headerlink" href="#kick-off" title="Permalink to this head
 <section id="environment-variables">
 <h2>Environment Variables<a class="headerlink" href="#environment-variables" title="Permalink to this heading">¶</a></h2>
 <details>
-<summary><strong>Sourcing environment variables from an env file</strong></summary><p>Environment variables can be sourced using any <code class="docutils literal notranslate"><span class="pre">plaintext</span></code> / <code class="docutils literal notranslate"><span class="pre">JSON</span></code> / <code class="docutils literal notranslate"><span class="pre">YAML</span></code> file.
-The filepath should be provided as an argument during object instantiation.</p>
-<blockquote>
-<div><p><em>By default, <code class="docutils literal notranslate"><span class="pre">Git2S3</span></code> will look for a <code class="docutils literal notranslate"><span class="pre">.env</span></code> file in the current working directory.</em></p>
+<summary><strong>Sourcing environment variables from an env file</strong></summary><blockquote>
+<div><p><em>By default, <code class="docutils literal notranslate"><span class="pre">Git2S3</span></code> will look for a <code class="docutils literal notranslate"><span class="pre">.env</span></code> file in the current working directory.</em><br>
+Refer <a class="reference external" href="https://github.com/thevickypedia/git2s3/tree/main/samples">samples</a> directory for examples.</p>
 </div></blockquote>
-<p><strong>Examples</strong></p>
-<ul class="simple">
-<li><p><strong>CLI</strong></p></li>
-</ul>
-<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>git2s3<span class="w"> </span>start<span class="w"> </span>--env-file<span class="w"> </span><span class="s2">&quot;/path/to/env/file&quot;</span>
-</pre></div>
-</div>
-<ul class="simple">
-<li><p><strong>IDE</strong></p></li>
-</ul>
-<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">git2s3</span>
-<span class="n">backup</span> <span class="o">=</span> <span class="n">git2s3</span><span class="o">.</span><span class="n">Git2S3</span><span class="p">(</span><span class="n">env_file</span><span class="o">=</span><span class="s1">&#39;/path/to/env/file&#39;</span><span class="p">)</span>
-<span class="n">backup</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
-</pre></div>
-</div>
 </details><ul class="simple">
 <li><p><strong>GIT_API_URL</strong> - GitHub API endpoint. Defaults to <code class="docutils literal notranslate"><span class="pre">https://api.github.com/</span></code></p></li>
 <li><p><strong>GIT_OWNER</strong> - GitHub profile owner or organization name.</p></li>
 <li><p><strong>GIT_TOKEN</strong> - GitHub token to get ALL repos (including private).</p></li>
-<li><p><strong>FIELDS</strong> - Fields options to restore. Defaults to all.</p></li>
+<li><p><strong>GIT_IGNORE</strong> - List of repositories/gists to ignore. Defaults to <code class="docutils literal notranslate"><span class="pre">[]</span></code></p></li>
+<li><p><strong>SOURCE</strong> - Source options <code class="docutils literal notranslate"><span class="pre">[repo,</span> <span class="pre">gist,</span> <span class="pre">wiki]</span></code> to back up. Defaults to all.</p></li>
 <li><p><strong>LOG</strong> - Log options to log to a <code class="docutils literal notranslate"><span class="pre">file</span></code> or <code class="docutils literal notranslate"><span class="pre">stdout</span></code>. <em>Does not apply when custom logger is used</em></p></li>
 <li><p><strong>DEBUG</strong> - Boolean flag to enable debug level logging. <em>Does not apply when custom logger is used</em></p></li>
 <li><p><strong>AWS_PROFILE_NAME</strong> - AWS profile name. Uses the CLI config value <code class="docutils literal notranslate"><span class="pre">AWS_DEFAULT_PROFILE</span></code> by default.</p></li>

docs/README.md

@@ -51,31 +51,16 @@ git2s3 start
 <details>
 <summary><strong>Sourcing environment variables from an env file</strong></summary>
 
-Environment variables can be sourced using any `plaintext` / `JSON` / `YAML` file.
-The filepath should be provided as an argument during object instantiation.
-
-> _By default, `Git2S3` will look for a `.env` file in the current working directory._
-
-**Examples**
-
-- **CLI**
-```shell
-git2s3 start --env-file "/path/to/env/file"
-```
-
-- **IDE**
-```python
-import git2s3
-backup = git2s3.Git2S3(env_file='/path/to/env/file')
-backup.start()
-```
+> _By default, `Git2S3` will look for a `.env` file in the current working directory._<br>
+> Refer [samples] directory for examples.
 
 </details>
 
 - **GIT_API_URL** - GitHub API endpoint. Defaults to `https://api.github.com/`
 - **GIT_OWNER** - GitHub profile owner or organization name.
 - **GIT_TOKEN** - GitHub token to get ALL repos (including private).
-- **FIELDS** - Fields options to restore. Defaults to all.
+- **GIT_IGNORE** - List of repositories/gists to ignore. Defaults to `[]`
+- **SOURCE** - Source options `[repo, gist, wiki]` to back up. Defaults to all.
 - **LOG** - Log options to log to a ``file`` or ``stdout``. _Does not apply when custom logger is used_
 - **DEBUG** - Boolean flag to enable debug level logging. _Does not apply when custom logger is used_
 - **AWS_PROFILE_NAME** - AWS profile name. Uses the CLI config value ``AWS_DEFAULT_PROFILE`` by default.
@@ -162,3 +147,4 @@ Licensed under the [MIT License][license]
 [license]: https://github.com/thevickypedia/git2s3/blob/master/LICENSE
 [runbook]: https://thevickypedia.github.io/git2s3/
 [Boto3 retry configuration]: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html#available-retry-modes
+[samples]: https://github.com/thevickypedia/git2s3/tree/main/samples

docs/_sources/README.md.txt

@@ -51,31 +51,16 @@ git2s3 start
 <details>
 <summary><strong>Sourcing environment variables from an env file</strong></summary>
 
-Environment variables can be sourced using any `plaintext` / `JSON` / `YAML` file.
-The filepath should be provided as an argument during object instantiation.
-
-> _By default, `Git2S3` will look for a `.env` file in the current working directory._
-
-**Examples**
-
-- **CLI**
-```shell
-git2s3 start --env-file "/path/to/env/file"
-```
-
-- **IDE**
-```python
-import git2s3
-backup = git2s3.Git2S3(env_file='/path/to/env/file')
-backup.start()
-```
+> _By default, `Git2S3` will look for a `.env` file in the current working directory._<br>
+> Refer [samples] directory for examples.
 
 </details>
 
 - **GIT_API_URL** - GitHub API endpoint. Defaults to `https://api.github.com/`
 - **GIT_OWNER** - GitHub profile owner or organization name.
 - **GIT_TOKEN** - GitHub token to get ALL repos (including private).
-- **FIELDS** - Fields options to restore. Defaults to all.
+- **GIT_IGNORE** - List of repositories/gists to ignore. Defaults to `[]`
+- **SOURCE** - Source options `[repo, gist, wiki]` to back up. Defaults to all.
 - **LOG** - Log options to log to a ``file`` or ``stdout``. _Does not apply when custom logger is used_
 - **DEBUG** - Boolean flag to enable debug level logging. _Does not apply when custom logger is used_
 - **AWS_PROFILE_NAME** - AWS profile name. Uses the CLI config value ``AWS_DEFAULT_PROFILE`` by default.
@@ -162,3 +147,4 @@ Licensed under the [MIT License][license]
 [license]: https://github.com/thevickypedia/git2s3/blob/master/LICENSE
 [runbook]: https://thevickypedia.github.io/git2s3/
 [Boto3 retry configuration]: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html#available-retry-modes
+[samples]: https://github.com/thevickypedia/git2s3/tree/main/samples

docs/_sources/index.rst.txt

@@ -14,6 +14,7 @@ Welcome to Git2S3's documentation!
 
 Git2S3 - Main
 =============
+
 .. automodule:: git2s3.main
 
 S3
@@ -22,19 +23,18 @@ S3
 
 Squire
 ======
+
 .. automodule:: git2s3.squire
 
 Configuration
 =============
 
-.. autoclass:: git2s3.config.Field(BaseModel)
-   :members: EnvConfig
-   :exclude-members: _abc_impl, model_config, model_fields
+.. autoclass:: git2s3.config.DataStore(BaseModel)
+   :exclude-members: _abc_impl, model_config, model_fields, model_computed_fields
 
 ====
 
 .. autoclass:: git2s3.config.EnvConfig(BaseSettings)
-   :members: EnvConfig
    :exclude-members: _abc_impl, model_config, model_fields, model_computed_fields
 
 ====
@@ -43,7 +43,12 @@ Configuration
 
 ====
 
-.. autoclass:: git2s3.config.Fields(StrEnum)
+.. autoclass:: git2s3.config.SourceControl(StrEnum)
+
+Exceptions
+==========
+
+.. automodule:: git2s3.exc
 
 Indices and tables
 ==================