Skip to content

Commit

Permalink
Release v0.5.10 (#36)
Browse files Browse the repository at this point in the history
* Bump abatilo/actions-poetry from 2.1.6 to 2.3.0

Bumps [abatilo/actions-poetry](https://github.com/abatilo/actions-poetry) from 2.1.6 to 2.3.0.
- [Release notes](https://github.com/abatilo/actions-poetry/releases)
- [Changelog](https://github.com/abatilo/actions-poetry/blob/master/.releaserc)
- [Commits](abatilo/actions-poetry@v2.1.6...v2.3.0)

---
updated-dependencies:
- dependency-name: abatilo/actions-poetry
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump mkdocstrings from 0.19.1 to 0.25.1

Bumps [mkdocstrings](https://github.com/mkdocstrings/mkdocstrings) from 0.19.1 to 0.25.1.
- [Release notes](https://github.com/mkdocstrings/mkdocstrings/releases)
- [Changelog](https://github.com/mkdocstrings/mkdocstrings/blob/main/CHANGELOG.md)
- [Commits](mkdocstrings/mkdocstrings@0.19.1...0.25.1)

---
updated-dependencies:
- dependency-name: mkdocstrings
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* (chore) lock update and update in CICD (#35)

* (chore) lock update and update in CICD
* (chore) updated preprocessor and added testing
* (chore) updated pre commit hooks and versions
* (chore) updated test to match boostrap distro, improved test error msg.
* (chore) upgraded missing versions of checkout and python setup in cicd

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
jspaezp and dependabot[bot] authored Aug 30, 2024
1 parent 82250d3 commit c57d586
Showing 14 changed files with 1,766 additions and 1,394 deletions.
20 changes: 10 additions & 10 deletions .github/workflows/CICD.yml
Original file line number Diff line number Diff line change
@@ -12,10 +12,10 @@ jobs:
cheap_checks:
runs-on: 'ubuntu-latest'
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

# Python setup
- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: '3.x'

@@ -39,7 +39,7 @@ jobs:
- name: Black
run: poetry run python -m black . --check --diff
- name: ruff
run: poetry run ruff .
run: poetry run ruff check .
- name: isort
run: poetry run isort . --check

@@ -54,7 +54,7 @@ jobs:
runs-on: ${{ matrix.os }}
steps:
# Initial Setup
- uses: actions/checkout@v3
- uses: actions/checkout@v4

# Documentation setup
- name: Setup Quarto
@@ -63,7 +63,7 @@ jobs:
run: quarto --version

# Python setup
- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: View Python --version
@@ -88,7 +88,7 @@ jobs:
- name: mkdocs
run: poetry run python -m mkdocs build --strict

- uses: codecov/codecov-action@v3
- uses: codecov/codecov-action@v4
with:
verbose: true # optional (default = false)

@@ -98,9 +98,9 @@ jobs:
needs: ci
if: endsWith(github.ref, 'heads/release') && (github.event_name == 'push')
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Release
uses: softprops/action-gh-release@v1
uses: softprops/action-gh-release@v2
if: startsWith(github.ref, 'refs/tags/')

# Documentation setup
@@ -110,15 +110,15 @@ jobs:
run: quarto --version

# Python setup
- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: View Python --version
run: python --version

# Poetry Setup
- name: Python Poetry Action
uses: abatilo/actions-poetry@v2.1.6
uses: abatilo/actions-poetry@v2.3.0
with:
poetry-version: '1.7.1'
- name: View poetry --version
4 changes: 2 additions & 2 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -6,10 +6,10 @@ jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Black
uses: psf/black@stable
- name: Set up Python environment
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: "3.9"
8 changes: 4 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -10,20 +10,20 @@ repos:
exclude: "(.*tests/data/.*/.*.md)"
- id: debug-statements
- repo: https://github.com/asottile/pyupgrade
rev: v3.15.2
rev: v3.17.0
hooks:
- id: pyupgrade
- repo: https://github.com/PyCQA/autoflake
rev: v2.3.1
hooks:
- id: autoflake
- repo: https://github.com/psf/black
rev: 24.4.0
rev: 24.8.0
hooks:
- id: black
language_version: python3.8
language_version: python3.9
- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: v0.3.7
rev: v0.6.3
hooks:
- id: ruff
- repo: https://github.com/pycqa/pydocstyle
42 changes: 26 additions & 16 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
---
toc-title: Table of contents
---

![Pypi
version](https://img.shields.io/pypi/v/mkquartodocs?style=flat-square.png)
@@ -42,28 +45,32 @@ copying outpus.
This will render code chunks and save the outputs! Check out
https://quarto.org/ for more examples on how to use the format.

This .
This ....

```` markdown

```{python}
print(1+1)
print(1+2)
```
````

Will become this
Will become this ...

``` python
print(1+1)
:::: {.cell execution_count="1"}
``` {.python .cell-code}
print(1+2)
```

2
::: {.cell-output .cell-output-stdout}
3
:::
::::

## Installation

1. Make sure you have quarto installed in your computer.

- https://quarto.org/docs/get-started/
- https://quarto.org/docs/get-started/

2. Install `mkquartodocs`

@@ -85,11 +92,11 @@ plugins:
Available configuration options:
- **quarto_path**: Specifies where to look for the quarto executable.
- **keep_output**: If true it will skip the cleanup step in the
directory.
- **ignore**: a python regular expressions that if matched will mark the
file to not be rendered. Note that they need to be full matches
- **quarto_path**: Specifies where to look for the quarto executable.
- **keep_output**: If true it will skip the cleanup step in the
directory.
- **ignore**: a python regular expressions that if matched will mark
the file to not be rendered. Note that they need to be full matches
``` yaml
# Whatever is in your mkdocs.yml configuration file....
@@ -113,7 +120,10 @@ files, render them, generate the output and clean after itself.

The things that need to/could be added to the project:

- [ ] quarto project support
- [ ] render in temporary directory, posibly with a ‘safe’ argument
- [ ] addition of files not in the docs directory, ‘include’ argument
- [ ] add readme to testing data
- [ ] quarto project support
- [ ] render in temporary directory, posibly with a 'safe' argument
- [ ] addition of files not in the docs directory, 'include' argument
- [ ] add readme to testing data
- [ ] move
`INFO - mkquartodocs: Running RemoveCellDataPreprocessor` to
debug log
4 changes: 2 additions & 2 deletions docs/README.qmd
Original file line number Diff line number Diff line change
@@ -37,15 +37,15 @@ This ....
````
```{{python}}
print(1+1)
print(1+2)
```
````

Will become this ...

```{python}
print(1+1)
print(1+2)
```

## Installation
18 changes: 11 additions & 7 deletions mkquartodocs/extension.py
Original file line number Diff line number Diff line change
@@ -15,9 +15,9 @@

class AdmotionCellDataPreprocessor(Preprocessor):
CELL_REGEX: Final = re.compile(r"^::: \{\.cell .*}\s*$")
CELL_END: Final = re.compile(r"^:::$")
CELL_END: Final = re.compile(r"^:{3,4}?$")
CELL_ELEM_REGEX: Final = re.compile(
r"^::: \{(.cell-\w+) (\.cell-[\w-]+)( execution_count=\"\d+\")?\}$"
r"^:{3,4} \{(\.cell(-\w+)?)\s?(\.cell-[\w-]+)?( execution_count=\"\d+\")?\}"
)
CODEBLOCK_REGEX: Final = re.compile(r"^```{\.(\w+) .*}")

@@ -34,7 +34,6 @@ def run(self, lines):
outs = [self._process_line(x) for x in lines]
log.debug(f"Removing {sum(1 for x in outs if x is None)} lines")
out = [x for x in outs if x is not None]

return out

def _process_line(self, line):
@@ -47,12 +46,17 @@ def _process_line(self, line):
out = "\n\n"

elif sr := self.CELL_ELEM_REGEX.search(line):
log.debug(f"Matched Cell element: {line}")
output_type = sr.groups()[1]
out = self.TYPE_MAPPING[output_type]
groups = {i: x for i, x in enumerate(sr.groups())}
log.debug(f"Matched Cell element: {line}, groups: {groups}")
if groups[3]:
out = "\n\n"
else:
output_type = groups[2]
out = self.TYPE_MAPPING[output_type]

elif sr := self.CODEBLOCK_REGEX.search(line):
log.debug(f"Matched codeblock: {line}")
groups = {i: x for i, x in enumerate(sr.groups())}
log.debug(f"Matched codeblock: {line} -> {groups}")
lang = sr.groups(1)
out = f"```{lang}"
else:
4 changes: 3 additions & 1 deletion mkquartodocs/plugin.py
Original file line number Diff line number Diff line change
@@ -85,10 +85,12 @@ def on_pre_build(self, config):
[quarto, "render", str(x), "--to=markdown"], check=True
)
break
except subprocess.CalledProcessError:
except subprocess.CalledProcessError as e:
# ERROR: Couldn't find open server
# it ocasionally fails with that error ...
if i == 4:
log.error(f"Quarto failed to render {x} after 5 tries")
log.error(f"Quarto failed with error: {e}")
raise
warnings.warn(f"Quarto failed to render {x}, retrying")
else:
1,626 changes: 917 additions & 709 deletions poetry.lock

Large diffs are not rendered by default.

16 changes: 8 additions & 8 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
[tool.poetry]
name = "mkquartodocs"
version = "0.5.9"
version = "0.5.10"
description = ""
authors = ["J. Sebastian Paez <jspaezp@users.noreply.github.com>"]
readme = "README.md"
packages = [{ include = "mkquartodocs" }]
license = "Apache 2.0"

[tool.poetry.dependencies]
python = ">=3.9,<3.13"
mkdocs = "^1.4.2"
python = ">=3.9,<3.14"
mkdocs = "^1.6.0"
jupyter = "^1.0.0"
nbformat = "^5.7.3"
pytest-cov = "^4.0.0"
@@ -18,13 +18,13 @@ pytest-cov = "^4.0.0"
black = "^22.8.0"
isort = "^5.10.1"
pylint = "^2.16.2"
ruff = "^0.0.46"
ruff = "^0.5.5"

[tool.poetry.group.dev.dependencies]
pydocstringformatter = "^0.7.2"
pyupgrade = "^2.37.3"
pytest-datadir = "^1.3.1"
pytest = "^7.2.1"
pytest = "^8.3.1"
pytest-cov = "^4.0.0"
mdformat = "^0.7.16"
mdformat-black = "^0.1.1"
@@ -34,9 +34,9 @@ mdformat_frontmatter = "^0.4.1"


[tool.poetry.group.docs.dependencies]
mkdocs-material = "^8.5.0"
mkdocs-autorefs = "^0.4.1"
mkdocstrings = { extras = ["python"], version = "^0.19.1" }
mkdocs-material = "^9.5.33"
mkdocs-autorefs = "^1.1.0"
mkdocstrings = { extras = ["python"], version = "^0.25.2" }


[build-system]
1,316 changes: 695 additions & 621 deletions requirements.txt

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions tests/data/docs_assets/docs/page.qmd
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# This is a file

> https://github.com/quarto-dev/quarto-cli/issues/10626
Right now mermaid diagrams are not rendered correctly on 1.5
So 1.6 pre-release is needed.

```{mermaid}
graph TD;
A-->B;
19 changes: 11 additions & 8 deletions tests/data/docs_ignore/expected_output.txt
Original file line number Diff line number Diff line change
@@ -14,22 +14,25 @@
./site/index.html
./site/css
./site/css/bootstrap.min.css
./site/css/font-awesome.min.css
# ./site/css/font-awesome.min.css # Changed in a later version of quarto
# In favor of fontawesome.min.css
./site/css/base.css
./site/js
./site/js/bootstrap.min.js
# ./site/js/bootstrap.min.js
# Replaced later for bootstrap.bundle.min.js
./site/js/base.js
./site/img
./site/img/favicon.ico
./site/img/grid.png
./site/sitemap.xml
./site/sitemap.xml.gz
./site/fonts
./site/fonts/fontawesome-webfont.svg
./site/fonts/fontawesome-webfont.woff2
./site/fonts/fontawesome-webfont.ttf
./site/fonts/fontawesome-webfont.woff
./site/fonts/fontawesome-webfont.eot
# Also chhanged in a later version of quarto/mkdocs
# ./site/fonts
# ./site/fonts/fontawesome-webfont.svg
# ./site/fonts/fontawesome-webfont.woff2
# ./site/fonts/fontawesome-webfont.ttf
# ./site/fonts/fontawesome-webfont.woff
# ./site/fonts/fontawesome-webfont.eot

# rendered my_ugly_page
./site/my_ugly_page
18 changes: 12 additions & 6 deletions tests/test_main.py
Original file line number Diff line number Diff line change
@@ -18,16 +18,22 @@ def test_build(shared_datadir, doc_dir):
build.build(cfg)

with open(doc_dir_base / "expected_output.txt") as f:
not_found = []
for line in f:
line = line.strip()
if line and line[0] != "#":
if not (doc_dir_base / line).exists():
# If missing, report the files that do
files = [str(x) for x in doc_dir_base.rglob("*")]
files = [x.replace(str(doc_dir_base), "") for x in files]
msg = f"File {line} not found"
msg += "\n" + "\n".join(files)
raise ValueError(msg)
not_found.append(line)

if not_found:
# If missing, report the files that do
files = [str(x) for x in doc_dir_base.rglob("*")]
files = [x.replace(str(doc_dir_base), "") for x in files]
msg = "Files expected but not found:"
msg += "\n" + "\n > ".join(not_found)
msg += "\n" + "\nFiles found:"
msg += "\n" + "\n".join(files)
raise ValueError(msg)

with open(doc_dir_base / "expected_missing.txt") as f:
for line in f:
60 changes: 60 additions & 0 deletions tests/test_preprocessor.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
from mkquartodocs.extension import AdmotionCellDataPreprocessor

sample_cell_elements = [
':::: {.cell execution_count="1"}',
"::: {.cell-output .cell-output-stdout}",
':::: {.cell execution_count="3"}',
"``` {.python .cell-code}",
"```",
"::::",
]

# Tests from quarto version 1.5.56
# Rendered with --to=markdown

EXAMMPLE_INPUT_FILE = """
:::: {.cell execution_count="3"}
``` {.python .cell-code}
import warnings
warnings.warn("This is a warning")
```
::: {.cell-output .cell-output-stderr}
... UserWarning: This is a warning
warnings.warn("This is a warning")
:::
::::
"""

EXAMPLE_OUTPUT_FILE = """
``` {.python .cell-code}
import warnings
warnings.warn("This is a warning")
```
!!! warning "stderr"
... UserWarning: This is a warning
warnings.warn("This is a warning")
"""


def test_conversion():
preprocessor = AdmotionCellDataPreprocessor()
out = [preprocessor._process_line(x) for x in sample_cell_elements]
assert out == [
"\n\n",
'!!! note "output"',
"\n\n",
"``` {.python .cell-code}",
"```",
"\n\n",
]


def test_conversion_file_chunk():
preprocessor = AdmotionCellDataPreprocessor()
file_lines = EXAMMPLE_INPUT_FILE.split("\n")
out = [preprocessor._process_line(x) for x in file_lines]

out_str = "\n".join(out)
assert EXAMPLE_OUTPUT_FILE.strip() == out_str.strip()

0 comments on commit c57d586

Please sign in to comment.