Audit of Features missing in JupyterBook

[rowanc1]

Although mystmd and JupyterBook/sphinx are targeted at slightly different use-cases, there is overlap in creating websites (see background). Below is a list of missing features/directives/roles/ui that are not yet implemented in mystmd, we hope to have parity at the rendering level first, then move on to extensibility (see #181 as a major step).

We will try to keep this updated as we audit more jupyterbooks.

Directives

• prf 👩🏽‍🏫 Initial support for proofs and theorems #384
• tasks Task list rendering #187
• grids
• cards (header/footer)
• dropdown
• tab-set
• sphinx proof 👩🏽‍🏫 Initial support for proofs and theorems #384
• sphinx exercise 🧑🏼‍🏫 Add support for sphinx exercise  #399
• glossary / terms 🕵🏽‍♀️ Add glossary and term support #411
• literalinclude https://github.com/orgs/executablebooks/discussions/1026, Support include & literalinclude jupyterlab-myst#189 📑 Add literalinclude directive #610
• table Myst Markdown table directive not supported #593
• raw 🥩 Add raw directive #939
• epigraph 💬 Add epigraph, pull-quote and blockquote directive #961
• margin Margin directive #516
• sidebar 🖌 Add aside,margin and sidebar directives #1012
• topic 💅 🖼️ Add topic directive #1132
• div
• index
• grid-item
• grid-item-card
• csv-table Support for CSV table directive #893 ✨ Add csv-table directive #1030
• Support tableofcontents, toctree directives #1338
• toggle (show option)
• button-ref
• only Provide a role to inject code in the latex output #613
• footbibliography
• bibliography
• figure-md
• nb-exec-table
• tab
• panels
• contents
• field lists
• rubric
• card-carousel
• eval-rst
• example
• glue, glue:, glue:figure, glue:math
• raw-cell

Roles

• doc 👩‍⚕️ Add a {doc} role to support sphinx #387
• term
• download ⤵️ Add {download} role #440
• bdg-primary
• cite:ps, cite:ts Support additional citation roles #82
• fa
• py:class
• glue:, glue:text
• guilabel

Config

• edit_page_button, repository button, issues_button
• ... audit the _config.yml
• read the relevant parts of the _config.yml for sphinx

Extensibility

See #181 as a roadmap.

• transforms turned on (e.g. dollarmath, etc.)
• directives / roles Refactor directive and role parsing #181

Execution

• running code execution for a notebook Incorporate execution #550
• Linking artifacts from a notebook (glue) - we are supporting this using ids/imports (see docs)

Improvements

• code-cell: tags (output_scroll, hide) Myst parser does not handle the code-cell directive #284
• hide code or output 🏷 Propagate block metatags to code node and output node #407
• toggle code input: Control code cell visibility and behavior with tags and metadata  #1108
• admonition: classes
• admonition: dropdown class
• intersphinx

UI

• TOC - collapse 🪆 Improve Table of Contents myst-theme#254
• logo more like JB
• toc footer
• page footer
• banner
• search Support Search in MyST Sites #100
• favicon Support custom favicons #925
• internationalization Add support for translations / internationalization / localization of page elements #166

URL:

• Collapsing things down to a project/slug isn't how JB works Support folder hierarchies and nested files in URLs without flattening them #670

Export

• static html Export as Static HTML #188

[jan-david-fischbach]

@rowanc1 To implement a feature on this list we need to simultaneously consider LaTeX and the Web output, correct? How could we go about sidebar/margin content? Using \minipage or similar does not really float my boat. At the moment I would be in favor of adding the sidebar content to the appendix, or to add it below the current paragraph as a content box.
What do you think?

[rowanc1]

It would certainly be appreciated to consider both html, latex and other exports, however, we can take contributions that get us there in a few iterations. The default falls back to rendering the child content, so it is just inline in the document. I think this might depend on what template you use and as you suggest, there isn't really a good default in latex!

If you are interested in implementing one of these features, can you open a specific issue on it? We can help give some guidance on where to look and how to get started!!

[jan-david-fischbach]

Maybe I'll find some time to work on the sidebar soon. Trying to come up with a workaround I noticed that html tags are not left unchanged:

The myst-spec doesn't seem to say anything on how raw html is to be treated. The cheatsheet (which seems to be jb-flavoured myst) seems to indicate that html is interpreted as is.

Edit:
It seems that one can set some allowDangerousHtml flag somewhere. I just don't know where yet

[rowanc1]

I have opened and linked #516 with a few pointers to start, just ping me if you start working on it and get stuck and I can try to help out fast!

There is an outstanding bug for the HTML here:

• Inline HTML tags are parsed separately and produce semantically incorrect AST node #418

It works fine when the round-trip is HTML (e.g. using the underlying js libraries), but our renderer is react, so we actually need the AST of the HTML correct, which is currently incorrect and not nested properly. For example, this AST of bold is [html, text, html] rather than [html > text] or converted to [strong > text] in this simple case. This is still going to be a bit of work to sort through.

[jan-david-fischbach]

Even with the "wrong" AST¹ one seems to produce correct html with the allowDangerousHtml option:

import { u } from 'unist-builder';
import { mystToHtml } from 'myst-to-html';

let ast = u('root', [u('html', '<b>'), u('text', 'test'), u('html', '</b>')]);

const skewed_html = mystToHtml(ast);
console.log(skewed_html); // test

const html = mystToHtml(ast, {'hast': {'allowDangerousHtml': true}, 'stringifyHtml': {'allowDangerousHtml': true}});
console.log(html); // <b>test</b>

Footnotes

1. The correct handling of html doesn't seem to be firmly defined: Does it all go into one html node, or using children for the content. How is markdown syntax inside html handeled etc. ↩

[rowanc1]

Yes, it will work for explicit HTML outputs which is string concatenation! However most of the tools are built on React, which operates on the AST and will try to create <b> as HTML, then 'test' text, then </b>. Which will end up with <b></b>test as the first node closes by default and the closing node isn't rendered. I think it is a bit worse actually as it looks like there is an extra paragraph, <b>test</b> turns into:

Not great, and something to fix up soon (#418)!