How to make HTML Output possible

[Guelakais]

As described for the tectonic -X compile command, html output should be possible in the current version of tectonic. However, I've tried it and it seems that it's not yet fully implemented or it needs heavy configuration to work fully. Does anyone have any experience with the html part of tectonic and knows what the key elements are to make it work? Is this part of tectonic still in development?
What is missing?
I've tried it myself, as you can see in the related issue.

[pkgw]

I'm the author and, as far as I know, sole user of the HTML support in its current form, so I'll do my best to describe the situation.

The support for HTML output in the engine is very much still in development. It's extremely low-level, and requires cooperation from the TeX code that you process in order to work correctly. So you can't just process a regular document and expect to get any output; the input needs to be designed for HTML processing. Hopefully one day there will be enough templates and pieces of support code in place so that regular documents can be processed in HTML mode to generate good results, but that's a ways off while the fundamentals are still being developed.

When the XeTeX engine is configured to use HTML output mode, the processing and outputs are nearly the same as in the standard mode. However, the output file is relabeled to be an "SPX" rather than "XDV" file, and that output is run through the spx2html engine rather than xdvipdfmx. SPX stands for "Semantically Paginated XDV", which is now a misnomer — my initial vision was to use the XDV pagination to separate the HTML output into different output files, but I have since concluded that that approach doesn't really work. The distinct filename is maintained since there are genuine differences in the output, the most notable one being that the paragraph processing routines are overridden to set each paragraph as one super-long line without linebreaking, so that the input text can be fed through to spx2html without hyphenations.

The spx2html stage uses the tera HTML templating engine. As the SPX file is processed, various \special items tell the templating engine to select a new template, to set various templating variables, and to write the currently buffered content to a named HTML file by populating the currently selected template. The typical error about being unable to open an input template occurs because at the end of the input, the engine wants to output the remaining content, but it doesn't know what template to use because the appropriate \special never occurred.

The magic about spx2html is that it has some special logic to render "canvases", which are things like equations where the positions of specific glyphs and rules are honored in detail. The engine computes magic CSS styles to position the glyphs correctly and also uses a technique that's basically like font subsetting to create special fonts on-the-fly that directly map various Unicode codepoints (the HTML text) to specific glyphs (particular symbols in font files). This is needed because for some typographic effects, especially when rendering mathematics, the XeTeX engine knows what glyphs need to be rendered but there isn't necessarily a way to emit a single Unicode codepoint (character) that will cause the browser to use that specific glyph. For instance, a font might contain a glyph for a superscript "2" character but simply including a "2" in the textual output can only ever cause a regular inline "2" to be rendered. This subsetting-like technique is the core innovation of the Tectonic HTML output mode. This vision is described in Williams, 2022 TUGboat 43(2) 120–126.

Anyway, there are two existing systems that use the HTML output mode: tt-weave, which translates the XeTeX literate programming source into an HTML book, and a prototype called tectonopedia that aims to one day be the Tectonic reference encyclopedia. And tt-weave needs updating to work with the latest iteration of the HTML output support in the engine. If you want to play around with the HTML output mode as things currently stand, you should read the source code of these tools, the spx2html engine, and the relevant customizations in the Tectonic version of the XeTeX engine. If you're not prepared to do that, just pretend that the HTML mode doesn't exist.

My goal is to flesh out the HTML support for the Tectonopedia use case. The key thing there is that after having worked on this topic for a while, I'm most interested in using TeX/Tectonic as a basis for online technical documentation that is much more sophisticated than a single static HTML page — in my opinion, proper HTML technical documentation should be viewed as a rich web application whose content just happens to be mostly text, and the way to explore that space is with a system specifically targeting that output format. Once the tools are developed to to manage the HTML output in that context, it's "just" a matter of developing a nice single-page template and adding hooks to the standard LaTeX classes to make it so that activating the HTML mode with a standard LaTeX document succeeds and produces a nice output.