Smoke tests for Hugo, Docusaurus, etc.

[eitsupi]

At this time, no Hugo or Docusaurus format testing is being done in this repository.
I believe this was one of the reasons for the serious regression in the 1.3 Docusaurus format (#5079).

I believe there has been discussion of adding tests for these formats (e.g., comment #5087), but it has not been possible due to additional dependencies (e.g., Docusaurus itself).

However, for Hugo and Docusaurus formats, the quarto render command only does markdown output, and I believe the functionality used in tests is not actually dependent on Hugo or Docusaurus.
So I think something like snapshot tests can be performed only with Quarto CLI itself, how about that?

(I think it would be easier for less experienced people like myself to make contributions to the code base if something like snapshot tests were in place.)

[cscheid]

This is a totally reasonable request! We made some progress in improving our test suite in 1.3, but there's plenty of room for improvement.

WRT docusaurus testing specifically, the "easy path" is annoying because it would require checking in the entirety of the NPM dependencies for docusaurus. I don't think that's great for folks who never need/want to test docusaurus. I think the solution would be for docusaurus tests to live in a separate test folder with its own setup harness (so we would npx install docusaurus or whatever on the top level of the documents that need testing before running the tests themselves).

[eitsupi]

I agree that setting up Docusaurus is a pain.

However, I believe that here there is little need to check the files that Docusaurus generates and that it is mostly sufficient to check the md files that Quarto generates independently of Docusaurus.

Also, it seems that npm is already used as a dependency for testing on CI.

[allenmanning]

Thank you for posting this @eitsupi - I've now moved this to an issue for us to track. It has my +1.

#5309

[eitsupi]

Thanks, I will close this.