test: Implement end-to-end testing with Playwright

.github/pull_request_template.md

@@ -1,4 +1,10 @@
-By submitting this PR, I am indicating to the Numberscope maintainers that I have read and understood the [contributing guidelines](https://numberscope.colorado.edu/doc/CONTRIBUTING/) and that this PR follows those guidelines to the best of my knowledge. I have also read the [pull request checklist](https://numberscope.colorado.edu/doc/doc/pull-request-checklist/) and followed the instructions therein.
+By submitting this PR, I am indicating to the Numberscope maintainers that I
+have read and understood the
+[contributing guidelines](https://numberscope.colorado.edu/doc/CONTRIBUTING/)
+and that this PR follows those guidelines to the best of my knowledge. I have
+also read the
+[pull request checklist](https://numberscope.colorado.edu/doc/doc/pull-request-checklist/)
+and followed the instructions therein.
 
 <hr/>
 

.github/workflows/ci.yaml

@@ -1,5 +1,10 @@
 name: 'CI'
-on: pull_request
+on:
+    push:
+        branches:
+            - main
+    pull_request:
+
 jobs:
     lint:
         runs-on: ubuntu-22.04
@@ -12,13 +17,15 @@ jobs:
             - name: Set up Node.js
               uses: actions/setup-node@v3
               with:
-                  node-version: '18.x'
+                  node-version: '20.x'
             - name: Install dependencies
               run: npm install
             - name: Lint
               run: npm run lint:check
+            - name: Type check
+              run: npm run typecheck
     test:
-        runs-on: ubuntu-22.04
+        runs-on: ubuntu-latest
         steps:
             - uses: actions/checkout@v3
             - name: Set up Python
@@ -28,8 +35,20 @@ jobs:
             - name: Set up Node.js
               uses: actions/setup-node@v3
               with:
-                  node-version: '18.x'
+                  node-version: '20.x'
             - name: Install dependencies
               run: npm install
-            - name: Test
+            - name: Unit test
               run: npm run test:unit
+            - name: Build production site
+              run: npm run build
+            - name: Install Playwright browsers
+              run: npx playwright install --with-deps
+            - name: End-to-end test
+              run: npx playwright test -c e2e/playci.config.ts
+### Uncomment when you need to re-extract snapshots
+#            - name: Extract snapshots
+#              uses: actions/upload-artifact@v4
+#              with:
+#                  name: updated_snapshots
+#                  path: e2e/tests/ci_snaps

.gitignore

@@ -6,12 +6,22 @@ npm-debug.log*
 # Subdirectories generated by various tools
 .venv
 coverage
-dist
 node_modules
-site
 
 # File for storing your local configuration
 *.local
 
 # Emacs backup files
 *~
+
+# Build files
+dist
+etc/npm_install_ran_at
+etc/typechecked_at
+site
+
+# End-to-end testing
+e2e/.last_cl_args
+e2e/certificate
+e2e/docker_image_built
+e2e/results

.husky/pre-commit

@@ -1,8 +1,8 @@
 #!/bin/sh
 . "$(dirname "$0")/_/husky.sh"
 
-npm run lint:list || echo 'Files listed above will be reformatted and staged.'
-npm run lint:staged
+node tools/cleancommit.js
 npm run lint:check
-npm run typecheck
 npm run test:unit
+npm run test:e2e
+

.prettierignore

@@ -8,15 +8,18 @@
 *.generator
 *.local
 *~
+.env
+.env.*
+.github/workflows
 .gitignore
 .husky
 .prettierignore
 .vscode
 coverage
 dist
+index.html
 node_modules
 package-lock.json
-index.html
+e2e/*-snapshots
+e2e/results
 site
-.env
-.env.*

CONTRIBUTING.md

@@ -24,12 +24,13 @@ applies to other repos as well.)
    [read about basic Git operations](./doc/working-with-git-and-github.md#basic-operations).
 5. [Push your branch to GitHub](./doc/working-with-git-and-github.md#push-a-branch).
 6. [Read Numberscope's coding principles guide](./doc/code-principles.md).
-7. [Read Numberscope's style guide](./doc/code-style.md). You should also
-   familiarize yourself with frontscope's internal code structure and APIs.
-   Documentation on these topics is generally incorporated into the relevant
-   source files (see [this example](src/shared/math.ts)). However, it is all
-   gathered in the "Internal code and APIs" section of the navigation bar in
-   the [online docs](https://numberscope.colorado.edu/doc).
+7. Familiarize yourself with frontscope's
+   [code organization](./doc/code-organization.md) and internal APIs. Some
+   information on the latter topic may only be found in comments in the
+   relevant sources files, as the documentation project is ongoing. However,
+   all such formal documentation currently being generated is gathered in the
+   "Internal code and APIs" section of the navigation bar in the
+   [online docs](https://numberscope.colorado.edu/doc).
 8. If you are working on a visualizer, read
    [the doc on making a visualizer](./doc/making-a-visualizer.md).
 9. [Work through Numberscope's pull request checklist](./doc/pull-request-checklist.md).
@@ -64,9 +65,9 @@ their factorizations, see `backscope`
 applies to other repos as well.)
 
 If you've read the
-[Numberscope code principles guide](./doc/code-principles.md) and the
-[Numberscope code style guide](./doc/code-style.md), and you think your code
-is ready to be reviewed by someone at Numberscope, follow these steps:
+[Numberscope code principles guide](./doc/code-principles.md) and have adhered
+to the [code organization](./doc/code-organization.md), and you think your
+code is ready to be reviewed by someone at Numberscope, follow these steps:
 
 1. Work through
    [Numberscope's pull request checklist](./doc/pull-request-checklist.md).

doc/behind-the-scenes.md

@@ -29,6 +29,9 @@ implement these interfaces yourself. That means including all the required
 properties and methods, and making sure they behave in the way the engine
 expects.
 
+And just as a reminder, only generate random numbers (if needed) using mathjs;
+see the [math documentation](../src/shared/math.md).
+
 ## Visualizers
 
 Source: `VisualizerInterface.ts`.

doc/code-organization.md

@@ -0,0 +1,84 @@
+# Code organization
+
+Here's a brief overview of the directory layout of this project, to help you
+find what you're looking for, and to guide you when you need to create new
+files.
+
+## Top level of project
+
+Here you will find the obligatory README.md and licensing/contributing
+information, project manifests like package.json and requirements.txt, and the
+top-level index.html for vite. There are also a few hidden (dotfile)
+configuration files, like git's own .git and .gitignore files, similar ones
+for github and Husky, and the .env files that control environment variables.
+Otherwise, we try to keep the top level uncluttered, and you should rarely
+need to create a file here.
+
+-   ### etc/ -- Configuration files
+
+    In particular, since frontscope uses so many tools, we have moved all of
+    their configuration files into etc/, as they were accumulating in the top
+    level. You should be able to find almost all configuration information
+    here, and any new tool that needs to be introduced should be invoked so as
+    to find its configuration file in the etc/ directory. As one example, the
+    Vite and Vitest configuration files are located here. As an exception, the
+    several tsconfig files for the TypeScript compiler are located in the
+    top-level directory itself -- we have not been able to get them to work in
+    any other location.
+
+-   ### src/ -- Source code
+
+    All of the actual TypeScript and Vue source code for the frontscope web
+    application is located here, along with assets that are used in-browser.
+    Only the top-level App.vue and main.ts are directly at the top level. The
+    rest are arranged in subdirectories:
+
+    -   #### src/assets/ and src/public/
+
+        Non-code assets needed when the site is deployed.
+
+    -   #### src/components/ -- Vue components
+
+    -   #### src/router/ -- Vue router
+
+    -   #### src/sequences/ -- Sequence implementations
+
+        These classes provide the integer sequences that can be analyzed with
+        Numberscope.
+
+    -   #### src/shared/
+
+        Code needed by multiple other directories is located here.
+
+    -   #### src/views/ -- Vue views selected by the router
+
+    -   #### visualizers/ and visualizers-workbench/
+
+        These directories contain the implementations of the various means of
+        turning a sequence of numbers into a graphical presentation. In some
+        sense, these are the heart of Numberscope. The workbench directory is
+        for Visualizers under development (together with a template
+        permanently kept there), and those in visualizers/ will automatically
+        appear in the user interface
+
+-   ### doc/ -- Documentation
+
+    This directory contains the bulk of the content for the integrated
+    documentation site. Some content is also extracted from the source code
+    and incorporated when the documentation site is built. The styling
+    information for the site is in etc/doc_theme/.
+
+-   ### e2e/ -- End-to-end in-browser testing
+
+    All files relating to end-to-end testing are located here. Configuration
+    files are directly in this folder (another minor exception to the rule
+    mentioned above that all configuration is in etc/).
+
+    -   #### e2e/tests/ -- End-to-end test specifications
+    -   #### e2e/results/ -- Files generated via testing.
+
+-   ### tools/ -- Internal utilities
+
+    Contains code for any custom utilities used for example in the building
+    and maintenance of frontscope. Nothing here should actually be needed by
+    the eventual web application.

doc/code-principles.md

@@ -28,3 +28,20 @@ Practically speaking this means:
     comments that merely repeat what the code already says
 -   only have code that actually does something -- don't include any commented
     out code or code that can't be reached or run
+-   minimize the number of return statements in a function
+    -   if there are conditions under which you can bail out easily, it's OK
+        to check for them near the top of the function and return from there
+    -   otherwise, strive to have all returns at or near the end of the
+        function
+
+## Code style/formatting
+
+We rely on automated tools (ESLint and Prettier as of this writing) to keep
+the layout/style of our code uniform for readability. Any new code will be
+required to conform in the commit/PR process. In fact, often the tools will be
+able to adjust the layout of the code for you: if a commit is rejected because
+of formatting non-compliance, try `npm run lint` -- just be aware that it will
+modify your source files. If you just want to see the issues without touching
+any of your files, use `npm run lint:check`. See
+[working with the package manager](working-with-pm.md) for details on these
+and other scripts you can run.