Merge Develop into Main for 0.5.0 (#427)

* run integration tests with 2 workers * Fix container name * Make modelica use datetimes instead of floats * fix simulation tests * add tests for multiple workers * add scaling tests to ci tests * add scale to pytest markers * add aliases to web * Add REST API endpoints required to move client to REST. Also consolidated Run and Point Classes and Models. And other changes... * fix integration tests * fix tests * fix ci tests * fix and consolidate worker simulation tests * fix site ui not displaying * re-add mongoengine connect * Small fixes * revert change * lock versions of dependent docker containers (#393) * lock versions of dependent docker containers * precommit gitlab -> github * change print statement to use logger * fix datetime print * Upgrade dependencies, use python virtualenv (#394) * use tilde for versioning * update versions of precommit * npm update * install assulimo and pyfmi in venv * fix start worker location * remove tox pieces since we are no longer using it * Bump setuptools from 65.3.0 to 65.5.1 (#396) Bumps [setuptools](https://github.com/pypa/setuptools) from 65.3.0 to 65.5.1. - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/CHANGES.rst) - [Commits](pypa/setuptools@v65.3.0...v65.5.1) --- updated-dependencies: - dependency-name: setuptools dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * add retry if alfalfa gets bad response from e+. This can possibly happen in cases of resource constraints * Bump luxon from 3.2.0 to 3.2.1 in /alfalfa_web Bumps [luxon](https://github.com/moment/luxon) from 3.2.0 to 3.2.1. - [Release notes](https://github.com/moment/luxon/releases) - [Changelog](https://github.com/moment/luxon/blob/master/CHANGELOG.md) - [Commits](moment/luxon@3.2.0...3.2.1) --- updated-dependencies: - dependency-name: luxon dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * extend timeout for e+ simulation to start and remove workaround (#399) * limit worker dockerfile to copying worker files * remove annual_run job * allow merging of osws without measures * add minimal resstock osw and consolidate basic model testing * fix scaling test * add aliases to Alfalfa * point to client with alias support * Ternary tweak * wait for alias to be written to database * Dependency update and cleanup * Update client branch * Overhaul Point Generation (#406) * overhaul energyplus point generation * add site meter sensors * remove debug print in variables.py * fix ems output variable in ems measures * push updated poetry config * update lock file * GraphQL Removal, Bug Fixes, & Improvements (#415) * - Migrated to AWS SDK v3 as v2 is being deprecated - Fixed broken `mc policy` usage - Fixed potential createModel race condition - Fixed writeInputPoint bugs - Added missing context argument to runSimResolver - Fixed deprecated `collection.insert` call - Fixed docs generation - Zero npm vulnerabilities! * GraphQL removal and major refactor * Fixed FMU simulations in UI * Add ErrorDialog * Add API Documentation to Main Menu (#422) * Added docs to main menu * Fixed getSignedUrl, improved watch * Redocly update * Bump versions to 0.5.0 (#425) * bump versions to 0.5.0 * Bump web to 0.5.0 --------- Co-authored-by: Alex Swindler <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Nicholas Long <[email protected]> Co-authored-by: Anya Petersen <[email protected]> Co-authored-by: Alex Swindler <[email protected]> Co-authored-by: Nicholas Long <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
NREL · Mar 28, 2023 · c91f054 · c91f054
1 parent 055e493
commit c91f054
Show file tree

Hide file tree

Showing 119 changed files with 71,863 additions and 32,724 deletions.
diff --git a/.dockerignore b/.dockerignore
@@ -1,11 +1,9 @@
 __pycache__
 .pytest_cache
-.tox
 .vscode
 .coverage
 dist
 docs
-tox.ini
 Dockerfile
 
 alfalfa_web/Dockerfile

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -22,7 +22,7 @@ jobs:
           python-version: "3.8"
 
       - name: Run pre-commit
-        uses: pre-commit/action@v2.0.0
+        uses: pre-commit/action@v3.0.0
         with:
           extra_args: --all-files
 
@@ -34,10 +34,6 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@v2
 
-      - name: Start MongoDB
-        uses: supercharge/[email protected]
-        # Defaults to the latest version of MongoDB
-
       - name: Install Python
         uses: actions/setup-python@v2
         with:
@@ -49,7 +45,7 @@ jobs:
           # use a newer version of poetry in this test, but
           # poetry in the dockerfile is a bit older because
           # of a urllib3 issue
-          poetry-version: 1.2.1
+          poetry-version: 1.3.1
 
       - name: Install poetry dependencies
         run: |
@@ -60,6 +56,11 @@ jobs:
           cat /etc/issue
           python --version
 
+      - name: Start dependencies
+        run: |
+          printenv
+          docker compose up -d mongo redis minio mc goaws
+
       - name: Run tests with pytest
         run: |
           poetry run pytest
@@ -80,7 +81,7 @@ jobs:
           GIT_COMMIT: ${{ github.sha }}
         run: |
           printenv
-          docker-compose -f docker-compose.yml -f docker-compose.dev.yml up --build -d worker
+          docker-compose -f docker-compose.yml -f docker-compose.dev.yml up --build -d worker mongo redis minio mc goaws
 
       - name: Wait another 30s for the containers to start
         uses: jakejarvis/wait-action@master
@@ -90,13 +91,9 @@ jobs:
       - name: Dump docker logs before tests
         uses: jwalton/gh-docker-logs@v1
 
-      - name: Run modelica job tests in Docker worker container
-        run: |
-          docker exec alfalfa_worker_1 bash -c "cd /alfalfa && pytest -m docker tests/jobs/modelica -o log_cli=true -o log_level=info"
-
-      - name: Run openstudio job tests in Docker worker container
+      - name: Run job tests in Docker worker container
         run: |
-          docker exec alfalfa_worker_1 bash -c "cd /alfalfa && pytest -m docker tests/jobs/openstudio -o log_cli=true -o log_level=info"
+          docker exec alfalfa_worker_1 bash -c "cd /alfalfa && pytest -m docker tests/jobs -o log_cli=true -o log_level=info"
 
       - name: Dump docker logs on failure
         if: failure()
@@ -121,14 +118,14 @@ jobs:
           # use a newer version of poetry in this test, but
           # poetry in the dockerfile is a bit older because
           # of a urllib3 issue
-          poetry-version: 1.2.1
+          poetry-version: 1.3.1
 
       - name: Build and run stack
         env:
           GIT_COMMIT: ${{ github.sha }}
         run: |
           printenv
-          docker-compose up --build -d
+          docker-compose up --build -d --scale worker=2
 
       - name: Wait for web server
         uses: nev7n/wait_for_response@v1
@@ -151,6 +148,11 @@ jobs:
           poetry install
           poetry run pytest --timeout=600 -m "integration"
 
+      - name: Run worker scale tests with pytest
+        run: |
+          poetry install
+          poetry run pytest -m "scale"
+
       - name: Dump docker logs on failure
         if: failure()
         uses: jwalton/gh-docker-logs@v1
@@ -174,15 +176,15 @@ jobs:
           # use a newer version of poetry in this test, but
           # poetry in the dockerfile is a bit older because
           # of a urllib3 issue
-          poetry-version: 1.2.1
+          poetry-version: 1.3.1
 
       - name: Build and run stack
         env:
           GIT_COMMIT: ${{ github.sha }}
         run: |
           export HISTORIAN_ENABLE=true
           printenv
-          docker-compose -f docker-compose.yml -f docker-compose.historian.yml up --build -d
+          docker-compose -f docker-compose.yml -f docker-compose.historian.yml up --build -d --scale worker=2
 
       - name: Wait for web server
         uses: nev7n/wait_for_response@v1
@@ -205,6 +207,11 @@ jobs:
           poetry install
           poetry run pytest --timeout=600 -m "integration"
 
+      - name: Run worker scale tests with pytest
+        run: |
+          poetry install
+          poetry run pytest -m "scale"
+
       - name: Dump docker logs on failure
         if: failure()
         uses: jwalton/gh-docker-logs@v1

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -11,7 +11,7 @@ exclude: |
 
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.3.0
+    rev: v4.4.0
     hooks:
       - id: trailing-whitespace
       - id: check-added-large-files
@@ -27,7 +27,7 @@ repos:
       - id: mixed-line-ending
         args: ["--fix=auto"]
   - repo: https://github.com/pre-commit/mirrors-autopep8
-    rev: v1.6.0
+    rev: v2.0.1
     hooks:
       - id: autopep8
         args:
@@ -39,8 +39,8 @@ repos:
             "--max-line-length=100",
             "--ignore=E501,E402,W503,W504,E731"
           ]
-  - repo: https://github.com/humitos/mirrors-autoflake.git
-    rev: v1.3
+  - repo: https://github.com/PyCQA/autoflake
+    rev: v2.0.0
     hooks:
       - id: autoflake
         args:
@@ -51,13 +51,13 @@ repos:
             "--remove-unused-variable",
             "--ignore-init-module-imports"
           ]
-  - repo: https://gitlab.com/pycqa/flake8
-    rev: 4.0.1
+  - repo: https://github.com/pycqa/flake8
+    rev: 6.0.0
     hooks:
       - id: flake8
         args: ["--ignore=E501,E402,W503,W504,E731,F401"]
   - repo: https://github.com/pre-commit/mirrors-prettier
-    rev: v2.7.1
+    rev: v3.0.0-alpha.4
     hooks:
       - id: prettier
         types_or: [css, yaml, markdown, html, scss, javascript]

diff --git a/alfalfa_web/.babelrc.js b/alfalfa_web/.babelrc.js
@@ -1,4 +1,4 @@
-const fs = require("fs");
+const fs = require("node:fs");
 if (fs.existsSync("../.env")) require("dotenv").config({ path: "../.env" });
 
 const isProd = process.env.NODE_ENV === "production";
@@ -8,7 +8,7 @@ const presets = [
   [
     "@babel/preset-env",
     {
-      corejs: "3.21",
+      corejs: "3.29",
       useBuiltIns: "usage"
     }
   ]

diff --git a/alfalfa_web/Dockerfile b/alfalfa_web/Dockerfile
@@ -23,9 +23,6 @@ FROM base AS dev
 # The docker-compose will require the mounting
 # of the files into the container, see docker-compose.dev.yml
 
-# install nodemon to restart the service when the files change
-RUN npm install -g nodemon
-
 WORKDIR /srv/alfalfa
 
 # Run in watch mode so it will automatically restart

diff --git a/alfalfa_web/alfalfa.svg b/alfalfa_web/alfalfa.svg
diff --git a/alfalfa_web/app.js b/alfalfa_web/app.js
diff --git a/alfalfa_web/components/App/App.js b/alfalfa_web/components/App/App.js
@@ -1,79 +1,63 @@
+import React from "react";
 import { AdapterLuxon } from "@mui/x-date-pickers/AdapterLuxon";
 import { LocalizationProvider } from "@mui/x-date-pickers/LocalizationProvider";
 import { AppBar, Grid, Toolbar, Typography } from "@mui/material";
 import { createTheme, ThemeProvider } from "@mui/material/styles";
-import { withStyles } from "@mui/styles";
-import PropTypes from "prop-types";
-import React from "react";
 import { Link, Navigate, Route, Routes } from "react-router-dom";
-import Sims from "../Sims/Sims";
-import Sites from "../Sites/Sites";
-import Upload from "../Upload/Upload";
+import { Docs } from "../Docs/Docs";
+import { Sims } from "../Sims/Sims";
+import { Sites } from "../Sites/Sites";
+import { Upload } from "../Upload/Upload";
+import styles from "./App.scss";
 
 const theme = createTheme();
 
-const styles = {
-  root: {
-    display: "flex",
-    flexDirection: "column",
-    minHeight: "100vh"
-  },
-  title: {
-    flex: 1
-  },
-  button: {
-    margin: `${theme.spacing(1)}!important`
-  }
-};
-
-class App extends React.Component {
-  render = () => {
-    const { classes } = this.props;
-
-    return (
-      <ThemeProvider theme={theme}>
-        <LocalizationProvider dateAdapter={AdapterLuxon}>
-          <div className={this.props.classes.root}>
-            <AppBar position="static">
-              <Toolbar>
-                <Link to={"/"} className={this.props.classes.title} style={{ textDecoration: "none", color: "unset" }}>
-                  <Typography variant="h5" color="inherit">
-                    Alfalfa
-                  </Typography>
-                </Link>
-                <Grid container justifyContent="flex-end" spacing={2} style={{ marginLeft: 0 }}>
-                  <Grid item>
-                    <Link to={"/sites"} style={{ textDecoration: "none", color: "unset" }}>
-                      <Typography className={classes.button} variant="button" color="inherit">
-                        Sites
-                      </Typography>
-                    </Link>
-                  </Grid>
-                  <Grid item>
-                    <Link to={"/sims"} style={{ textDecoration: "none", color: "unset" }}>
-                      <Typography className={classes.button} variant="button" color="inherit">
-                        Completed-Simulations
-                      </Typography>
-                    </Link>
-                  </Grid>
+export const App = () => {
+  return (
+    <ThemeProvider theme={theme}>
+      <LocalizationProvider dateAdapter={AdapterLuxon}>
+        <div className={styles.root}>
+          <AppBar position="static" sx={{ zIndex: 1 }}>
+            <Toolbar>
+              <Link to={"/"} className={styles.title} style={{ textDecoration: "none", color: "unset" }}>
+                <Typography variant="h5" color="inherit">
+                  Alfalfa
+                </Typography>
+              </Link>
+              <Grid container justifyContent="flex-end" spacing={2} style={{ marginLeft: 0 }}>
+                <Grid item>
+                  <Link to={"/sites"} style={{ textDecoration: "none", color: "unset" }}>
+                    <Typography variant="button" color="inherit" sx={{ m: 1 }}>
+                      Sites
+                    </Typography>
+                  </Link>
                 </Grid>
-              </Toolbar>
-            </AppBar>
-            <Routes>
-              <Route path="/" element={<Upload />} />
-              <Route path="/sites" element={<Sites />} />
-              <Route path="/sims" element={<Sims />} />
-              <Route path="*" element={<Navigate to="/" />} />
-            </Routes>
-          </div>
-        </LocalizationProvider>
-      </ThemeProvider>
-    );
-  };
-}
-
-App.propTypes = {
-  classes: PropTypes.object.isRequired
+                <Grid item>
+                  <Link to={"/sims"} style={{ textDecoration: "none", color: "unset" }}>
+                    <Typography variant="button" color="inherit" sx={{ m: 1 }}>
+                      Completed Simulations
+                    </Typography>
+                  </Link>
+                </Grid>
+                <Grid item>
+                  <Link to={"/docs"} style={{ textDecoration: "none", color: "unset" }}>
+                    <Typography variant="button" color="inherit" sx={{ m: 1 }}>
+                      API Docs
+                    </Typography>
+                  </Link>
+                </Grid>
+              </Grid>
+            </Toolbar>
+          </AppBar>
+          <Routes>
+            <Route path="/" element={<Upload />} />
+            <Route path="/sites" element={<Sites />} />
+            <Route path="/sims" element={<Sims />} />
+            <Route path="/docs" element={<Docs />} />
+            <Route path="*" element={<Navigate to="/" />} />
+          </Routes>
+        </div>
+      </LocalizationProvider>
+    </ThemeProvider>
+  );
 };
-
-export default withStyles(styles)(App);
diff --git a/alfalfa_web/components/App/App.scss b/alfalfa_web/components/App/App.scss
@@ -0,0 +1,9 @@
+.root {
+  display: flex;
+  flex-direction: column;
+  min-height: 100vh;
+}
+
+.title {
+  flex: 1;
+}
diff --git a/alfalfa_web/components/Docs/Docs.js b/alfalfa_web/components/Docs/Docs.js
@@ -0,0 +1,9 @@
+import React from "react";
+
+export const Docs = () => {
+  return (
+    <div style={{ display: "flex", flex: 1 }}>
+      <iframe src="/redoc" style={{ border: 0, width: "100vw" }}></iframe>
+    </div>
+  );
+};