Initial Framework Setup

[Taepper]

resolves #3

This is a first version for the framework we envisioned.

The general idea: The test runner automatically detects all testset folders and starts a corresponding

-> This makes the barrier to add new tests very low. Just copy paste an old directory or other data that is found and no need to adjust any config file

For every testset all queries are executed. I know that in #1 it was wished not to be defined in json, but this was closer to what we also did similar in SILO so I started with this and hoped we could iterate on it afterwards.

The queries must be able to define their result in external files, which is currently also the only version of specifying query results.

There are still some things that do not quite work well. The docker-compose file and thus the connection between SILO and LAPIS is sometimes broken and LAPIS throws some Java Exception when a request is received (related to GenSpectrum/LAPIS#934):

{
  "error": {
    "type": "about:blank",
    "title": "Internal Server Error",
    "status": 500,
    "detail": "Could not connect to silo: class java.net.ConnectException null"
  },
  "info": {
    "dataVersion": null,
    "requestId": "1685ba49-6832-40d7-b0e3-605ecfabee36",
    "requestInfo": "sars_cov-2_minimal_test_config on localhost at 2024-09-03T16:32:58.448522382",
    "reportTo": "Please report to https://github.com/GenSpectrum/LAPIS/issues in case you encounter any unexpected issues. Please include the request ID and the requestInfo in your report."
  }
}

[fengelniederhammer]

resolves #1

Does it already resolve the whole issue? Or should we rather split it into smaller subtasks?

[Taepper]

No, I will change #1 to an epic and add issues that I can link to here

[corneliusroemer]

Great work. A few comments:

• You could try to get ChatGPT to turn your .json files into .ts and change the test steps as well - I feel like this is a task that ChatGPT could excel at
• Please always add a README with instructions how to run stuff, otherwise others cannot reproduce/test/help
• It didn't work locally for me, see Failure at step execSync(dockerComposeUpCommand, { stdio: 'inherit' }); when running locally #8