Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ocrd_network base integration tests #1164

Merged
merged 96 commits into from
Jan 30, 2024
Merged

ocrd_network base integration tests #1164

merged 96 commits into from
Jan 30, 2024

Conversation

MehmedGIT
Copy link
Contributor

@MehmedGIT MehmedGIT commented Jan 10, 2024

This PR is just a basic beginning for the ocrd_network integration tests.

What does this PR include:

  • Some extensions to MongoDB methods in ocrd_network for easier testing of the DB functionalities
  • New state UNSET for jobs for easier testing instead of relying on None
  • New endpoints to the Processing Server:
    • endpoint / for just testing the connectivity to the Processing Server
    • endpoint /workflow/job-simple/{workflow_job_id} - simplified endpoint for checking the overall status of a workflow
  • Processing Server adds ocrd-dummy processor and queue by default when started:
    • This is a hack since there is no endpoint for worker registration yet
    • This will be refactored in the next PR when refactoring the ocrd_network and adding a dedicated endpoint
  • A conftest.py provided for using fixtures for the MongoDB and RabbitMQ
  • MongoDB integration tests
  • RabbitMQ integration tests
  • A dummy workflow consisting of 3 ocrd-dummy processors
  • Basic Processing Server integration tests:
    • processing endpoint test with ocrd-dummy (1 processing worker deployed)
    • workflow endpoint execution and status check test with the dummy workflow
  • Separate ocrd_logging.conf for the ocrd_network integration tests for easier catching of bugs when tests are failing
  • Fixing .dockerignore to include relevant parts for the tests
  • Extension to core Dockerfile for creating an ocrd_core_test image to be used by the testing containers
  • Extension to core Makefile to allow make integration-test call
  • Integration test workflow for Github Actions.

@MehmedGIT MehmedGIT marked this pull request as ready for review January 30, 2024 10:59
Copy link
Member

@kba kba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent, great work. a few minor questions, but ready for merge AFAICS. I'm running the tests and making sure that the multi-stage setup does not interfere with our other Docker deployments.

macos: I'll try to get it to work but it's not a blocker.

tests/conftest.py Outdated Show resolved Hide resolved
Comment on lines +21 to +25
healthcheck:
test: echo 'db.runCommand("ping").ok' | mongosh localhost:27017/test --quiet
interval: 1s
timeout: 3s
retries: 30
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, that is a useful pattern we should apply widely.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that works, I prefer using that instead of sleeping here and there to wait. So far I have not seen issues with that health check here and in Operandi.

tests/network/docker-compose.yml Outdated Show resolved Hide resolved
Copy link
Contributor

@joschrew joschrew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This are my first comments after I read through the changed source-code. I still have to find out why the tests fail on my side and then I will comment again.

Dockerfile Show resolved Hide resolved
tests/network/test_rabbitmq.py Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
@MehmedGIT
Copy link
Contributor Author

Good to be shipped.

@kba kba merged commit b0739dd into master Jan 30, 2024
22 checks passed
@kba kba deleted the network-tests branch January 30, 2024 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants