Add raredisease scout upload

[rannick]

Description

Added

• scout upload for raredisease

Changed

• unified scout upload for balsamic, rnafusion, raredisease and mip-dna using inherited functions

Fixed

How to prepare for test

• Ssh to relevant server (depending on type of change)
• Use stage: us
• Paxa the environment: paxa
• Install on stage (example for Hasta):

  bash /home/proj/production/servers/resources/hasta.scilifelab.se/update-tool-stage.sh -e S_cg -t cg -b [THIS-BRANCH-NAME] -a

How to test

• Do ...

Expected test outcome

• Check that ...
• Take a screenshot and attach or copy/paste the output.

Review

• Tests executed by
• "Merge and deploy" approved by
  Thanks for filling in who performed the code review and the test!

This version is a

• MAJOR - when you make incompatible API changes
• MINOR - when you add functionality in a backwards compatible manner
• PATCH - when you make backwards compatible bug fixes or documentation/instructions

Implementation Plan

• Document in ...
• Deploy this branch on ...
• Inform to ...

[Vince-janv]

Rgarding the config building I think we can still improve the readability by not reassigning the variable and also removing the self.load_config field. Also I'm a bit hesitant to add a Store object to all the config builders

[rannick]

I have to say I am really unsure about what exactly you would like to see with removing the self.load_config attribute and not returningload_config. To me it seems impossible to avoid returning if we are not placing the information in self.load_config. I think I am misinterpreting something

[Vince-janv]

Great job @rannick! Thanks for cleaning up a lot of this for the other pipelines as well 🥇

[sonarqubecloud]

Quality Gate failed

Failed conditions
4.6% Duplication on New Code (required ≤ 3%)

See analysis details on SonarCloud

[rannick]

Log deploy... done.
cg, version 64.1.0
remote: Enumerating objects: 576, done.
remote: Counting objects: 100% (559/559), done.
remote: Compressing objects: 100% (250/250), done.
remote: Total 576 (delta 385), reused 442 (delta 303), pack-reused 17 (from 1)
Receiving objects: 100% (576/576), 165.73 KiB | 0 bytes/s, done.
Resolving deltas: 100% (387/387), completed with 71 local objects.
From https://github.com/Clinical-Genomics/cg
   b1cf2ee..9edc3c9  master     -> origin/master
   65914eb..edae9e1  2396-update-latest-archivedf-lowcell -> origin/2396-update-latest-archivedf-lowcell
 * [new branch]      3714-validating-sample-sex-in-add-relationship -> origin/3714-validating-sample-sex-in-add-relationship
   ac5d9d0..e5d6d40  3843-same-file-archived-to-pdc-the-same-day -> origin/3843-same-file-archived-to-pdc-the-same-day
 * [new branch]      add-mutant-upload-api -> origin/add-mutant-upload-api
 * [new branch]      add-ordertype-endpoint -> origin/add-ordertype-endpoint
   48d4850..88f638c  balsamic_v16 -> origin/balsamic_v16
   6cddd76..89af5ef  ordertype-application-creation -> origin/ordertype-application-creation
 * [new branch]      patch-balsamic-uploads -> origin/patch-balsamic-uploads
 * [new branch]      rank_model_to_rd_scout -> origin/rank_model_to_rd_scout
 * [new tag]         v64.1.0    -> v64.1.0
From https://github.com/Clinical-Genomics/cg
 * [new tag]         v64.0.3    -> v64.0.3
 * [new tag]         v64.0.4    -> v64.0.4
Already on 'master'
Your branch is behind 'origin/master' by 6 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)
Updating b1cf2ee..9edc3c9
Fast-forward
 .bumpversion.cfg                                                                 |   2 +-
 cg/__init__.py                                                                   |   2 +-
 cg/cli/deliver/base.py                                                           |  48 +++++++++++++++++++
 cg/cli/upload/scout.py                                                           |   6 ++-
 cg/constants/scout.py                                                            |  40 ++++++++++++++--
 cg/meta/upload/raredisease/raredisease.py                                        |   8 ++++
 cg/meta/upload/scout/balsamic_config_builder.py                                  |  41 ++++++++--------
 cg/meta/upload/scout/balsamic_umi_config_builder.py                              |  30 ++++++++++--
 cg/meta/upload/scout/hk_tags.py                                                  |  16 +++++++
 cg/meta/upload/scout/mip_config_builder.py                                       | 148 ++++++++++++++++-----------------------------------------
 cg/meta/upload/scout/raredisease_config_builder.py                               | 141 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 cg/meta/upload/scout/rnafusion_config_builder.py                                 |  60 ++++++++++++------------
 cg/meta/upload/scout/scout_config_builder.py                                     | 166 +++++++++++++++++++++++++++++++++++++++++++++++-----------------
 cg/meta/upload/scout/uploadscoutapi.py                                           |  52 ++++++++++----------
 cg/meta/workflow/nf_analysis.py                                                  |   2 +-
 cg/meta/workflow/utils/{get_genome_build.py => genome_build_helpers.py}          |  14 ++++++
 cg/models/scout/scout_load_config.py                                             |  56 ++++++++++++++++++++--
 cg/resources/raredisease_bundle_filenames.yaml                                   | 100 +++++++++++++++++++--------------------
 cg/services/deliver_files/deliver_files_service/deliver_files_service.py         |  25 ++++++++++
 cg/services/deliver_files/deliver_files_service/deliver_files_service_factory.py |   2 +
 cg/services/deliver_files/file_filter/abstract.py                                |  10 ++++
 cg/services/deliver_files/file_filter/sample_service.py                          |  11 +++++
 cg/store/crud/create.py                                                          |   4 ++
 pyproject.toml                                                                   |   2 +-
 tests/conftest.py                                                                |  10 +++-
 tests/meta/upload/scout/conftest.py                                              | 177 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 tests/meta/upload/scout/test_generate_load_config.py                             | 110 ++++++++++++++++++++++++++++++++++++-------
 tests/meta/upload/scout/test_meta_upload_scoutapi.py                             |  19 ++++----
 tests/meta/upload/scout/test_meta_upload_scoutapi_rna.py                         |   2 +-
 tests/meta/upload/scout/test_scout_config_builder.py                             |  90 ++++++++++++++++++++++-------------
 tests/services/file_delivery/delivery_file_service/test_service.py               |   1 +
 tests/services/file_delivery/file_filter/test_sample_filter_service.py           |  22 +++++++++
 32 files changed, 1054 insertions(+), 363 deletions(-)
 create mode 100644 cg/meta/upload/scout/raredisease_config_builder.py
 rename cg/meta/workflow/utils/{get_genome_build.py => genome_build_helpers.py} (59%)
 create mode 100644 cg/services/deliver_files/file_filter/abstract.py
 create mode 100644 cg/services/deliver_files/file_filter/sample_service.py
 create mode 100644 tests/services/file_delivery/file_filter/test_sample_filter_service.py
/home/hiseq.clinical
INFO  [alembic.runtime.migration] Context impl MySQLImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.