bgcflow_roadmap

[matinnuhamunada]

Some crucial features to add or fix:

Overlapping ids between project should not be an issue

• Currently, it is hardcoded to make the run fail when two projects have overlapping genome id
• Theoretically, this should not be an issue as snakemake will trace the filenames, so there should not be an ambiguous files/output
• The challenge here is to make sure that all genome ids refer to the same genome entity: same annotation, same taxonomic placement
• I think using python classes might help solve this issue. Another approach is to use mysql intermediate to store the project metadata

Adopt PEP for managing projects

• BGCFlow aims to manage running multiple genome mining projects. I did not aware of the Portable Encapsulated Projects
• The solution I came up seems to be already addressed by PEP, therefore it is better to adopt their style

Containerization with Singularity

• For reproducibility, I think it is better to use both conda and singularity to pack environments
• https://snakemake.readthedocs.io/en/stable/snakefiles/deployment.html#ad-hoc-combination-of-conda-package-management-with-containers

Storage efficiency

• BGCFlow currently generate some redundant intermediate files.
• This should be identified and cleaned to make sure we use storage efficiently

Wrap BGCFlow scripts as a python package

• This is the idea at the beginning and should be followed up

BGCFlow wrapper for snakemake

• As there are more functionalities and rules, it will be nice to build a snakemake CLI Wrapper and BGCFlow functionalities to aid users

Ability to choose input types

• Taking into account different cases so far, I think it would be useful to make BGCFlow to accept two types of input: (1) .fna, and (2) .gbk.

Customize output folders

• There should be a feature to tell BGCFlow the path to the data folder

Customize log folders

• Log files should be at the root of BGCFlow or customized