Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge Haploflow into master #1015

Draft
wants to merge 41 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
18dcfe3
Use Haploflow instead of IVA
CBeelen Jul 23, 2021
4d505d5
Merge branch 'master' into Haploflow to prevent splitting contigs mat…
CBeelen Jul 30, 2021
8eb81be
Read in Haploflow's input arguments
CBeelen Aug 4, 2021
550bb55
Add debug mode to generate graphs
CBeelen Aug 4, 2021
b0a7e0a
Correct default error rate
CBeelen Aug 4, 2021
7960ef9
Trim and merge contigs after assembly using IVA's tools
CBeelen Aug 9, 2021
8a36f33
Add optional scaffolding and patching
CBeelen Aug 11, 2021
d8b1e46
Correct contig paths
CBeelen Aug 11, 2021
06c043f
Handle unsuccessful scaffolding or patching
CBeelen Aug 11, 2021
75cb275
Better handle unsuccessful scaffolding or patching
CBeelen Aug 11, 2021
bdf5704
Correct nucmer options
CBeelen Aug 12, 2021
89d5c2e
Add option for a second try at assembly with filtered reads
CBeelen Aug 12, 2021
30e34a5
Merge newest changes in master, and resolve conflicts
CBeelen Aug 18, 2021
7b75e64
Use information from remap.csv to separate reads
CBeelen Aug 19, 2021
cb5949b
Correct filename
CBeelen Aug 19, 2021
adf13f5
Add option for IVA assembly and modify scaffolding parameters
CBeelen Sep 1, 2021
667255d
Merge branch 'whole-genome-consensus' into Haploflow to create whole-…
CBeelen Sep 27, 2021
199e98c
Merge branch 'whole-genome-consensus' into Haploflow
CBeelen Sep 28, 2021
5e34e3e
Merge branch 'whole-genome-consensus' into Haploflow
CBeelen Sep 29, 2021
e5b50ee
Merge branch 'whole-genome-consensus' into Haploflow
CBeelen Sep 29, 2021
789cd9c
Merge branch 'whole-genome-consensus' into Haploflow
CBeelen Oct 14, 2021
79ccbfd
Merge branch 'master' into Haploflow
CBeelen Oct 20, 2021
b0e53d5
Merge branch 'master' into Haploflow
CBeelen Nov 1, 2021
4951a05
Merge branch 'master' into Haploflow
CBeelen Nov 3, 2021
d36d3bd
Merge branch 'Consensus_insertions' into Haploflow
CBeelen Nov 18, 2021
cdbfc3e
Merge branch 'Consensus_insertions' into Haploflow
CBeelen Nov 19, 2021
c076787
Merge branch 'Consensus_insertions' into Haploflow
CBeelen Nov 20, 2021
11a7f50
Merge branch 'Consensus_insertions' into Haploflow
CBeelen Dec 2, 2021
47613a9
Merge branch 'Consensus_insertions' into Haploflow
CBeelen Dec 3, 2021
e70d131
Merge branch 'Consensus_insertions' into Haploflow
CBeelen Dec 9, 2021
d34852e
Merge branch 'Consensus_insertions' into Haploflow
CBeelen Dec 10, 2021
94e4ce5
Merge branch 'master' into Haploflow
CBeelen Jan 15, 2022
2dd8e78
Merge branch 'master' into Haploflow
CBeelen Feb 4, 2022
2dfc1dd
Merge branch 'master' into Haploflow
CBeelen Mar 17, 2022
bcbcc70
Merge branch 'master' into Haploflow
CBeelen Jun 7, 2022
d5ac6b2
Merge branch 'master' into Haploflow
Donaim Sep 15, 2023
138dcb2
Install Haploflow during docker initialization
Donaim Sep 19, 2023
c621522
Install Haploflow on CI
Donaim Sep 19, 2023
272635d
Pin exact Haploflow version
Donaim Sep 19, 2023
5ecb109
Switch to Debian in Singularity image
Donaim Sep 20, 2023
d0407a8
Merge branch 'master' into Haploflow-test1
Donaim Sep 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,17 @@ jobs:
with:
go-version: '^1.15.6'

- name: Install Haploflow
run: |
sudo apt-get update
sudo apt-get install -y build-essential git ronn
cd /opt/
git clone https://github.com/hzi-bifo/Haploflow
cd Haploflow
git checkout 9a5a0ff6c3a0435e723e41f98fe82ec2ad19cf50
sh build.sh
sudo ln -s /opt/Haploflow/build/haploflow ~/bin/haploflow

- name: Install IVA assembler dependencies
run: |
sudo apt-get install -qq zlib1g-dev libncurses5-dev libncursesw5-dev mummer ncbi-blast+
Expand Down
10 changes: 10 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,16 @@ RUN apt-get install -q -y zlib1g-dev libncurses5-dev libncursesw5-dev && \
tar -xzf smalt-0.7.6-bin.tar.gz --no-same-owner && \
ln -s /opt/smalt-0.7.6-bin/smalt_x86_64 /bin/smalt

## Install Haploflow
RUN apt-get update && \
apt-get install -y build-essential sudo git ronn cmake && \
cd /opt/ && \
git clone https://github.com/hzi-bifo/Haploflow && \
cd Haploflow && \
git checkout 9a5a0ff6c3a0435e723e41f98fe82ec2ad19cf50 && \
yes | sh build.sh && \
ln -s /opt/Haploflow/build/haploflow /bin/haploflow

## Install dependencies for genetracks/drawsvg
RUN apt-get install -q -y libcairo2-dev

Expand Down
79 changes: 25 additions & 54 deletions Singularity
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Generate the Singularity container to run MiCall on Kive.
Bootstrap: docker
From: centos:7
From: python:3.8

%help
MiCall maps all the reads from a sample against a set of reference
Expand Down Expand Up @@ -53,48 +53,28 @@ From: centos:7

%post
echo ===== Installing Prerequisites ===== >/dev/null
yum update -q -y

yum groupinstall -q -y 'development tools'
yum install -q -y epel-release
yum install -q -y unzip wget fontconfig bzip2-devel xz-devel openssl-devel \
libffi-devel sqlite-devel

echo ===== Installing Python ===== >/dev/null
wget -q https://www.python.org/ftp/python/3.8.3/Python-3.8.3.tar.xz
tar xJf Python*
rm Python*.xz
cd Python*
./configure --enable-optimizations
make altinstall
cd ..
rm -rf Python*
ln -s /usr/local/bin/python3.8 /usr/local/bin/python3
apt-get update -q
apt-get install -q -y unzip wget

echo ===== Installing blast ===== >/dev/null
cd /root
# Saved our own copy, because download was slow from ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.6.0/ncbi-blast-2.6.0+-1.x86_64.rpm
wget -q https://github.com/cfe-lab/MiCall/releases/download/v7.12.dev28/ncbi-blast-2.6.0+-1.x86_64.rpm
yum install -q -y ncbi-blast-2.6.0+-1.x86_64.rpm
rm ncbi-blast-2.6.0+-1.x86_64.rpm
python3 /opt/micall/micall/blast_db/make_blast_db.py
apt-get install -q -y ncbi-blast+

echo ===== Installing Rust and merge-mates ===== >/dev/null
yum install -q -y rust cargo
wget -qO rustup.sh https://sh.rustup.rs
chmod +x /rustup.sh
/rustup.sh -y -q
. /root/.cargo/env
rm rustup.sh
cargo install --quiet --root / --git https://github.com/jeff-k/merge-mates.git --rev 2fec61363f645e2008a4adff553d098beae21469

## Miniconda (Python 2) (Don't use this)
#wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -O miniconda.sh
#bash miniconda.sh -b -p /opt/miniconda

echo ===== Installing bowtie2 ===== >/dev/null
wget -q -O bowtie2.zip https://github.com/BenLangmead/bowtie2/releases/download/v2.2.8/bowtie2-2.2.8-linux-x86_64.zip
unzip -qq bowtie2.zip -d /opt/
unzip bowtie2.zip -d /opt/
ln -s /opt/bowtie2-2.2.8/ /opt/bowtie2
rm bowtie2.zip

echo ===== Installing IVA dependencies ===== >/dev/null
yum install -q -y tcsh ncurses-devel zlib-devel
apt-get install -q -y zlib1g-dev libncurses5-dev libncursesw5-dev
cd /bin
wget -q http://sun.aei.polsl.pl/kmc/download-2.1.1/linux/kmc
wget -q http://sun.aei.polsl.pl/kmc/download-2.1.1/linux/kmc_dump
Expand Down Expand Up @@ -122,32 +102,23 @@ From: centos:7
tar -xzf smalt-0.7.6-bin.tar.gz --no-same-owner
ln -s /opt/smalt-0.7.6-bin/smalt_x86_64 /bin/smalt

echo ===== Installing Haploflow ===== >/dev/null
apt-get install -q -y libboost-all-dev build-essential sudo git ronn cmake
cd /opt/
git clone https://github.com/hzi-bifo/Haploflow
cd Haploflow
git checkout 9a5a0ff6c3a0435e723e41f98fe82ec2ad19cf50
yes | sh build.sh

echo ===== Installing Python packages ===== >/dev/null
# Install dependencies for genetracks/drawsvg
apt-get install -q -y libcairo2-dev
# Also trigger matplotlib to build its font cache.
wget -q https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py
rm get-pip.py
cd /opt
pip install --quiet -r /opt/micall/requirements.txt
ln -s /usr/local/bin/cutadapt /usr/local/bin/cutadapt-1.11
python3 -c 'import matplotlib; matplotlib.use("Agg"); import matplotlib.pyplot'

# Install dependencies for genetracks/drawsvg
yum install -q -y cairo-devel cairo cairo-tools zlib-devel

yum groupremove -q -y 'development tools'
yum remove -q -y epel-release wget unzip
yum autoremove -q -y
yum clean all

rm -rf /var/cache/yum

## CAUTION! This changes the default python command to python3!
## This breaks many things, including yum!
## To switch back to python2, use this command:
# sudo alternatives --set python /usr/bin/python2
alternatives --install /usr/bin/python python /usr/bin/python2 50
alternatives --install /usr/bin/python python /usr/local/bin/python3 60
pip install --upgrade pip
pip install -r /opt/micall/requirements-basespace.txt
python -c 'import matplotlib; matplotlib.use("Agg"); import matplotlib.pyplot'
python /opt/micall/micall/blast_db/make_blast_db.py

%environment
export PATH=/opt/bowtie2:/bin:/usr/local/bin
Expand Down
Loading
Loading