Skip to content

Commit

Permalink
Addressing PO's comments
Browse files Browse the repository at this point in the history
  • Loading branch information
paulstretenowich committed Oct 29, 2024
1 parent 97bda57 commit 07daf05
Show file tree
Hide file tree
Showing 5 changed files with 39 additions and 57 deletions.
40 changes: 0 additions & 40 deletions .github/workflows/apptainer-sif-push.yml

This file was deleted.

22 changes: 22 additions & 0 deletions .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,28 @@ jobs:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Build and push Apptainer image
if: ${{ ! startsWith(github.ref, 'refs/tags/') }}
steps:
- uses: actions/checkout@v4
- uses: eWaterCycle/setup-apptainer@v2
with:
apptainer-version: 1.3.4
- name: Create sif image
run: |
apptainer build -F container/wrapper_genpipes/images/genpipes-${{ env.TAG_NAME }}.sif docker-daemon://${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.TAG_NAME }}
cd container/wrapper_genpipes/images/
ln -sf genpipes-${{ env.TAG_NAME }}.sif genpipes.sif
cd ../../
tar -zcvf wrapper_genpipes.tgz wrapper_genpipes
- name: Upload sif image
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: container/wrapper_genpipes.tgz
asset_name: wrapper_genpipes.tgz
tag: ${{ github.ref }}
overwrite: true

# This step generates an artifact attestation for the image, which is an unforgeable statement about where and how it was built.
# It increases supply chain security for people who consume the image.
Expand Down
16 changes: 8 additions & 8 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,25 +9,25 @@ ENV MODULE_VERSION=4.1.2

# All apt-get cmd
RUN apt-get update -y && \
apt-get install -y wget lsb-release
RUN wget https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-${CVMFS_VERSION}_all.deb && \
apt-get install -y wget lsb-release && \
wget https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-${CVMFS_VERSION}_all.deb && \
dpkg -i cvmfs-release-${CVMFS_VERSION}_all.deb && \
rm -f cvmfs-release-${CVMFS_VERSION}_all.deb && \
apt-get update -y && \
apt-get install -y cvmfs
RUN wget https://package.computecanada.ca/yum/cc-cvmfs-public/prod/other/cvmfs-config-computecanada-${CC_STACK}.all.deb && \
apt-get install -y cvmfs && \
wget https://package.computecanada.ca/yum/cc-cvmfs-public/prod/other/cvmfs-config-computecanada-${CC_STACK}.all.deb && \
apt-get install -y ./cvmfs-config-computecanada-${CC_STACK}.all.deb && \
rm -f cvmfs-config-computecanada-${CC_STACK}.all.deb
RUN apt-get install -y libpng-dev libjpeg-dev libtiff-dev && \
rm -f cvmfs-config-computecanada-${CC_STACK}.all.deb && \
apt-get install -y libpng-dev libjpeg-dev libtiff-dev && \
apt-get install -y imagemagick pigz which zip unzip make gcc expect file dejagnu less tcl-dev cvmfs-config-computecanada cvmfs-fuse3 cvmfs-config-default && \
apt-get clean all

RUN mkdir /cvmfs-cache && chmod 777 /cvmfs-cache /cvmfs
RUN mkdir /cvmfs/ref.mugqic /cvmfs/soft.mugqic /cvmfs/cvmfs-config.computecanada.ca

# module
RUN wget https://github.com/cea-hpc/modules/releases/download/v${MODULE_VERSION}/modules-${MODULE_VERSION}.tar.gz
RUN tar xzf modules-${MODULE_VERSION}.tar.gz && \
RUN wget https://github.com/cea-hpc/modules/releases/download/v${MODULE_VERSION}/modules-${MODULE_VERSION}.tar.gz && \
tar xzf modules-${MODULE_VERSION}.tar.gz && \
rm modules-${MODULE_VERSION}.tar.gz \
&& cd modules-${MODULE_VERSION} && ./configure && make -j 7 && make install \
&& cd .. && rm -rf modules-${MODULE_VERSION} && rm -rf /usr/local/Modules/modulefiles/*
Expand Down
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ While you can use (GiaC) to debug GenPipes on your laptop, [GenPipes](https://bi

## Install a compatible container technology on your machine

Follow installation procedure from the [Singularity install page](https://github.com/hpcng/singularity/blob/master/INSTALL.md) or the [Docker install page](https://docs.docker.com/get-docker/).
Follow installation procedure from the [Apptainer install page](https://apptainer.org/docs/user/latest/quick_start.html#installation) or the [Docker install page](https://docs.docker.com/get-docker/).

You can also [Read the GenPipes documentation](https://genpipes.readthedocs.io/).

Expand All @@ -35,14 +35,14 @@ You can now configure the `genpipes/resources/container/etc/wrapper.conf` file:
# GEN_SHARED_CVMFS should have a sufficient amount of space to load full reference files
export GEN_SHARED_CVMFS=$HOME/cvmfs
BIND_LIST=
GEN_CONTAINERTYPE=singularity
GEN_CONTAINERTYPE=apptainer
```

`GEN_SHARED_CVMFS` will hold a cache for GiaC [CVMFS](https://cernvm.cern.ch/portal/filesystem) system, it will hold the genomes and software being used by GenPipes. This folder will grow with GenPipes usage. You can delete it in between usage, but keep in mind that once deleted it will need to be rebuild by downloading data form the internet.

`BIND_LIST` is a list of file system, separated by comma, you need GenPipes to have access to, by default, only your $HOME is mounted. For example if you are on an HPC system with a `/scratch` and `/data` space, you would have `BIND_LIST=/scratch,/data`. The string will be fed to Singularity `--bind` option, see `singularity --help` for more details.
`BIND_LIST` is a list of file system, separated by comma, you need GenPipes to have access to, by default, only your $HOME is mounted. For example if you are on an HPC system with a `/scratch` and `/data` space, you would have `BIND_LIST=/scratch,/data`. The string will be fed to Singularity `--bind` option, see `apptainer --help` for more details.

`GEN_CONTAINERTYPE` is the container to use, either `singularity` (default), `apptainer` or `docker`.
`GEN_CONTAINERTYPE` is the container to use, either `apptainer` (default), `singularity`, `docker` or `podman`.

You do not need any other setup on your machine.

Expand All @@ -62,10 +62,10 @@ https://genpipes.readthedocs.io/
You can also run the `genpipes/resources/container/bin/container_wrapper.sh` command to get inside the container with the right configuration. You will then have access to all the GenPipes tools be able to run them directly inside the container, on a single host without the `--wrap` option.

## Whitout the wrapper
### Using Singularity
### Using Apptainer
With `GEN_SHARED_CVMFS` being the cache directory on the host, `BIND_LIST` the file system to be accessed by genpipes, {IMAGE_PATH}/genpipes.sif the [latest sif file released](https://github.com/c3g/genpipes_in_a_container/releases/latest).
```
singularity run \
apptainer run \
--cleanenv \
-S /var/run/cvmfs \
-B ${GEN_SHARED_CVMFS}:/cvmfs-cache \
Expand All @@ -75,10 +75,10 @@ With `GEN_SHARED_CVMFS` being the cache directory on the host, `BIND_LIST` the f
--fusemount "container:cvmfs2 ref.mugqic /cvmfs/ref.mugqic" \
${IMAGE_PATH}/genpipes.sif
```
### Using Apptainer
### Using Singularity
With `GEN_SHARED_CVMFS` being the cache directory on the host, `BIND_LIST` the file system to be accessed by genpipes, {IMAGE_PATH}/genpipes.sif the [latest sif file released](https://github.com/c3g/genpipes_in_a_container/releases/latest).
```
apptainer run \
singularity run \
--cleanenv \
-S /var/run/cvmfs \
-B ${GEN_SHARED_CVMFS}:/cvmfs-cache \
Expand Down
2 changes: 1 addition & 1 deletion container/wrapper_genpipes/etc/wrapper.conf.tpl
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# GEN_SHARED_CVMFS should have a sufficient amount of space to load full reference files
export GEN_SHARED_CVMFS=$HOME/cvmfs
BIND_LIST=
GEN_CONTAINERTYPE=singularity
GEN_CONTAINERTYPE=apptainer

0 comments on commit 07daf05

Please sign in to comment.