What are the best practices for using poetry with docker?

[axiangcoding]

Most tutorials recommend using pip to install poetry. Will this affect code dependencies?

Can I install poetry using pipx in docker and exclude poetry in the final image via a multi-stage build?

[sbrunner]

I don't know if it's the best practice, but personally, I use multistage build like:

FROM ubuntu as base-all

RUN --mount=type=cache,target=/var/lib/apt/lists \
    --mount=type=cache,target=/var/cache,sharing=locked \
    apt-get update \
    && apt-get upgrade --assume-yes \
    && apt-get install --assume-yes --no-install-recommends python3-pip

FROM base-all as poetry

# Install Poetry
WORKDIR /tmp
COPY requirements.txt ./ # that contains poetry=<version>
RUN --mount=type=cache,target=/root/.cache \
    python3 -m pip install --disable-pip-version-check --requirement=requirements.txt

# Do the conversion
COPY poetry.lock pyproject.toml ./
RUN poetry export --output=requirements.txt

# Base, the biggest thing is to install the Python packages
FROM base-all as base

RUN --mount=type=cache,target=/root/.cache \
    --mount=type=bind,from=poetry,source=/tmp,target=/poetry \
    && python3 -m pip install --disable-pip-version-check --no-deps --requirement=/poetry/requirements.txt

[axiangcoding]

Thanks for the reply!

I boil this dockerfile down into a few steps

1. Use pip to install poetry (stage 1)
2. Use poetry to export dependencies as requirements.txt (stage 1)
3. Install requirements.txt (stage 2)
4. other stuff (stage 2)

These steps are similar to most dockerfiles use poetry to manage dependencies.
But still, i don't know if this is the best practice
Perhaps poetry team should suggest a best practice and provide it in the documentation.

[elephantum]

I do mostly the same as @sbrunner , but sometimes, especially for machine learning stuff, I produce several requirement files for different groups of dependencies. So that heavy stuff like pytorch or tensorflow are not reinstalled when some minor dependency change.

[agirault]

why would get they reinstalled if you do a mount cache ? wouldn't it reuse that right away?

[clintonroy]

Best practice is whatever works for you. Different people use tools in different ways. There is no one perfect way.
Things that might work in docker might not work in other container environments. Different ways will work better for different linux distributions, different python versions, the poetry team is never going to be able to pick just one way.

[axiangcoding]

Thanks for the reply!

Maybe provide a recommendation instead of a best practice? Users can adjust it to suit themself based on the recommended dockerfile.
The reason I bring this up is that when I tried to install poetry using pipx in docker it kept failing. I think if there is a recommended way, beginners can get started better