Adding Python Dependencies

Add packages to an image

Start with a Flyte base image and add dependency layers with the fluent Image API. For example, this creates an image for a task environment with two PyPI requirements:

import flyte

env = flyte.TaskEnvironment(
    name="dataframe_image",
    image=(
        flyte.Image.from_debian_base(python_version=(3, 12))
        .with_pip_packages("pandas", "scikit-learn")
    ),
)

Image.from_debian_base() is the normal starting point when the image will receive Flyte layers. Each with_* call returns a new immutable Image and appends one typed layer through Image.clone(addl_layer=...). The inline package call creates a PipPackages layer. The local Docker builder consumes that layer with PipAndRequirementsHandler and renders an uv pip install command.

An unmodified image returns its base URI from Image.uri. Once a dependency layer is added, the image is marked as cloned and its URI uses the configured registry and name with a content-derived tag. The package strings and pip options are included in the layer hash, so changing them changes the image identity.

Image.from_base() starts from an already-built image URI. To add layers to such an image, give the clone a name and make it extendable first:

import flyte

image = (
    flyte.Image.from_base("python:3.12-slim")
    .clone(registry="localhost", name="analytics", extendable=True)
    .with_pip_packages("numpy", "requests>=2.0,<3")
)

Do not construct Image(...) directly. Its __post_init__ rejects direct construction; use one of the from_* constructors.

Choose a dependency source

Source	Image method	Layer	Files copied by default	Install-project behavior
Individual requirement specifiers	with_pip_packages(*packages, ...)	PipPackages	No dependency file	Not applicable
A pip requirements file	with_requirements(file, ...)	Requirements	The .txt file	Not applicable
A uv project	with_uv_project(pyproject_file, ...)	UVProject	pyproject.toml and uv.lock when present	install_project copies the project directory
A Poetry project	with_poetry_project(pyproject_file, ...)	PoetryProject	pyproject.toml and poetry.lock	install_project copies the project directory

All four methods accept build-time secret_mounts where credentials are needed. PipPackages, Requirements, and UVProject also inherit the pip options index_url, extra_index_urls, pre, and extra_args. PoetryProject intentionally does not inherit PipOption; Poetry has its own installation arguments.

Install individual packages

Pass each package as a separate positional argument. Package strings use pip requirement syntax, including version constraints:

import flyte

image = (
    flyte.Image.from_debian_base()
    .with_pip_packages(
        "numpy",
        "pandas>=2.0",
        index_url="https://pypi.example.com/simple",
        extra_index_urls=["https://extra.example.com/simple"],
        pre=True,
        extra_args="--no-deps",
    )
)

extra_index_urls may be a string, list, or tuple at the public method boundary; Image.with_pip_packages normalizes it to a tuple in PipPackages. pre=True becomes the installer’s pre-release option. The Docker builder shell-quotes specifications containing characters such as < and >, so constraints should remain part of the package string as shown.

Do not pass a list as the package collection. PipPackages stores an optional tuple and its layer validation rejects lists because layers must remain hashable. Use .with_pip_packages("numpy", "pandas") instead.

Install a .txt requirements file

Use with_requirements when the dependency set is already maintained in a requirements file:

from pathlib import Path

import flyte

requirements = Path("requirements.txt")
image = (
    flyte.Image.from_debian_base(registry="localhost", name="requirements-image")
    .with_requirements(
        requirements,
        index_url="https://private.example.com/simple",
        extra_index_urls=["https://extra.example.com"],
        pre=True,
        extra_args="--no-deps",
    )
)

Image.with_requirements converts a string path to Path, requires a .txt suffix, normalizes extra indexes and secrets to tuples, and appends a Requirements layer. During a local build, PipAndRequirementsHandler verifies that the path exists and is a regular file, copies it into the build context, and invokes installation with --requirement requirements.txt. The file contents are included in the layer hash.

The suffix check happens when with_requirements is called; existence and regular-file checks happen when the builder processes the layer. Consequently, a missing or directory-valued path can fail later during the image build.

Install a uv project

Point with_uv_project at pyproject.toml. If uvlock is omitted, Flyte checks for pyproject.toml’s sibling uv.lock and uses it only when that file exists:

from pathlib import Path

import flyte

image = (
    flyte.Image.from_debian_base(registry="localhost", name="uv-image")
    .with_uv_project(
        pyproject_file=Path("project/pyproject.toml"),
        project_install_mode="dependencies_only",
    )
)

UVProject.validate requires an existing pyproject.toml; an explicitly supplied uvlock must also exist. A lockfile is therefore optional for this method. In the default dependencies_only mode, UVProjectHandler copies only the project metadata and lockfile (when available), adds --no-install-project unless that argument was already supplied, and runs uv synchronization. The layer hash includes those metadata files.

Use install_project when the current project itself must be installed:

from pathlib import Path

import flyte

image = flyte.Image.from_debian_base(registry="localhost", name="uv-project").with_uv_project(
    pyproject_file=Path("project/pyproject.toml"),
    uvlock=Path("project/uv.lock"),
    project_install_mode="install_project",
)

In this mode the builder copies the whole project directory. Docker-ignore patterns affect both the copied content and the content-derived hash. The accepted project_install_mode values are "dependencies_only" and "install_project".

Install a Poetry project

Poetry projects require a lockfile. The method selects pyproject_file.parent / "poetry.lock" when poetry_lock is not provided:

from pathlib import Path

import flyte

image = flyte.Image.from_debian_base(registry="localhost", name="poetry-image").with_poetry_project(
    pyproject_file=Path("project/pyproject.toml"),
    project_install_mode="dependencies_only",
)

PoetryProject.validate requires both pyproject.toml and poetry.lock. In dependencies_only mode, PoetryProjectHandler copies only those two files and adds --no-root unless it is already included in extra_args. The layer hash covers both files and the relevant installer arguments.

To install the project as well, select install_project:

from pathlib import Path

import flyte

image = flyte.Image.from_debian_base(registry="localhost", name="poetry-project").with_poetry_project(
    pyproject_file=Path("project/pyproject.toml"),
    poetry_lock=Path("project/poetry.lock"),
    extra_args="--only main",
    project_install_mode="install_project",
)

Install-project mode copies the full project directory, applies Docker-ignore patterns, and does not force --no-root. The Poetry build path installs Poetry with uv before invoking Poetry’s installation. Although the with_poetry_project docstring mentions copying the entire project by default, the actual default is dependencies_only, and the builder copies only the two metadata/lock files in that mode.

Use private indexes and build secrets

Use index_url and extra_index_urls for private or additional package indexes. If the index requires a credential, provide it as a build-time secret through secret_mounts. A string is accepted directly and is normalized to a Secret request internally:

import flyte

image = (
    flyte.Image.from_debian_base()
    .with_pip_packages(
        "private-package",
        index_url="https://private.example.com/simple",
        secret_mounts=["PYPI_TOKEN"],
    )
)

secret_mounts accepts a string, a Secret, or a list containing strings and Secret objects. The source’s private-package pattern uses a token in a Git URL:

import flyte

private_package = "git+https://$GITHUB_PAT@github.com/flyteorg/flytex.git@2e20a2acebfc3877d84af643fdd768edea41d533"

image = (
    flyte.Image.from_debian_base()
    .with_pip_packages(
        private_package,
        secret_mounts=["GITHUB_PAT"],
    )
)

The local Docker builder turns these requests into BuildKit secret mounts on the install command. The secret is available for that build command; it is not written as an ordinary image environment variable. A Secret describes the key, optional group, mount, and environment-variable configuration used for the mount.

The builder resolves the material before generating the build command by checking an uppercase environment variable formed from group and key first. If it is absent, it checks /etc/secrets/<group>_<key> (or /etc/secrets/<key> without a group). If neither exists, the build raises FileNotFoundError. Repeated secrets are deduplicated using the deterministic hash supplied by Secret.

Secret derives an environment variable name from the key when neither mount nor as_env_var is supplied, replacing hyphens with underscores and uppercasing the result. An explicitly supplied as_env_var must match ^[A-Z_][A-Z0-9_]*$; for example, PYPI_TOKEN is valid while a lowercase or hyphenated name is rejected. File mounts currently accept only /etc/flyte/secrets.

This build-time use is separate from runtime task secrets. A Secret passed to TaskEnvironment(..., secrets=...) describes a secret available to task execution; placing it in an image layer’s secret_mounts makes it available while that layer is built.

Troubleshoot image dependency builds

Check the base image before adding a layer

clone(addl_layer=...) requires an extendable image with a name. from_debian_base() supplies a suitable Flyte base. A Dockerfile-based image is explicitly non-extendable because Flyte does not parse its package-manager setup; put dependencies directly in that Dockerfile instead of calling a dependency with_* method afterward.

Check the expected files

• with_requirements requires a .txt suffix; the builder additionally requires that the path exists and is a regular file.
• with_uv_project requires pyproject.toml, but permits uvlock=None. Sibling uv.lock detection occurs only when the file exists.
• with_poetry_project requires both pyproject.toml and poetry.lock, either through the sibling default or an explicit poetry_lock path.
• With project_install_mode="install_project", the entire project is copied and Docker-ignore rules affect the copy and its hash. In dependencies_only, uv adds --no-install-project and Poetry adds --no-root.

Avoid common configuration mistakes

• Use Image.from_*() rather than Image(...).
• Pass package arguments separately rather than supplying a list to the layer.
• Do not assume dependencies_only installs the current project; select install_project when that is required.
• Do not expect secret_mounts credentials to persist in the resulting image. They are mounted for the build command, and missing environment-variable/file material raises FileNotFoundError.
• Do not use an arbitrary Secret.mount path. Secret currently accepts only /etc/flyte/secrets.