Scheduling and Automation

Image Management in Flyte

Flyte provides a robust system for defining and building container images, which are essential for executing tasks in a consistent and reproducible environment. This section details the core components involved in image management: the Image class, the ImageBuild class, and the build function.

The Image Class

The flyte.Image class (found at flyte._image.Image) is a dataclass used to specify the characteristics of a container image. It follows a fluent, two-step pattern for construction:

1. Base Constructors (from_* methods): These methods define the base image.

   • from_debian_base(python_version, flyte_version, install_flyte, registry, ...): Creates a Debian-based image with a specified Python version.
   • from_base(image_uri): Uses any existing base image by its URI (e.g., "python:3.12-slim").
   • from_uv_script(script, name, registry, ...): Builds an image from a uv-compatible script with inline dependencies.
   • from_dockerfile(file, registry, name, platform): Creates an image from a custom Dockerfile.
   • from_ref_name(name): References a pre-built image by name.

2. Customization Methods (with_* methods): These methods allow for further customization of the image, each returning a new Image object.

   • with_pip_packages(*packages): Adds Python packages using pip.
   • with_apt_packages(*packages): Adds system packages via apt-get.
   • with_workdir(workdir): Sets the working directory.
   • with_requirements(file, ...): Installs packages from a requirements.txt file.

Example Usage:

import flyte

image = (
    flyte.Image.from_debian_base(python_version=(3, 12))
    .with_pip_packages("pandas", "scikit-learn")
    .with_apt_packages("curl", "git")
)

The Image class is extensively used throughout the Flyte SDK, particularly in examples, to define the execution environment for tasks.

The ImageBuild Class

The flyte.ImageBuild class (found at flyte._build.ImageBuild) represents the result of an image build operation. It contains the following attributes:

• uri: The fully qualified URI of the built image. This will be None if the build was started asynchronously and has not yet completed.
• remote_run: An optional remote.Run object, present if the image build was initiated using a remote builder.

The build Function

The flyte.build function (defined in src/flyte/_build.py) is used to trigger the image build process. Its signature is:

async def build(
    image: Image,
    dry_run: bool = False,
    force: bool = False,
    wait: bool = True,
):
    pass

Parameters:

• image: The flyte.Image object that specifies the image to be built.
• dry_run: A boolean flag. If True, the build process will be simulated without actually creating the image.
• force: A boolean flag. If True, it bypasses any existence checks and forces a rebuild. For remote builds, this also sets overwrite_cache=True.
• wait: A boolean flag. If True, the function will block until the image build is complete. If False, it returns immediately, and the build proceeds in the background.

Returns:

• An ImageBuild object, providing details about the completed or ongoing build operation.

This function allows users to programmatically build and manage their container images, integrating seamlessly with Flyte's task execution workflow.