Skip to main content

flyte-sdk

Reliably orchestrate ML pipelines, models, and agents at scale — in pure Python.

Overview​

The flyte-sdk is the primary Python interface for building, deploying, and managing compound AI applications and distributed workflows. It allows developers to define complex execution environments, manage dependencies, and orchestrate tasks across local and remote infrastructure without leaving their Python IDE. By treating infrastructure as code, the SDK ensures that your ML pipelines are reproducible, scalable, and easy to debug.

At its core, the SDK provides a high-level API for defining Tasks and Environments that encapsulate your code, its dependencies, and its resource requirements. Whether you are running a simple data processing script or a massive distributed training job, the SDK handles the heavy lifting of containerization, resource allocation, and remote execution. It integrates seamlessly with popular tools like FastAPI for serving, Spark and Ray for distributed compute, and MLflow for experiment tracking.

Key Concepts​

  • Task Environments Introduction to AppEnvironment: The foundational building block that defines the execution context for your code, including the base image, Python dependencies, and system packages.
  • Tasks Tasks and Environments: Individual units of work decorated with @env.task that can be executed locally for development or offloaded to a remote cluster for production scale.
  • Remote Execution Remote Execution and Inspection: A powerful mechanism to trigger, monitor, and inspect runs on a remote Flyte cluster directly from Python or the CLI.
  • Data I/O & DataFrames Data I/O: Native support for handling large-scale data, including files, directories, and interoperable DataFrames with built-in validation.
  • Checkpointing Introduction to Task Checkpointing: A mechanism for saving and restoring state within long-running tasks, enabling resilience against failures and efficient retries.
  • Plugins & Integrations Plugins and Integrations: Extensible support for distributed compute engines (Spark, Ray, Dask), specialized hardware (GPUs, TPUs), and external services like Snowflake or BigQuery.

Common Use Cases​

  • ML Pipeline Orchestration: Building end-to-end machine learning workflows from data ingestion and preprocessing to model training and evaluation.
  • Distributed Training: Scaling PyTorch or Spark jobs across multiple nodes with automated resource management and environment consistency.
  • Model Serving: Deploying AI models as scalable web services using FastAPI integrations and webhook apps.
  • Scheduled Data Processing: Automating recurring data transformation tasks using cron or fixed-rate triggers.
  • Compound AI Applications: Developing complex agents and applications that combine multiple LLMs, tools, and data sources.

Getting Started​

To begin building with the SDK, start by Connect to Flyte and Authenticate Remote Requests to your cluster. Once connected, you can Define Task Environments to specify your dependencies and Invoke a Task for Remote Execution to see your code running at scale. For a step-by-step walkthrough, visit the Getting Started guide.