Skip to main content

Architecture Overview

This section contains architecture diagrams and documentation for flyte-sdk.

Available Diagrams​

Flytekit SDK Ecosystem Context​

This context diagram is grounded in the Python package and its declared entry points. A Developer can use the flyte Click CLI (pyproject.toml exposes flyte = flyte.cli.main:main) or call the Flytekit Python SDK directly to author tasks/workflows, run and deploy them, build images, serve apps, and inspect resources. The CLI creates SDK configuration and exposes commands including run, deploy, build, serve, get, and prefetch.\n\nThe SDK initializes a ClientSet with ConnectRPC clients for project/domain, task, app, run, run logs, secrets, identity, triggers, cluster, and the cluster-aware DataProxy services. These clients communicate with the Flyte Backend / Control Plane for registration, execution, app and trigger management, logs, metadata, and status. API-key initialization reads FLYTE_API_KEY, decodes endpoint and client credentials, and establishes the authenticated session.\n\nThe SDK's storage layer explicitly supports S3, GCS, and Azure Blob/ABFS through obstore/fsspec-style access. Run creation uploads offloaded inputs through the DataProxy service, while files, directories, dataframes, code bundles, checkpoints, and model artifacts use remote object-storage URIs. Image definitions support registries and both local and remote image builders; the remote builder uses the backend image-builder service and checks image availability in a registry. The model-loader extra has a real PyTorch dependency and converts streamed safetensors data into PyTorch tensors. Finally, local notification sending uses HTTP webhooks for Slack, Teams, and custom webhooks plus SMTP, while remote notification rules are represented and submitted through the backend.\n\nThe diagram intentionally groups the numerous backend RPCs and storage providers into context-level systems rather than exposing internal classes. Tasks and Environments and Remote Execution and Inspection are the principal SDK usage concepts represented by the flows.

Key Architectural Findings:

  • pyproject.toml packages the project as flyte and exposes flyte, a0, fserve, and c0 command-line entry points.
  • ClientSet constructs ConnectRPC clients for project, task, app, run, logs, secrets, identity, trigger, cluster, and DataProxy backend services.
  • Run creation explicitly uploads inputs through DataProxyServiceClient before calling RunServiceClient.create_run.
  • Storage configuration and implementation explicitly support S3, GCS, and Azure Blob/ABFS; S3 can use boto3-backed credential resolution.
  • Image definitions include registry targets and local/remote image builders; the remote builder calls the Flyte image-builder service and checks remote registry images.
  • The model-loader extra imports PyTorch and streams safetensors/model data from object storage into PyTorch tensors.
  • Local notifications make HTTP calls to Slack, Teams, and custom webhook URLs and connect to SMTP on localhost:25.

Flytekit SDK Internal Architecture​

The diagram illustrates the internal component architecture of the Flytekit SDK Ecosystem Context (flyte), highlighting how user code and CLI commands flow through core authoring abstractions, the Type Engine, runtime execution controllers, image building utilities, plugins, and the remote client.

Key Components:​

  • CLI (flyte.cli): Entry point for command-line operations (run, build, deploy, serve, get, devbox), orchestrating task resolution, image builds, and remote dispatch.
  • Core SDK (flyte): The authoring and definition layer providing decorators and configuration entities (@task, @run, @serve, Environment, Resources, Secret, Checkpoint, Trigger).
  • Runtime & Execution (flyte._internal.runtime & flyte._internal.controllers): Handles local workflow and task execution, context management (ctx), serialization/deserialization of task metadata, and dispatch to local or remote backends.
  • Type Engine (flyte.types._type_engine): Translates Python native types, Pydantic models, dataclasses, and Flyte IO types into Flyte IDL literals and type signatures with serialization codecs (JSON, Msgpack).
  • Storage & IO Layer (flyte.storage & flyte.io): Manages lazy uploads, artifact downloads, checkpoints, and offloaded literal representations with underlying cloud/local storage backends.
  • Image Builder (flyte._internal.imagebuild): Resolves dependencies, generates Dockerfiles, bundles user code via _code_bundle, and interfaces with local Docker or remote image building services.
  • Plugin System (flyteplugins): Extensible integrations for distributed computing (Spark, Ray, Dask, PyTorch), data processing & validation (Polars, Pandera), ML lifecycle (MLflow, WandB), and LLM/agent frameworks (Anthropic, Gemini, OpenAI, Codegen).
  • Remote Client (flyte.remote): gRPC/ConnectRPC client communicating with the Flyte Control Plane for deployment, run orchestration, secret sync, and execution tracking with authentication (PKCE/API key).

Key Architectural Findings:

  • The SDK is organized cleanly into an authoring layer (flyte), execution runtime (flyte._internal.runtime / controllers), remote backend communication (flyte.remote), type translation (flyte.types._type_engine), image construction (flyte._internal.imagebuild), and a rich CLI (flyte.cli).
  • flyteplugins integrates seamlessly by registering custom TypeTransformer instances with TypeEngine (e.g., Pandera, Polars, OmegaConf) and extending task execution decorators (e.g., PyTorch, Ray, Spark, Papermill).
  • The Type Engine uses Mashumaro and Pydantic under the hood for dataclass and model serialization, converting complex nested structures and lazy Flyte IO types (File, Dir, DataFrame) into Flyte IDL literals.
  • Remote execution interacts via flyte.remote._client using ConnectRPC protocols with PKCE, API keys, and cluster-aware dataproxy for efficient data transfers.
  • Image building (flyte._internal.imagebuild) is decoupled from core task authoring, supporting local Docker builders and remote builders alongside code bundling mechanisms (_code_bundle).

Workflow Execution and Registration Flow​

This diagram illustrates the sequence of events for both registering a new workflow and executing it using the Flytekit SDK and CLI.

Registration Flow:

  1. A developer uses the flyte deploy command to register their workflow code.
  2. The CLI invokes the flyte.deploy function within the Flytekit SDK.
  3. The SDK then plans the deployment, which involves analyzing the tasks and their dependencies.
  4. The apply function orchestrates the deployment, calling _deploy_task_env and then _deploy_task for each individual task within the environment.
  5. Finally, a DeployTaskRequest is sent to the Flyte Backend API to create or update the task registration.

Execution Flow:

  1. A user or developer initiates a remote execution using the flyte run --remote command, specifying the task and its inputs.
  2. The CLI uses the with_runcontext and run methods in the SDK to prepare the execution.
  3. The internal _Runner class's _run_remote method is called.
  4. The SDK first sends an UploadInputsRequest to the Flyte backend's dataproxy service to offload any large input data.
  5. With the inputs uploaded, the SDK sends a CreateRunRequest to the backend's run service to launch the workflow.
  6. The backend returns a CreateRunResponse, and the SDK provides a Run object back to the CLI, which then displays the execution URL and status to the user.

Key Architectural Findings:

  • The flyte deploy command initiates the registration process, which involves a series of internal SDK calls culminating in a DeployTaskRequest to the backend.
  • The core registration logic is found in flyte._deploy.py, with _deploy_task being the function that makes the final API call.
  • The flyte run --remote command triggers the execution flow.
  • Execution involves a _Runner class within flyte._run.py that handles the remote execution.
  • Remote execution is a two-step process: first uploading inputs via dataproxy_service.upload_inputs and then creating the run via run_service.create_run.

Flytekit Core Domain Model​

This diagram reflects the domain model actually implemented in this repository’s modern flyte SDK. A TaskEnvironment owns registered TaskTemplate objects; each template carries a NativeInterface, execution/configuration fields, and optional code-bundle serialization metadata. NativeInterface.inputs stores (Type, default) pairs while outputs stores named types. Deployment converts those maps into FlyteIDL Variable entries, whose type is a LiteralType. The inspected JSON-schema converter confirms the supported LiteralType variants shown here: simple, collection, map, enum, blob, structured dataset, union, and struct metadata.

The remote lifecycle is modeled separately: Task wraps a task protobuf and TaskDetails exposes its resolved task template; Run owns an Action, ActionDetails exposes action execution state and references cached ActionInputs and ActionOutputs. ActionPhase is the lifecycle enum used by actions and runs, while the runtime conversion layer models an ExecutionError with a recoverable flag.

The requested legacy Workflow, Node, LaunchPlan, flyte.models.literals.LiteralType, flyte.models.core.types.Variable, and flyte.models.execution.Execution classes were not found in this codebase. The repository instead uses TaskEnvironment plus task templates and FlyteIDL2 protobuf messages; therefore those absent classes are intentionally not fabricated as entities. This is a library-level authoring and remote-execution model, not an ORM schema: the exploration found no database entities, migrations, foreign keys, or ORM relationship declarations. The interface/type boundary is closely related to the TypeEngine, and remote action state is exposed through the flyte.remote.

Key Architectural Findings:

  • TaskEnvironment is the authoring container and stores _tasks: Dict[str, TaskTemplate]; there is no Workflow or LaunchPlan class in src.
  • TaskTemplate has a concrete dataclass field named interface: NativeInterface plus task execution configuration such as task_type, resources, retries, timeout, entrypoint, and triggers.
  • NativeInterface has inputs: Dict[str, Tuple[Type, Any]] and outputs: Dict[str, Type]; runtime serialization creates TypedInterface containing input/output VariableMaps.
  • interface_pb2.Variable is constructed with type=TypeEngine.to_literal_type(x), establishing the Variable-to-LiteralType relationship.
  • Run constructs an Action from pb2.action; RunDetails constructs ActionDetails from pb2.action, and action details lazily cache ActionInputs and ActionOutputs.
  • No ORM annotations, ForeignKey declarations, migrations, or database schema definitions were found; relationships are object composition and protobuf message references.

Flyte Task Execution Deployment Model​

This deployment diagram illustrates the runtime execution architecture of a Flyte task authored using the Flyte Python SDK.

The workflow begins on the Developer Workstation, where the user authors tasks with task environments, images, and Kubernetes pod configurations (PodTemplate). Using the Flyte CLI/SDK (ImageBuildEngine), task container images are built locally (or remotely via RemoteImageBuilder) and pushed to a Container Registry (such as GitHub Container Registry, ECR, or Docker Hub).

When a task execution or run is triggered:

  1. The Flyte Python SDK serializes task templates, native inputs, and execution metadata, uploading input payloads to Cloud Storage (S3, GCS, or Azure Blob Storage).
  2. The SDK's RemoteController submits an action request to the Flyte Backend / Control Plane (FlyteAdmin / Union gRPC API).
  3. The Flyte Backend orchestrates execution on a Kubernetes Cluster, dispatching the task to run within a dedicated Kubernetes Pod.
  4. The Kubernetes Pod pulls the task image from the Container Registry, runs the Flyte task entrypoint container (alongside any sidecars or custom mounts defined in PodTemplate), downloads inputs from Cloud Storage, executes the user logic, and writes output literals/artifacts back to Cloud Storage.
  5. Execution status and completion signals are reported back to the Flyte Backend.

Key Architectural Findings:

  • Developers author tasks decorated with @env.task using Image (Debian base, UV scripts, or custom Docker images) and optional PodTemplate (Kubernetes V1PodSpec).
  • The Flyte CLI / SDK builds images via ImageBuildEngine / DockerImageBuilder / RemoteImageBuilder and pushes them to a container registry (e.g., ghcr.io, ECR, GCR).
  • RemoteController and ControllerClient handle communication with Flyte Backend / Control Plane gRPC endpoints for task submission and lifecycle monitoring.
  • Task inputs and outputs are serialized/deserialized and stored in Object/Cloud Storage (flyte.storage.S3, GCS, ABFS, or local sandbox storage) via remote filesystems (fsspec/RemoteFSPathResolver).
  • The Flyte backend orchestrates the task as a Kubernetes Pod executing the Flyte entrypoint in the primary container with configured resources and secrets.

Workflow Execution State Machine​

This diagram illustrates the lifecycle of a Flyte workflow execution, as represented by the ActionPhase enum in the Flyte SDK. The execution progresses through a series of states, starting from QUEUED upon submission and ending in one of the terminal states: SUCCEEDED, FAILED, ABORTED, or TIMED_OUT. The state transitions are managed by the Flyte backend, and the SDK provides a view into this state machine.

Key Architectural Findings:

  • The primary states of a workflow execution are defined in the ActionPhase enum located in src/flyte/models.py.
  • The states include QUEUED, WAITING_FOR_RESOURCES, INITIALIZING, RUNNING, SUCCEEDED, FAILED, ABORTED, and TIMED_OUT.
  • The terminal states are SUCCEEDED, FAILED, ABORTED, and TIMED_OUT, as confirmed by the is_terminal property in the ActionPhase enum.
  • The Flyte SDK acts as a client to a remote backend, which manages the state transitions. The SDK reflects the state of the remote execution.
  • For local executions, a simplified state machine (running, succeeded, failed) is used for tracking, as seen in src/flyte/_persistence/_run_store.py.