Local and Remote Serving Workflow

When you build web applications or model serving services with AppEnvironment, testing them against a local HTTP server and promoting them to a remote Flyte cluster requires consistent lifecycle management across both targets. In flyte-sdk, local execution and remote cluster deployments share the uniform AppHandle protocol, configured via flyte.with_servecontext() and managed on the command line through flyte serve.

Serving Apps Programmatically

Use flyte.with_servecontext() to set the serve mode ("local" or "remote"), configure network health checks, inject environment variables, and trigger serving via .serve().

import httpx
import flyte
from flyte.app.extras import FastAPIAppEnvironment
from fastapi import FastAPI

api = FastAPI()

@api.get("/health")
def health():
    return {"status": "ok"}

@api.get("/predict")
def predict(val: int) -> dict:
    return {"result": val * 2}

app_env = FastAPIAppEnvironment(
    name="prediction-service",
    app=api,
    port=8080,
)

# Start serving locally in non-blocking mode
serve_ctx = flyte.with_servecontext(mode="local")
local_app = serve_ctx.serve(app_env)

# Use the synchronous ephemeral context manager to ensure clean activation and teardown
with local_app.ephemeral_ctx_sync():
    print(f"App endpoint: {local_app.endpoint}")
    response = httpx.get(f"{local_app.endpoint}/predict", params={"val": 10})
    print(response.json())  # {"result": 20}

# Once context exits, the server thread/process is deactivated
assert local_app.is_deactivated()

Understanding the AppHandle Protocol

Both local serving (_LocalApp from flyte._serve) and remote serving (App from flyte.remote) conform to the runtime-checkable protocol AppHandle:

Member	Return Type	Description
`name`	`str`	Name of the app environment
`endpoint`	`str`	Reachable base HTTP URL (e.g. `http://127.0.0.1:8080` locally, or public ingress URL remotely)
`url`	`str`	Reachable URL or Flyte console UI URL
`is_active()`	`bool`	Checks whether the app is currently running and healthy
`is_deactivated()`	`bool`	Checks whether the app process/thread is stopped or marked deactivated
`activate(wait=False)`	`AppHandle`	Starts the app or spins up resources; blocks if `wait=True`
`deactivate(wait=False)`	`AppHandle`	Gracefully terminates server threads/subprocesses or stops remote instances
`ephemeral_ctx()`	`AsyncContextManager`	Async context manager that activates on enter and deactivates on exit
`ephemeral_ctx_sync()`	`ContextManager`	Synchronous context manager that activates on enter and deactivates on exit

Because _LocalApp and App share this interface, test suites and calling workflows can interact with apps using identical code paths regardless of where the app is hosted.

Local Serving Architecture

When mode="local" is selected, _Serve._serve_local inspects the AppEnvironment definition to choose an execution strategy:

Server Function Daemon Thread (@app_env.server or FastAPIAppEnvironment): Runs the Python function or ASGI event loop in a background daemon thread named flyte-local-app-<name>. Startup functions registered on app_env._on_startup run within the loop before the main server starts.
Subprocess (command or args): Executes external processes (such as Django, Streamlit, or Triton) via subprocess.Popen(..., start_new_session=True).

During local initialization, _Serve populates process environment variables from app_env.parameters and user-specified env_vars, sets _RUN_MODE="local", and registers the app handle into a global active-apps registry. Signal handlers for SIGINT/SIGTERM and an atexit hook guarantee that background threads and subprocesses terminate when the parent Python process exits.

import flyte

serve_ctx = flyte.with_servecontext(
    mode="local",
    health_check_path="/health",
    health_check_timeout=2.0,
    health_check_interval=1.0,
    activate_timeout=30.0,
    deactivate_timeout=5.0,
    env_vars={"MODEL_PATH": "/opt/models/model.bin"},
)

local_app = serve_ctx.serve(app_env)
local_app.activate(wait=True)
try:
    print(f"App running at {local_app.endpoint}")
finally:
    local_app.deactivate(wait=True)

Remote Serving and Deployment

When mode="remote" is selected (or inferred because a Flyte client is initialized via flyte.init), _Serve._serve_remote executes the full deployment pipeline:

Deployment Planning: Resolves app dependencies and dependent environments using plan_deploy(app_env).
Image Building: Builds container images via build_images and caches image URIs.
Code Bundling: Packages project code based on copy_style ("loaded_modules", "all", or "none" via build_code_bundle, or pickled bundle via build_pkl_bundle).
Serialization: Creates a SerializationContext with computed version digests, image cache, and bundle specifications.
IDL Deployment: Deploys the app spec (flyteidl2.app.app_definition_pb2.App) to the remote Flyte backend using _deploy_app and App.create / App.replace.
Activation Watch: Watches backend deployment conditions until the app reaches DEPLOYMENT_STATUS_ACTIVATED and returns a flyte.remote.App handle.

import flyte

# Deploy remotely with custom cluster configuration and overrides
app = flyte.with_servecontext(
    mode="remote",
    project="my-project",
    domain="development",
    version="v1.0.0",
    cluster_pool="gpu-pool",
    env_vars={"LOG_LEVEL": "DEBUG"},
    parameter_values={"prediction-service": {"batch_size": "32"}},
).serve(app_env)

print(f"Remote ingress URL: {app.endpoint}")
print(f"Console link: {app.url}")

Accessing Runtime Metadata with AppContext

App endpoints and worker tasks can query their execution environment using flyte.app._context.ctx() (or flyte.app.ctx()):

from flyte.app._context import ctx

@api.get("/info")
def app_info():
    current_ctx = ctx()
    return {
        "mode": current_ctx.mode,               # 'local' or 'remote'
        "project": current_ctx.project,         # Execution project
        "domain": current_ctx.domain,           # Execution domain
        "raw_data_path": current_ctx.raw_data_path,  # Defaults to /tmp/flyte/raw_data in local mode
    }

When running in local multithreaded server functions, _Serve propagates raw_data_path to worker threads by calling set_raw_data_path() from flyte.app._context and setting the _FLYTE_APP_RAW_DATA_PATH environment variable.

Serving via the Command-Line Interface

The Flyte CLI provides the flyte serve command (implemented in flyte.cli._serve using AppFiles, AppPerFileGroup, and ServeAppCommand) to run app environments directly from Python source files.

Local Serving via CLI

To run a server locally on localhost, supply the --local flag. The CLI sets up the serve context, blocks until the health check passes, prints the local endpoint, and waits until you send Ctrl+C to trigger graceful deactivation:

flyte serve --local app_file.py app_env_name

Inject local environment variables with the -e / --env-var option:

flyte serve --local -e LOG_LEVEL=DEBUG -e DB_PORT=5432 app_file.py app_env_name

Remote Cluster Deployment via CLI

Without --local, flyte serve builds the required container images, uploads the code bundle, deploys the app definition to the backend, and displays the public endpoint and console URL:

flyte serve \
  --project flytesnacks \
  --domain development \
  --copy-style loaded_modules \
  --image app-image=ghcr.io/my-org/my-app:v1.0.0 \
  -e ENVIRONMENT=production \
  app_file.py app_env_name

CLI Serve Options Reference

--local: Run on localhost instead of deploying to the Flyte backend.
--project / --domain: Override project and domain target.
--copy-style: Choose bundle copy mechanism: loaded_modules, all, or none.
--root-dir: Override root source directory (useful for monorepos).
--image: Override image mapping (name=uri). Can be repeated.
-e, --env-var: Pass environment variables in KEY=VALUE format. Can be repeated.
--service-account: Specify the Kubernetes service account for remote containers.
--no-sync-local-sys-paths: Disable syncing local sys.path entries under root_dir.

Variations and Integration Patterns

Chaining Dependent Apps Locally

When testing microservice architectures locally, start upstream dependencies first and query their dynamic endpoints:

import httpx
import flyte

serve_ctx = flyte.with_servecontext(mode="local")

# Start upstream dependency
app1 = serve_ctx.serve(app1_env)
app1.activate(wait=True)

# Start downstream service that consumes app1
app2 = serve_ctx.serve(app2_env)
app2.activate(wait=True)

try:
    response = httpx.get(f"{app2.endpoint}/calculate", params={"val": 5})
    assert response.status_code == 200
finally:
    app2.deactivate(wait=True)
    app1.deactivate(wait=True)

Invoking Local Apps from Tasks

You can test Flyte tasks against locally running app environments using ephemeral_ctx and flyte.with_runcontext(mode="local"):

import asyncio
import httpx
import flyte

@task_env.task
async def fetch_predictions(inputs: list[int]) -> list[int]:
    results = []
    async with httpx.AsyncClient() as client:
        for x in inputs:
            resp = await client.get(f"{app_env.endpoint}/predict", params={"val": x})
            resp.raise_for_status()
            results.append(resp.json()["result"])
    return results

async def main():
    local_app = flyte.with_servecontext(mode="local").serve(app_env)
    async with local_app.ephemeral_ctx():
        # Execute task locally against the running server
        run_res = await flyte.with_runcontext(mode="local").run.aio(fetch_predictions, inputs=[1, 2, 3])
        print(run_res.outputs()[0])

asyncio.run(main())

Troubleshooting and Constraints

Custom Health Check Paths

Local readiness probing polls health_check_path (default: "/health") expecting an HTTP status code < 500. If your application routes health checks to a different path (such as "/healthz" or "/"), activate(wait=True) will poll until activate_timeout (default: 60s) and raise TimeoutError. Specify the correct path in with_servecontext:

local_app = flyte.with_servecontext(
    mode="local",
    health_check_path="/healthz",
    activate_timeout=15.0,
).serve(app_env)

Interactive / Pickle Serves with `include` Files

When serving interactively without source files (using pickled code bundles), setting file includes (Environment.include) is not supported and raises: ValueError: Environment.include is not supported in interactive/pkl serves. Serve from a file or remove include from the environment. Deploy from a Python file using flyte serve <file>.py <app_env> or use copy_style="loaded_modules" instead.

Log Streaming

The --follow / -f flag on flyte serve is reserved for log streaming. For app serving, remote logs should be inspected directly in the Flyte console URL provided in the serve output panel.

Serving Apps Programmatically​

Understanding the AppHandle Protocol​

Local Serving Architecture​

Remote Serving and Deployment​

Accessing Runtime Metadata with AppContext​

Serving via the Command-Line Interface​

Local Serving via CLI​

Remote Cluster Deployment via CLI​

CLI Serve Options Reference​

Variations and Integration Patterns​

Chaining Dependent Apps Locally​

Invoking Local Apps from Tasks​

Troubleshooting and Constraints​

Custom Health Check Paths​

Interactive / Pickle Serves with include Files​

Log Streaming​