Getting Started with FastAPI Serving

Learn how to deploy and serve interactive FastAPI applications within your Flyte environment using the FastAPIAppEnvironment. This specialized environment handles the boilerplate of setting up a Uvicorn server, patching FastAPI's state for picklability, and automatically integrating with the Flyte UI.

By the end of this guide, you will have a functional FastAPI application running as a Flyte App, complete with automatic OpenAPI documentation and secure authentication passthrough.

Prerequisites

To follow this tutorial, ensure you have the following installed in your environment:

• flyte-sdk
• fastapi
• uvicorn

Step 1: Define the FastAPI Application

First, create a standard FastAPI application. You can define routes, models, and logic just as you would in a standalone FastAPI project.

from fastapi import FastAPI

app = FastAPI(title="Flyte Math Service")

@app.get("/add")
async def add(x: int, y: int) -> dict[str, int]:
    return {"result": x + y}

Step 2: Wrap the App in FastAPIAppEnvironment

To make this application deployable on Flyte, wrap the app instance in a FastAPIAppEnvironment. This class (found in flyte.app.extras) configures how Flyte should serve the application.

import flyte
from flyte.app.extras import FastAPIAppEnvironment

app_env = FastAPIAppEnvironment(
    name="math-service",
    app=app,
    # Define the container image with required dependencies
    image=flyte.Image.from_debian_base().with_pip_packages("fastapi", "uvicorn"),
)

The FastAPIAppEnvironment performs several critical tasks:

1. State Patching: It replaces the default Starlette State object with a PicklableState. This is necessary because the standard FastAPI state contains circular references that prevent it from being serialized (pickled) for Flyte deployment.
2. UI Integration: It automatically adds a link to the Flyte UI that points to /docs, giving you one-click access to the interactive Swagger UI.
3. Uvicorn Setup: It manages the underlying uvicorn server configuration, ensuring it listens on the correct port assigned by the Flyte platform.

Step 3: Enable Authentication Passthrough

If your FastAPI application needs to interact with the Flyte control plane (e.g., to launch tasks or fetch user info), you must pass the user's authentication credentials through the app.

Use the FastAPIPassthroughAuthMiddleware from flyte.app.extras to automatically extract headers and inject them into the Flyte context.

from flyte.app.extras import FastAPIPassthroughAuthMiddleware

# Add middleware to extract 'Authorization' and 'Cookie' headers
app.add_middleware(
    FastAPIPassthroughAuthMiddleware, 
    excluded_paths={"/health"}
)

@app.get("/me")
async def get_current_user():
    from flyte.remote import User
    # The middleware ensures 'User.get.aio()' has the correct auth context
    user = await User.get.aio()
    return {"subject": user.subject()}

When requires_auth=True is set in the FastAPIAppEnvironment, the Flyte platform handles the initial authentication at the gateway, and this middleware ensures those credentials are available within your route handlers.

Step 4: Local Serving and Testing

You can test your application locally before deploying it to a remote Flyte cluster. Use flyte.with_servecontext to simulate the serving environment.

if __name__ == "__main__":
    # Start the app in local mode
    local_app = flyte.with_servecontext(mode="local").serve(app_env)
    
    # Activate the server and wait for it to be ready
    local_app.activate(wait=True)
    
    print(f"FastAPI app is running at: {local_app.endpoint}")
    print(f"OpenAPI docs available at: {local_app.endpoint}/docs")

When you run this script, flyte-sdk starts a local Uvicorn server. You can visit the printed endpoint in your browser to interact with your API.

Complete Example

Combining the steps above, here is a complete script for a Flyte-powered FastAPI application:

from fastapi import FastAPI
import flyte
from flyte.app.extras import FastAPIAppEnvironment, FastAPIPassthroughAuthMiddleware

# 1. Initialize FastAPI
app = FastAPI(title="Flyte User Service")
app.add_middleware(FastAPIPassthroughAuthMiddleware)

@app.get("/whoami")
async def whoami():
    from flyte.remote import User
    user = await User.get.aio()
    return {"user": user.subject()}

# 2. Configure Flyte Environment
app_env = FastAPIAppEnvironment(
    name="user-service",
    app=app,
    requires_auth=True,
    image=flyte.Image.from_debian_base().with_pip_packages("fastapi", "uvicorn"),
)

# 3. Local Execution Logic
if __name__ == "__main__":
    local_app = flyte.with_servecontext(mode="local").serve(app_env)
    local_app.activate(wait=True)
    print(f"App ready at {local_app.endpoint}")

Next Steps

• Custom Uvicorn Config: If you need to tune performance, pass a uvicorn.Config object to the uvicorn_config parameter of FastAPIAppEnvironment.
• Deployment: Use flyte.deploy(app_env) to push your application to a remote Flyte cluster.
• Webhooks: For applications triggered by external events, explore FlyteWebhookAppEnvironment, which builds upon the FastAPI implementation.