Skip to main content

App Parameters and Dynamic Inputs

App parameters as a serving-time data flow​

An app parameter can be a literal, a reference to storage, or a dependency whose value is resolved later. In flyte-sdk, configure all three through flyte.app.Parameter; use flyte.io.File and flyte.io.Dir for storage references, flyte.app.RunOutput for a task/run output, and flyte.app.AppEndpoint for an upstream app endpoint.

parameters=[
    flyte.app.Parameter(name="config", value="config.yaml"),
    flyte.app.Parameter(
        name="model",
        value=flyte.io.File.from_existing_remote("s3://bucket/model.pkl"),
    ),
    flyte.app.Parameter(
        name="data",
        value=flyte.io.Dir.from_existing_remote("s3://bucket/data/"),
    ),
]

Parameter is a dataclass with name, value, type, env_var, download, mount, and ignore_patterns. Its accepted values are strings, File, Dir, RunOutput, and AppEndpoint. download defaults to True, but literal strings are serialized as non-downloadable values. A parameter with value=None is allowed while configuring the environment, but it must be replaced by a serve-time override before deployment serialization.

File and Dir are references containing paths; they do not eagerly load file contents into memory. File.from_existing_remote() and Dir.from_existing_remote() create references without uploading:

model = flyte.io.File.from_existing_remote("s3://my-bucket/model.pkl")
data = flyte.io.Dir.from_existing_remote("s3://my-bucket/data/")

The storage types are also the values an app receives after serving has materialized its parameters. A File can be read with open() or open_sync(), while a Dir can be traversed with walk()/walk_sync(), listed with list_files()/list_files_sync(), or downloaded with download()/download_sync(). Dir.walk() yields File references and uses the configured filesystem; it does not perform an up-front full directory download.

Files, directories, mounts, and environment variables​

Use mount when the app needs a predictable local location. A mount causes SerializableParameter.from_parameter() to set dest and to force downloading for File, Dir, and file-like RunOutput values, even when download was not explicitly enabled. ignore_patterns is carried with the serialized directory parameter.

The examples/apps/file_parameter_app.py app declares two mounted files and one raw file reference:

app_env = FastAPIAppEnvironment(
    name="file-parameter-demo",
    app=app,
    description="Serves content from a task-produced file",
    image=image,
    resources=flyte.Resources(cpu=1, memory="512Mi"),
    requires_auth=False,
    parameters=[
        Parameter(name="data", type="file", mount=DATA_MOUNT_PATH, env_var=DATA_ENV_VAR),
        Parameter(name="data_with_dir", type="file", mount=DATA_DIR_MOUNT_PATH, env_var=DATA_DIR_ENV_VAR),
        Parameter(name="data_raw", type="file"),
    ],
    env_vars={"LOG_LEVEL": "10"},
)

At serving time, flyte._bin.serve.sync_parameters() starts with the remote path and constructs a flyte.io.File or flyte.io.Dir. If download is true, it calls storage.get() and replaces both the serializable value and materialized object with their local-path forms. Directory downloads pass recursive=True. For a file mount whose destination ends with the operating-system path separator, the runtime appends the source filename; otherwise it treats the destination as the target path. The destination must be writable.

If env_var is set, sync_parameters() injects the resulting serializable value into that environment variable. Therefore a downloaded file's variable contains its local path, while an un-downloaded file or directory variable contains its stored path. The parameter name is separately available to the app's materialized input handling.

To feed the file-parameter app from a task output, retrieve the output reference and supply it as a serve-time override:

run = flyte.run(produce_file)
print(f"Task run: {run.url}")
run.wait()

output_file: flyte.io.File = run.outputs()[0]
remote_path = output_file.path

app_handle = flyte.with_servecontext(
    parameter_values={
        app_env.name: {
            "data": flyte.io.File.from_existing_remote(remote_path),
            "data_with_dir": flyte.io.File.from_existing_remote(remote_path),
            "data_raw": flyte.io.File.from_existing_remote(remote_path),
        }
    }
).serve(app_env)

Deferred task outputs with RunOutput​

Use RunOutput when the app should resolve a task output rather than embedding a path known at deployment time. The model accepts exactly one of run_name and task_name:

# A specific run; getter=(0,) selects its first output.
run_output = flyte.app.RunOutput(
    type="file",
    run_name="my-run-123",
)

# The newest successful run for this task.
latest_output = flyte.app.RunOutput(
    type="file",
    task_name="env.my_task",
)

# A task version selected explicitly.
versioned_output = flyte.app.RunOutput(
    type="file",
    task_name="env.my_task",
    task_version="xyz",
)

# Resolve task metadata using the requested automatic version mode.
deployed_output = flyte.app.RunOutput(
    type="file",
    task_name="env.my_task",
    task_auto_version="latest",
)

RunOutput defaults getter to (0,). During materialization, the implementation obtains run outputs and applies every getter component as output[getter]. For a nested or named result, provide the tuple explicitly, for example getter=("result",) or getter=(0, "key").

With run_name, RunOutput calls Run.get.aio() and reads that run's details. With task_name and task_auto_version, it first fetches task details through Task.get(...).fetch.aio() and uses the resolved task version. With an explicit task_version and no automatic version, it uses that version directly. With neither version setting, it searches Run.listall.aio() for succeeded runs, sorts by created_at descending, and limits the search to one run. A missing task run becomes flyte.errors.ParameterMaterializationError. RunOutput rejects both identities being supplied, neither identity being supplied, and type="app_endpoint".

A model-serving app uses this dependency directly:

parameters=[
    flyte.app.Parameter(
        # 'model' matches the argument name in @env.on_startup below.
        # Union downloads the file and injects it as a flyte.io.File.
        name="model",
        value=flyte.app.RunOutput(
            task_name="cv-export.export_model_to_pt2",
            type="file",
        ),
        download=True,
    )
]

Its startup hook receives the downloaded reference and uses its local path:

@env.on_startup
async def startup(model: flyte.io.File):
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    logger.info("Loading model on device: %s", device)

    exported = torch.export.load(model.path)
    logger.info("Loaded ExportedProgram from %s", model.path)

Internally, _DelayedValue is the Pydantic base for RunOutput and AppEndpoint. Its materialize() method is abstract, while get() awaits materialization and checks that the result is a string, File, Dir, or AppEndpoint. The runtime helper _materialize_parameters_with_delayed_values() detects _DelayedValue values, awaits get(), and replaces the parameter value before app IDL translation. This is distinct from the serving executable's later file download step.

Upstream app endpoints​

Use AppEndpoint when an app needs the URL of another app. The endpoint is logically a string, but its lookup is deferred:

env2 = FastAPIAppEnvironment(
    name="app2-calls-app1",
    app=app2,
    image=image,
    resources=flyte.Resources(cpu=1, memory="512Mi"),
    requires_auth=True,
    parameters=[
        flyte.app.Parameter(
            name="app1_url",
            value=flyte.app.AppEndpoint(app_name="app1-is-called-by-app2"),
            env_var="APP1_URL",
        ),
    ],
    depends_on=[env1],
    env_vars={"LOG_LEVEL": "10"},
)

Declare depends_on=[env1] alongside the endpoint parameter when the upstream environment is part of the deployment. The example then reads the injected value with os.getenv("APP1_URL").

AppEndpoint.materialize() deliberately returns the AppEndpoint object. URL retrieval occurs in sync_parameters(): the serving path reconstructs the JSON value, calls _retrieve_endpoint(), and replaces the serializable value with the URL. For public=True, _retrieve_endpoint() calls flyte.remote.App.get.aio(app_name) and returns app.endpoint. For a private endpoint, it reads INTERNAL_APP_ENDPOINT_PATTERN and formats it with app_fqdn=app_name; if that variable is unset, it raises ValueError. AppEndpoint is never downloaded and SerializableParameter.from_parameter() always records download=False for it.

Deployment serialization and serving resolution​

The lifecycle has two representations:

Parameter values
    -> SerializableParameter.from_parameter()
    -> SerializableParameterCollection.from_parameters()
    -> JSON -> gzip (mtime=0) -> Base64: collection.to_transport
    -> flyte._bin.serve.sync_parameters()
    -> serializable values + materialized values + environment variables

AppEnvironment._serialize_parameters() chooses either serve-time overrides or the environment's configured parameters, calls SerializableParameterCollection.from_parameters(), and returns the cached to_transport property. to_transport is a property, not a method. It JSON-serializes the collection, gzip-compresses it with deterministic mtime=0, and Base64-encodes the compressed bytes. Reverse the operation with:

collection = SerializableParameterCollection.from_parameters(parameters)
transport = collection.to_transport
restored = SerializableParameterCollection.from_transport(transport)

SerializableParameter.from_parameter() performs the type-specific conversion. Strings remain strings and are forced to download=False; File and Dir become their .path values; delayed values become JSON through model_dump_json(); and AppEndpoint is marked as app_endpoint with downloading disabled. An unset value raises ValueError, stating that a concrete deployment value or a parameter_values serve-time value is required.

On the serving side, sync_parameters() maintains separate dictionaries for serializable and materialized values. A non-downloaded file remains a File(path=remote_path) materialized reference. A downloaded file becomes File(path=local_path). The same distinction applies to directories. An endpoint becomes a URL string in both the serialized value and the environment-variable injection.

Serve-time overrides​

Overrides are nested by app environment name and then parameter name:

flyte.with_servecontext(
    parameter_values={
        app_env.name: {
            "data": flyte.io.File.from_existing_remote(remote_path),
        }
    }
).serve(app_env)

flyte._serve._Serve applies these values before AppEnvironment._serialize_parameters() serializes the selected parameter list. An override may be a string, flyte.io.File, or flyte.io.Dir. This is the required route for parameters configured with value=None; passing such a parameter unchanged to SerializableParameter.from_parameter() raises ValueError rather than producing an incomplete wire value.

Operational constraints and edge cases​

  • Validate env_var names against shell syntax. Parameter.__post_init__() accepts only names matching ^[_a-zA-Z][_a-zA-Z0-9]*$; names containing dashes or starting with digits are rejected.
  • Initialize Flyte before remote-dependent operations. RunOutput.materialize(), private/public endpoint retrieval, File.new_remote(), and other remote storage operations are guarded by requires_initialization or use initialized storage context.
  • Use File.from_existing_remote() and Dir.from_existing_remote() when you already have a remote path. They create references; they do not upload data. File.new_remote() instead allocates a new path from the initialized raw-data context, after which open("wb") or open_sync("wb") can write it.
  • Keep mount destinations writable. A file mount ending in the OS separator receives the source filename beneath that directory; a non-directory-looking destination is used directly.
  • Remember that a default RunOutput.getter of (0,) selects the first output, and task-name lookup considers only succeeded runs. Supply an explicit getter and version information when those defaults do not identify the desired output.
  • Private AppEndpoint resolution requires INTERNAL_APP_ENDPOINT_PATTERN, such as http://{app_fqdn}.internal:8080; the implementation formats the {app_fqdn} field with the upstream app name.
  • Local serving is not identical to the remote serving path: _Serve local mode stringifies non-string parameter values rather than performing the full remote download/materialization flow.
  • Runtime parameter lookup is separate from the materialized startup inputs. get_parameter() reads the JSON file named flyte-parameters.json through the environment variable used by the cached loader; without that environment setting it raises ValueError("Parameters are not mounted"), and an unknown name raises KeyError.
  • For optional directory outputs, flyte.io.Dir.empty() returns a sentinel Dir; check dir.is_empty rather than relying on isinstance(dir, EmptyDir). The sentinel is used because the documented Optional[Dir] round-trip does not work through the serializable type engine.