Skip to main content

ContainerTask

This is an intermediate class that represents Flyte Tasks that run a container at execution time. This is the vast majority of tasks - the typical @task decorated tasks; for instance, all run a container. An example of something that doesn't run a container would be something like the Athena SQL task.

Attributes

AttributeTypeDescription
MetadataFormatLiteral["JSON", "YAML", "PROTO"]The format of the output file. This can be "JSON", "YAML", or "PROTO".

Constructor

Signature

def ContainerTask(
name: str,
image: Union[str, [Image](../../image/image.md?sid=flyte__image_image)],
command: List[str],
inputs: Optional[Dict[str, Type]] = None,
arguments: Optional[List[str]] = None,
outputs: Optional[Dict[str, Type]] = None,
input_data_dir: str | pathlib.Path = "/var/inputs",
output_data_dir: str | pathlib.Path = "/var/outputs",
metadata_format: MetadataFormat = "JSON",
local_logs: bool = True,
block_network: bool = False
) - > null

Parameters

NameTypeDescription
namestrName of the task
imageUnion[str, [Image](../../image/image.md?sid=flyte__image_image)]The container image to use for the task. This can be a string or an Image object.
commandList[str]The command to run in the container. This can be a list of strings or a single string.
inputsOptional[Dict[str, Type]] = NoneThe inputs to the task. This is a dictionary of input names to types.
argumentsOptional[List[str]] = NoneThe arguments to pass to the command. This is a list of strings.
outputsOptional[Dict[str, Type]] = NoneThe outputs of the task. This is a dictionary of output names to types.
input_data_dir`strpathlib.Path` = "/var/inputs"
output_data_dir`strpathlib.Path` = "/var/outputs"
metadata_formatMetadataFormat = "JSON"The format of the output file. This can be "JSON", "YAML", or "PROTO".
local_logsbool = TrueIf True, logs will be printed to the console in the local execution.
block_networkbool = FalseIf True, blocks all outbound network access. Locally this sets Docker network_mode=none. On-cluster it applies the pod template sandboxed-pod-template. Defaults to False.

Methods


execute()

@classmethod
def execute(
**kwargs: Any
) - > Any

Executes the container task locally using Docker. This method prepares the command, sets up volume bindings, runs the Docker container, waits for its completion, and retrieves the output.

Parameters

NameTypeDescription
**kwargsAnyKeyword arguments representing the inputs to the task, which are passed to the container.

Returns

TypeDescription
AnyThe output of the container task, converted to the specified Python types.

data_loading_config()

@classmethod
def data_loading_config(
sctx: [SerializationContext](../../models/serializationcontext.md?sid=flyte_models_serializationcontext)
) - > tasks_pb2.DataLoadingConfig

Generates the DataLoadingConfig protobuf message for the task. This configuration specifies how input and output data should be handled by the Flyte platform.

Parameters

NameTypeDescription
sctx[SerializationContext](../../models/serializationcontext.md?sid=flyte_models_serializationcontext)The serialization context, providing information needed for protobuf serialization.

Returns

TypeDescription
tasks_pb2.DataLoadingConfigA tasks_pb2.DataLoadingConfig object containing the data loading configuration for the task.

container_args()

@classmethod
def container_args(
sctx: [SerializationContext](../../models/serializationcontext.md?sid=flyte_models_serializationcontext)
) - > List[str]

Returns the combined command and arguments that will be executed within the container. This provides the full command line for the container.

Parameters

NameTypeDescription
sctx[SerializationContext](../../models/serializationcontext.md?sid=flyte_models_serializationcontext)The serialization context, though not directly used in this method, it's part of the method signature for consistency.

Returns

TypeDescription
List[str]A list of strings representing the full command and arguments for the container.