ContainerTask
This is an intermediate class that represents Flyte Tasks that run a container at execution time. This is the vast majority of tasks - the typical @task decorated tasks; for instance, all run a container. An example of something that doesn't run a container would be something like the Athena SQL task.
Attributes
| Attribute | Type | Description |
|---|---|---|
| MetadataFormat | Literal["JSON", "YAML", "PROTO"] | The format of the output file. This can be "JSON", "YAML", or "PROTO". |
Constructor
Signature
def ContainerTask(
name: str,
image: Union[str, [Image](../../image/image.md?sid=flyte__image_image)],
command: List[str],
inputs: Optional[Dict[str, Type]] = None,
arguments: Optional[List[str]] = None,
outputs: Optional[Dict[str, Type]] = None,
input_data_dir: str | pathlib.Path = "/var/inputs",
output_data_dir: str | pathlib.Path = "/var/outputs",
metadata_format: MetadataFormat = "JSON",
local_logs: bool = True,
block_network: bool = False
) - > null
Parameters
| Name | Type | Description |
|---|---|---|
| name | str | Name of the task |
| image | Union[str, [Image](../../image/image.md?sid=flyte__image_image)] | The container image to use for the task. This can be a string or an Image object. |
| command | List[str] | The command to run in the container. This can be a list of strings or a single string. |
| inputs | Optional[Dict[str, Type]] = None | The inputs to the task. This is a dictionary of input names to types. |
| arguments | Optional[List[str]] = None | The arguments to pass to the command. This is a list of strings. |
| outputs | Optional[Dict[str, Type]] = None | The outputs of the task. This is a dictionary of output names to types. |
| input_data_dir | `str | pathlib.Path` = "/var/inputs" |
| output_data_dir | `str | pathlib.Path` = "/var/outputs" |
| metadata_format | MetadataFormat = "JSON" | The format of the output file. This can be "JSON", "YAML", or "PROTO". |
| local_logs | bool = True | If True, logs will be printed to the console in the local execution. |
| block_network | bool = False | If True, blocks all outbound network access. Locally this sets Docker network_mode=none. On-cluster it applies the pod template sandboxed-pod-template. Defaults to False. |
Methods
execute()
@classmethod
def execute(
**kwargs: Any
) - > Any
Executes the container task locally using Docker. This method prepares the command, sets up volume bindings, runs the Docker container, waits for its completion, and retrieves the output.
Parameters
| Name | Type | Description |
|---|---|---|
| **kwargs | Any | Keyword arguments representing the inputs to the task, which are passed to the container. |
Returns
| Type | Description |
|---|---|
Any | The output of the container task, converted to the specified Python types. |
data_loading_config()
@classmethod
def data_loading_config(
sctx: [SerializationContext](../../models/serializationcontext.md?sid=flyte_models_serializationcontext)
) - > tasks_pb2.DataLoadingConfig
Generates the DataLoadingConfig protobuf message for the task. This configuration specifies how input and output data should be handled by the Flyte platform.
Parameters
| Name | Type | Description |
|---|---|---|
| sctx | [SerializationContext](../../models/serializationcontext.md?sid=flyte_models_serializationcontext) | The serialization context, providing information needed for protobuf serialization. |
Returns
| Type | Description |
|---|---|
tasks_pb2.DataLoadingConfig | A tasks_pb2.DataLoadingConfig object containing the data loading configuration for the task. |
container_args()
@classmethod
def container_args(
sctx: [SerializationContext](../../models/serializationcontext.md?sid=flyte_models_serializationcontext)
) - > List[str]
Returns the combined command and arguments that will be executed within the container. This provides the full command line for the container.
Parameters
| Name | Type | Description |
|---|---|---|
| sctx | [SerializationContext](../../models/serializationcontext.md?sid=flyte_models_serializationcontext) | The serialization context, though not directly used in this method, it's part of the method signature for consistency. |
Returns
| Type | Description |
|---|---|
List[str] | A list of strings representing the full command and arguments for the container. |