AsyncConnector

This is the base class for all async connectors, and it defines the interface that all connectors must implement. The connector service is responsible for invoking connectors. The executor will communicate with the connector service to create tasks, get the status of tasks, and delete tasks.

Attributes

Attribute	Type	Description
name	str = "Async Connector"	The name of the connector, used for identification and registration in the ConnectorRegistry.
task_type_name	str	The name of the task type that this connector handles, used by the Connector Service to look up the appropriate connector.
task_type_version	int = 0	The version of the task type that this connector handles, used by the Connector Service to look up the appropriate connector.
metadata_type	[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The type of resource metadata that this connector uses to represent tasks.

---

Methods

---

create()

@classmethod
def create(
    task_template: tasks_pb2.TaskTemplate,
    output_prefix: str,
    inputs: Optional[Dict[str, typing.Any]] = None,
    task_execution_metadata: Optional[TaskExecutionMetadata] = None,
    kwargs: **kwargs
) - > [ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)

Return a resource meta that can be used to get the status of the task.

Parameters

Name	Type	Description
task_template	tasks_pb2.TaskTemplate	The template defining the task to be created, including its configuration and specifications.
output_prefix	str	A string prefix used to construct the output location for the task's results.
inputs	Optional[Dict[str, typing.Any]] = None	Optional dictionary of input values required by the task.
task_execution_metadata	Optional[TaskExecutionMetadata] = None	Optional metadata related to the task's execution context.
kwargs	**kwargs	Additional keyword arguments that may be specific to the connector implementation.

Returns

Type	Description
[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	A ResourceMeta object that uniquely identifies the created task and can be used to query its status.

---

get()

@classmethod
def get(
    resource_meta: [ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta),
    kwargs: **kwargs
) - > [Resource](resource.md?sid=flyte_connectors__connector_resource)

Return the status of the task, and return the outputs in some cases. For example, bigquery job can't write the structured dataset to the output location, so it returns the output literals to the propeller, and the propeller will write the structured dataset to the blob store.

Parameters

Name	Type	Description
resource_meta	[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The ResourceMeta object identifying the task to retrieve.
kwargs	**kwargs	Additional keyword arguments that may be specific to the connector implementation.

Returns

Type	Description
[Resource](resource.md?sid=flyte_connectors__connector_resource)	A Resource object containing the current status of the task and potentially its outputs.

---

delete()

@classmethod
def delete(
    resource_meta: [ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta),
    kwargs: **kwargs
)

Delete the task. This call should be idempotent. It should raise an error if fails to delete the task.

Parameters

Name	Type	Description
resource_meta	[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The ResourceMeta object identifying the task to delete.
kwargs	**kwargs	Additional keyword arguments that may be specific to the connector implementation.

---

get_metrics()

@classmethod
def get_metrics(
    resource_meta: [ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta),
    kwargs: **kwargs
) - > GetTaskMetricsResponse

Return the metrics for the task.

Parameters

Name	Type	Description
resource_meta	[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The ResourceMeta object identifying the task for which to retrieve metrics.
kwargs	**kwargs	Additional keyword arguments that may be specific to the connector implementation.

Returns

Type	Description
GetTaskMetricsResponse	A GetTaskMetricsResponse object containing performance and operational metrics for the specified task.

---

get_logs()

@classmethod
def get_logs(
    resource_meta: [ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta),
    kwargs: **kwargs
) - > GetTaskLogsResponse

Return the metrics for the task.

Parameters

Name	Type	Description
resource_meta	[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The ResourceMeta object identifying the task for which to retrieve logs.
kwargs	**kwargs	Additional keyword arguments that may be specific to the connector implementation.

Returns

Type	Description
GetTaskLogsResponse	A GetTaskLogsResponse object containing log entries for the specified task.

---