hf_model

Store a HuggingFace model to remote storage.

This function downloads a model from the HuggingFace Hub and prefetches it to remote storage. It supports optional sharding using vLLM for large models.

The prefetch behavior follows this priority:

1. If the model isn't being sharded, stream files directly to remote storage.
2. If streaming fails, fall back to downloading a snapshot and uploading.
3. If sharding is configured, download locally, shard with vLLM, then upload.

def hf_model(
    repo: str,
    *,
    raw_data_path: str | None = None,
    artifact_name: str | None = None,
    architecture: str | None = None,
    task: str = "auto",
    modality: tuple[str, ...] = ("text",),
    serial_format: str | None = None,
    model_type: str | None = None,
    short_description: str | None = None,
    shard_config: ShardConfig | None = None,
    hf_token_key: str = "HF_TOKEN",
    resources: Resources = Resources(cpu="2", memory="8Gi", disk="50Gi"),
    force: int = 0,
) -> Run: ...

Store a HuggingFace model to remote storage. This function downloads a model from the HuggingFace Hub and prefetches it to remote storage. It supports optional sharding using vLLM for large models.

Parameters

Name	Type	Description
repo	str	The HuggingFace repository ID (e.g., 'meta-llama/Llama-2-7b-hf').
raw_data_path	`str	None` = None
artifact_name	`str	None` = None
architecture	`str	None` = None
task	str = "auto"	Model task (e.g., 'generate', 'classify', 'embed'). Default: 'auto'.
modality	tuple[str, ...] = ("text",)	Modalities supported by the model. Default: ('text',).
serial_format	`str	None` = None
model_type	`str	None` = None
short_description	`str	None` = None
shard_config	`ShardConfig	None` = None
hf_token_key	str = "HF_TOKEN"	Name of the secret containing the HuggingFace token. Default: 'HF_TOKEN'.
resources	Resources = Resources(cpu="2", memory="8Gi", disk="50Gi")	Resource requirements (CPU, memory, disk, etc.) for the prefetch task.
force	int = 0	Force re-prefetch. Increment to force a new prefetch. Default: 0.

Returns

Type	Description
Run	A Run object representing the prefetch task execution.