ShardConfig

Configuration for model sharding.

Attributes

Attribute	Type	Description
engine	Literal["vllm"] = "vllm"	The sharding engine to use (currently only "vllm" is supported).
args	[VLLMShardArgs](vllmshardargs.md?sid=flyte_prefetch__hf_model_vllmshardargs) = VLLMShardArgs()	Arguments for the sharding engine.

---

Constructor

Signature

def ShardConfig(
    engine: Literal["vllm"] = "vllm",
    args: [VLLMShardArgs](vllmshardargs.md?sid=flyte_prefetch__hf_model_vllmshardargs) = Field(default_factory=VLLMShardArgs)
)

Parameters

Name	Type	Description
engine	Literal["vllm"] = "vllm"	The sharding engine to use (currently only "vllm" is supported).
args	[VLLMShardArgs](vllmshardargs.md?sid=flyte_prefetch__hf_model_vllmshardargs) = Field(default_factory=VLLMShardArgs)	Arguments for the sharding engine.

---