Checkpoint helper using flyte.io.File for all checkpoint blob I/O (load/save, async and sync). Remote paths are a single object.
Attributes
| Attribute | Type | Description |
|---|
| remote_destination | str | Object-store prefix where flyte.Checkpoint.save writes. |
| remote_source | Optional[str] | Object-store prefix for the previous attempt's checkpoint, if any. |
| path | pathlib.Path | Local directory for reading checkpoint files. |
Constructor
Signature
def Checkpoint(
checkpoint_dest: str,
checkpoint_src: str | None = None
) - > null
Parameters
| Name | Type | Description |
|---|
| checkpoint_dest | str | The destination path for the checkpoint. |
| checkpoint_src | `str | None` = None |
Methods
remote_destination()
@classmethod
def remote_destination() - > str
Object-store prefix where flyte.Checkpoint.save writes.
Returns
| Type | Description |
|---|
str | The object-store prefix for saving checkpoints. |
remote_source()
@classmethod
def remote_source() - > Optional[str]
Object-store prefix for the previous attempt's checkpoint, if any.
Returns
| Type | Description |
|---|
Optional[str] | The object-store prefix for loading previous checkpoints, or None if not set. |
path()
@classmethod
def path() - > pathlib.Path
Local directory for reading checkpoint files.
Returns
| Type | Description |
|---|
pathlib.Path | A pathlib.Path object representing the local directory where checkpoint files are stored. |
prev_exists()
@classmethod
def prev_exists() - > bool
Checks if a previous checkpoint source URI is available. This indicates whether there's a checkpoint from a prior attempt to potentially load.
Returns
| Type | Description |
|---|
bool | True if a remote source for a previous checkpoint exists, False otherwise. |
load()
@classmethod
def load() - > Optional[pathlib.Path]
Loads the previous checkpoint asynchronously if one exists and has not been restored yet. This method is suitable for async tasks.
Returns
| Type | Description |
|---|
Optional[pathlib.Path] | The pathlib.Path to the loaded checkpoint data, or None if no previous checkpoint was available or successfully loaded. |
load_sync()
@classmethod
def load_sync() - > Optional[pathlib.Path]
Loads the previous checkpoint synchronously if one exists and has not been restored yet. This method is suitable for ordinary def tasks.
Returns
| Type | Description |
|---|
Optional[pathlib.Path] | The pathlib.Path to the loaded checkpoint data, or None if no previous checkpoint was available or successfully loaded. |
save()
@classmethod
def save(
data: pathlib.Path | str | bytes
) - > None
Asynchronously saves the provided data (a file path, directory path, or bytes) to the remote checkpoint destination. Directories are saved as gzipped tarballs, while files and bytes are uploaded directly.
Parameters
| Name | Type | Description |
|---|
| data | `pathlib.Path | str |
Returns
| Type | Description |
|---|
None | This method does not return any value. |
save_sync()
@classmethod
def save_sync(
data: pathlib.Path | str | bytes
) - > None
Synchronously saves the provided data (a file path, directory path, or bytes) to the remote checkpoint destination. Directories are saved as gzipped tarballs, while files and bytes are uploaded directly.
Parameters
| Name | Type | Description |
|---|
| data | `pathlib.Path | str |
Returns
| Type | Description |
|---|
None | This method does not return any value. |