get_stream

Get a stream of data from a remote location. This is useful for downloading streaming data from a remote location. Example usage:

import flyte.storage as storage
async for chunk in storage.get_stream(path="s3://my_bucket/my_file.txt"):
    process(chunk)

:param path: Path to the remote location where the data will be downloaded. :param kwargs: Additional arguments to be passed to the underlying filesystem. :param chunk_size: Size of each chunk to be read from the file. :return: An async iterator that yields chunks of bytes.

async def get_stream(path: str, chunk_size=10 * 2**20, **kwargs) -> AsyncGenerator[bytes, None]: ...

Get a stream of data from a remote location. This is useful for downloading streaming data from a remote location.

Parameters

Name	Type	Description
path	str	Path to the remote location where the data will be downloaded.
chunk_size	int = 10485760	Size of each chunk to be read from the file.
kwargs	Any	Additional arguments to be passed to the underlying filesystem.

Returns

Type	Description
AsyncGenerator[bytes, None]	An async iterator that yields chunks of bytes.