get_underlying_filesystem

Retrieves an fsspec filesystem instance for a given protocol or path, incorporating configured keyword arguments and additional user-provided arguments.

def get_underlying_filesystem(
    protocol: typing.Optional[str] = None,
    anonymous: bool = False,
    path: typing.Optional[str] = None,
    **kwargs: Any
) - > fsspec.AbstractFileSystem

Retrieves an fsspec filesystem instance based on the provided protocol or path. This function allows callers to interact with various storage backends (e.g., local, S3, GCS) in a unified way.

Parameters

Name	Type	Description
protocol	typing.Optional[str] = None	The filesystem protocol (e.g., 's3', 'file', 'gcs'). If None, the protocol is inferred from the 'path'.
anonymous	bool = False	A boolean indicating whether to access the filesystem anonymously. This is useful for public buckets or resources that do not require authentication.
path	typing.Optional[str] = None	A file path or URL from which to infer the protocol if 'protocol' is not provided. For example, 's3://my-bucket/file.txt' would infer 's3'.
**kwargs	Any	Additional keyword arguments to pass directly to the fsspec.filesystem constructor. These arguments can be used to configure specific filesystem options.

Returns

Type	Description
fsspec.AbstractFileSystem	An fsspec.AbstractFileSystem instance configured for the specified protocol.