Source code for dantro.data_loaders

"""This module implements loaders mixin classes for use with the

All these mixin classes should follow the following signature:

.. testcode::

    from dantro.data_loaders import add_loader
    from dantro.base import BaseDataContainer

    class TheTargetContainerClass(BaseDataContainer):

    class LoadernameLoaderMixin:

        def _load_loadername(filepath: str, *, TargetCls: type):
            # ...
            return TargetCls(...)

As ensured by the :py:func:`~dantro.data_loaders._registry.add_loader`
decorator, each ``_load_loadername`` method gets supplied with the path to a
file and the ``TargetCls`` argument, which can be called to create an object
of the correct type and name.
In addition, the decorator registers the load function with the dantro
:py:data:`~dantro.data_loaders._registry.DATA_LOADERS` registry, making it
available to :py:class:`~dantro.data_mngr.DataManager` instances that do not
have the mixin added.

By default, and to decouple the loader from the container, it should be
considered to be a static method; in other words: the first positional argument
should ideally *not* be ``self``!
If ``self`` is required for some reason, set the ``omit_self`` option of the
decorator to ``False``, making it a regular (instead of a static) method.

from ._registry import add_loader
from .fspath import FSPathLoaderMixin
from .hdf5 import Hdf5LoaderMixin
from .numpy import NumpyLoaderMixin
from .pandas import PandasLoaderMixin
from .pickle import PickleLoaderMixin
from .text import TextLoaderMixin
from .xarray import XarrayLoaderMixin
from .yaml import YamlLoaderMixin

[docs]class AllAvailableLoadersMixin( TextLoaderMixin, FSPathLoaderMixin, YamlLoaderMixin, PickleLoaderMixin, Hdf5LoaderMixin, XarrayLoaderMixin, PandasLoaderMixin, NumpyLoaderMixin, ): """A mixin bundling all data loaders that are available in dantro. See the individual mixins for a more detailed documentation. If you want all these loaders available in your data manager, inherit from this mixin class and :py:class:`~dantro.data_mngr.DataManager`: .. testcode:: import dantro class MyDataManager( dantro.data_loaders.AllAvailableLoadersMixin, dantro.DataManager, ): pass """
# fmt: off LOADER_BY_FILE_EXT = { "txt": "text", "log": "text", "yml": "yml", "yaml": "yaml", "pickle": "pickle", "pkl": "pkl", "hdf5": "hdf5", "h5": "hdf5", "nc": "xr_dataarray", "netcdf": "xr_dataarray", "nc_da": "xr_dataarray", "xrdc": "xr_dataarray", "nc_ds": "xr_dataset", "npy": "numpy_binary", "np_txt": "numpy_txt", "csv": "pandas_csv", } """A map of file extensions to preferred loader names"""