dantro.containers package#

Implements BaseDataContainer specializations.

isort:skip_file

Submodules#

dantro.containers._registry module#

Implements a registry for dantro container types based on ObjectRegistry.

class ContainerRegistry[source]#

Bases: dantro._registry.ObjectRegistry

_DESC: str = 'container'#

A description string for the entries of this registry

_SKIP: bool = False#

Default behavior for skip_existing argument

_OVERWRITE: bool = True#

Default behavior for overwrite_existing argument

_EXPECTED_TYPE: Optional[Union[tuple, type]] = (<class 'type'>,)#

If set, will check for expected types

_check_object(obj: Any) None[source]#

Checks whether the object is valid.

_register_via_decorator(obj: type, name: Optional[str] = None, **kws)[source]#

Performs the registration operations when the decorator is used to register an object.

__contains__(obj_or_key: Union[Any, str]) bool#

Whether the given argument is part of the keys or values of this registry.

_decorator(arg: Optional[Union[Any, str]] = None, /, **kws)#

Method that can be used as a decorator for registering objects with this registry.

Parameters
  • arg (Union[Any, str], optional) – The name that should be used or the object that is to be added. If not a string, this refers to the @is_container call syntax

  • **kws – Passed to register()

_determine_name(obj: Any, *, name: Optional[str]) str#

Determines the object name, using a potentially given name

property classname: str#
property desc: str#
items()#
keys()#
register(obj: Any, name: Optional[str] = None, *, skip_existing: Optional[bool] = None, overwrite_existing: Optional[bool] = None) str#

Adds an entry to the registry.

Parameters
  • obj (Any) – The object to add to the registry.

  • name (Optional[str], optional) – The name to use. If not given, will deduce a name from the given object.

  • skip_existing (bool, optional) – Whether to skip registration if an object of that name already exists. If None, the classes default behavior (see _SKIP) is used.

  • overwrite_existing (bool, optional) – Whether to overwrite an entry if an object with that name already exists. If None, the classes default behavior (see _OVERWRITE) is used.

values()#
CONTAINERS = <dantro.containers._registry.ContainerRegistry object>#

The dantro data container registry object.

register_container(Cls: type, name: str, *, skip_existing: bool = False, overwrite_existing: bool = True) None[source]#

Adds an entry to the shared container registry.

Parameters
  • Cls (type) – The class that is to be registered as a container.

  • name (str) – The name to use for registration.

  • skip_existing (bool, optional) – Whether to skip registration if the container name is already registered. This suppresses the ValueError raised on existing container name.

  • overwrite_existing (bool, optional) – Whether to overwrite a potentially already existing container of the same name. If set, this takes precedence over skip_existing.

is_container(arg: Optional[Union[type, str]] = None, /, **kws)[source]#

Decorator for registering containers with the container type registry.

As an alternative to register_container(), this decorator can be used to register a container right where its defined:

from dantro.containers import BaseDataContainer, is_container

# Container name deduced from class name
@is_container
class MyDataContainer(BaseDataContainer):
    # ... do stuff here ...
    pass

# Custom container name
@is_container("my_container")
class MyDataContainer(BaseDataContainer):
    # ... do stuff here ...
    pass

# Overwriting a registered container of the same name
@is_container("my_container", overwrite_existing=True)
class MyDataContainer(BaseDataContainer):
    # ... do stuff here ...
    pass

dantro.containers.general module#

This module implements general specialisations of the BaseDataContainer

class ObjectContainer(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)[source]#

Bases: dantro.mixins.base.ItemAccessMixin, dantro.base.BaseDataContainer

Generically stores any Python object

This allows item access, but not more.

_format_info() str[source]#

A __format__ helper function: returns info about the stored data

_ATTRS_CLS#

alias of dantro.base.BaseDataAttrs

__delitem__(key)#

Deletes an item

__eq__(other) bool#

Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the _data and _attrs attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.

If types do not match exactly, NotImplemented is returned, thus referring the comparison to the other side of the ==.

__format__(spec_str: str) str#

Creates a formatted string from the given specification.

Invokes further methods which are prefixed by _format_.

__getitem__(key)#

Returns an item.

__init__(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)#

Initialize a BaseDataContainer, which can store data and attributes.

Parameters
  • name (str) – The name of this data container

  • data (Any) – The data to store in this container

  • attrs (Dict[str, Any], optional) – A mapping that is stored as data attributes.

  • parent (AbstractDataGroup, optional) – If known, the parent group, which can be used to extract information during initialization. Note that linking occurs only after the container was added to the parent group using the add() method. The child object is not responsible of linking or adding itself to the group.

__repr__() str#

Same as __str__

__setitem__(key, val)#

Sets an item.

__sizeof__() int#

Returns the size of the data (in bytes) stored in this container’s data and its attributes.

Note that this value is approximate. It is computed by calling the sys.getsizeof() function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.

Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.

__str__() str#

An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.

_abc_impl = <_abc._abc_data object>#
_attrs = None#

The attribute that data attributes will be stored to

_check_data(data: Any) None#

This method can be used to check the data provided to this container

It is called before the data is stored in the __init__ method and should raise an exception or create a warning if the data is not as desired.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Note

The CheckDataMixin provides a generalised implementation of this method to perform some type checks and react to unexpected types.

Parameters

data (Any) – The data to check

_check_name(new_name: str) None#

Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Parameters

new_name (str) – The new name, which is to be checked.

_format_cls_name() str#

A __format__ helper function: returns the class name

_format_logstr() str#

A __format__ helper function: returns the log string, a combination of class name and name

_format_name() str#

A __format__ helper function: returns the name

_format_path() str#

A __format__ helper function: returns the path to this container

_item_access_convert_list_key(key)#

If given something that is not a list, just return that key

property attrs#

The container attributes.

property classname: str#

Returns the name of this DataContainer-derived class

property data: Any#

The stored data.

property logstr: str#

Returns the classname and name of this object

property name: str#

The name of this DataContainer-derived object.

property parent#

The associated parent of this container or group

property path: str#

The path to get to this container or group from some root path

class PassthroughContainer(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)[source]#

Bases: dantro.mixins.general.ForwardAttrsToDataMixin, dantro.containers.general.ObjectContainer

An object container that forwards all attribute calls to .data

FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#

Attributes to not forward. Evaluated after FORWARD_ATTR_ONLY

FORWARD_ATTR_ONLY: Sequence[str] = None#

If set, the only attributes to be forwarded

FORWARD_ATTR_TO: str = 'data'#

The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if __getattr__ was not called.

_ATTRS_CLS#

alias of dantro.base.BaseDataAttrs

__delitem__(key)#

Deletes an item

__eq__(other) bool#

Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the _data and _attrs attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.

If types do not match exactly, NotImplemented is returned, thus referring the comparison to the other side of the ==.

__format__(spec_str: str) str#

Creates a formatted string from the given specification.

Invokes further methods which are prefixed by _format_.

__getattr__(attr_name: str)#

Forward attributes that were not available in this class to some other attribute of the group or container.

Parameters

attr_name (str) – The name of the attribute that was tried to be accessed but was not available in self.

Returns

The attribute attr_name of getattr(self, self.FORWARD_ATTR_TO)

__getitem__(key)#

Returns an item.

__getstate__() dict#

Returns the object’s __dict__

__init__(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)#

Initialize a BaseDataContainer, which can store data and attributes.

Parameters
  • name (str) – The name of this data container

  • data (Any) – The data to store in this container

  • attrs (Dict[str, Any], optional) – A mapping that is stored as data attributes.

  • parent (AbstractDataGroup, optional) – If known, the parent group, which can be used to extract information during initialization. Note that linking occurs only after the container was added to the parent group using the add() method. The child object is not responsible of linking or adding itself to the group.

__repr__() str#

Same as __str__

__setitem__(key, val)#

Sets an item.

__setstate__(d: dict)#

Sets the object’s __dict__ to the given one

__sizeof__() int#

Returns the size of the data (in bytes) stored in this container’s data and its attributes.

Note that this value is approximate. It is computed by calling the sys.getsizeof() function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.

Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.

__str__() str#

An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.

_abc_impl = <_abc._abc_data object>#
_attrs = None#

The attribute that data attributes will be stored to

_check_data(data: Any) None#

This method can be used to check the data provided to this container

It is called before the data is stored in the __init__ method and should raise an exception or create a warning if the data is not as desired.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Note

The CheckDataMixin provides a generalised implementation of this method to perform some type checks and react to unexpected types.

Parameters

data (Any) – The data to check

_check_name(new_name: str) None#

Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Parameters

new_name (str) – The new name, which is to be checked.

_format_cls_name() str#

A __format__ helper function: returns the class name

_format_info() str#

A __format__ helper function: returns info about the stored data

_format_logstr() str#

A __format__ helper function: returns the log string, a combination of class name and name

_format_name() str#

A __format__ helper function: returns the name

_format_path() str#

A __format__ helper function: returns the path to this container

_forward_attr_get_forwarding_target()#

Get the object that the attribute call is to be forwarded to

_forward_attr_post_hook(attr)#

Invoked before attribute forwarding occurs

_forward_attr_pre_hook(attr_name: Optional[str] = None)#

Invoked before attribute forwarding occurs

_item_access_convert_list_key(key)#

If given something that is not a list, just return that key

property attrs#

The container attributes.

property classname: str#

Returns the name of this DataContainer-derived class

property data: Any#

The stored data.

property logstr: str#

Returns the classname and name of this object

property name: str#

The name of this DataContainer-derived object.

property parent#

The associated parent of this container or group

property path: str#

The path to get to this container or group from some root path

class MutableSequenceContainer(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)[source]#

Bases: dantro.mixins.base.CheckDataMixin, dantro.mixins.base.ItemAccessMixin, dantro.mixins.base.CollectionMixin, dantro.base.BaseDataContainer, collections.abc.MutableSequence

The MutableSequenceContainer stores data that is sequence-like

DATA_EXPECTED_TYPES: tuple = (<class 'collections.abc.MutableSequence'>, <class 'list'>)#

Which types to allow. If None, all types are allowed.

DATA_ALLOW_PROXY: bool = False#

Whether to allow all proxy types, i.e. classes derived from AbstractDataProxy.

DATA_UNEXPECTED_ACTION = 'warn'#

The action to take when an unexpected type was supplied. Can be: raise, warn, ignore.

insert(idx: int, val) None[source]#

Insert an item at a given position. The first argument is the index of the element before which to insert, so a.insert(0, x) inserts at the front of the list, and a.insert(len(a), x) is equivalent to a.append(x).

Parameters
  • idx (int) – The index before which to insert

  • val – The value to insert

_ATTRS_CLS#

alias of dantro.base.BaseDataAttrs

__contains__(key) bool#

Whether the given key is contained in the items.

__delitem__(key)#

Deletes an item

__eq__(other) bool#

Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the _data and _attrs attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.

If types do not match exactly, NotImplemented is returned, thus referring the comparison to the other side of the ==.

__format__(spec_str: str) str#

Creates a formatted string from the given specification.

Invokes further methods which are prefixed by _format_.

__getitem__(key)#

Returns an item.

__init__(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)#

Initialize a BaseDataContainer, which can store data and attributes.

Parameters
  • name (str) – The name of this data container

  • data (Any) – The data to store in this container

  • attrs (Dict[str, Any], optional) – A mapping that is stored as data attributes.

  • parent (AbstractDataGroup, optional) – If known, the parent group, which can be used to extract information during initialization. Note that linking occurs only after the container was added to the parent group using the add() method. The child object is not responsible of linking or adding itself to the group.

__iter__()#

Iterates over the items.

__len__() int#

The number of items.

__repr__() str#

Same as __str__

__setitem__(key, val)#

Sets an item.

__sizeof__() int#

Returns the size of the data (in bytes) stored in this container’s data and its attributes.

Note that this value is approximate. It is computed by calling the sys.getsizeof() function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.

Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.

__str__() str#

An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.

_abc_impl = <_abc._abc_data object>#
_attrs = None#

The attribute that data attributes will be stored to

_check_data(data) None#

A general method to check the received data for its type

Parameters

data – The data to check

Raises
  • TypeError – If the type was unexpected and the action was ‘raise’

  • ValueError – Illegal value for DATA_UNEXPECTED_ACTION class variable

Returns

None

_check_name(new_name: str) None#

Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Parameters

new_name (str) – The new name, which is to be checked.

_format_cls_name() str#

A __format__ helper function: returns the class name

_format_info() str#

A __format__ helper function: returns info about the content of this data container.

_format_logstr() str#

A __format__ helper function: returns the log string, a combination of class name and name

_format_name() str#

A __format__ helper function: returns the name

_format_path() str#

A __format__ helper function: returns the path to this container

_item_access_convert_list_key(key)#

If given something that is not a list, just return that key

append(value)#

S.append(value) – append value to the end of the sequence

property attrs#

The container attributes.

property classname: str#

Returns the name of this DataContainer-derived class

clear() None -- remove all items from S#
count(value) integer -- return number of occurrences of value#
property data: Any#

The stored data.

extend(values)#

S.extend(iterable) – extend sequence by appending elements from the iterable

index(value[, start[, stop]]) integer -- return first index of value.#

Raises ValueError if the value is not present.

Supporting start and stop arguments is optional, but recommended.

property logstr: str#

Returns the classname and name of this object

property name: str#

The name of this DataContainer-derived object.

property parent#

The associated parent of this container or group

property path: str#

The path to get to this container or group from some root path

pop([index]) item -- remove and return item at index (default last).#

Raise IndexError if list is empty or index is out of range.

remove(value)#

S.remove(value) – remove first occurrence of value. Raise ValueError if the value is not present.

reverse()#

S.reverse() – reverse IN PLACE

class MutableMappingContainer(*, name: str, data=None, **dc_kwargs)[source]#

Bases: dantro.mixins.base.CheckDataMixin, dantro.mixins.base.MappingAccessMixin, dantro.base.BaseDataContainer, collections.abc.MutableMapping

The MutableMappingContainer stores mutable mapping data, e.g. dicts

DATA_EXPECTED_TYPES: tuple = (<class 'collections.abc.MutableMapping'>, <class 'dict'>)#

Which types to allow. If None, all types are allowed.

DATA_ALLOW_PROXY: bool = False#

Whether to allow all proxy types, i.e. classes derived from AbstractDataProxy.

DATA_UNEXPECTED_ACTION = 'warn'#

The action to take when an unexpected type was supplied. Can be: raise, warn, ignore.

__init__(*, name: str, data=None, **dc_kwargs)[source]#

Initialize a MutableMappingContainer, storing mapping data.

Parameters
  • name (str) – The name of this container

  • data – The mapping-like data to store. If not given, an empty dict is created

  • **dc_kwargs – Additional arguments for container initialization

_ATTRS_CLS#

alias of dantro.base.BaseDataAttrs

__contains__(key) bool#

Whether the given key is contained in the items.

__delitem__(key)#

Deletes an item

__eq__(other) bool#

Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the _data and _attrs attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.

If types do not match exactly, NotImplemented is returned, thus referring the comparison to the other side of the ==.

__format__(spec_str: str) str#

Creates a formatted string from the given specification.

Invokes further methods which are prefixed by _format_.

__getitem__(key)#

Returns an item.

__iter__()#

Iterates over the items.

__len__() int#

The number of items.

__repr__() str#

Same as __str__

__setitem__(key, val)#

Sets an item.

__sizeof__() int#

Returns the size of the data (in bytes) stored in this container’s data and its attributes.

Note that this value is approximate. It is computed by calling the sys.getsizeof() function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.

Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.

__str__() str#

An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.

_abc_impl = <_abc._abc_data object>#
_attrs = None#

The attribute that data attributes will be stored to

_check_data(data) None#

A general method to check the received data for its type

Parameters

data – The data to check

Raises
  • TypeError – If the type was unexpected and the action was ‘raise’

  • ValueError – Illegal value for DATA_UNEXPECTED_ACTION class variable

Returns

None

_check_name(new_name: str) None#

Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Parameters

new_name (str) – The new name, which is to be checked.

_format_cls_name() str#

A __format__ helper function: returns the class name

_format_info() str#

A __format__ helper function: returns info about the content of this data container.

_format_logstr() str#

A __format__ helper function: returns the log string, a combination of class name and name

_format_name() str#

A __format__ helper function: returns the name

_format_path() str#

A __format__ helper function: returns the path to this container

_item_access_convert_list_key(key)#

If given something that is not a list, just return that key

property attrs#

The container attributes.

property classname: str#

Returns the name of this DataContainer-derived class

clear() None.  Remove all items from D.#
property data: Any#

The stored data.

get(key, default=None)#

Return the value at key, or default if key is not available.

items()#

Returns an iterator over data’s (key, value) tuples

keys()#

Returns an iterator over the data’s keys.

property logstr: str#

Returns the classname and name of this object

property name: str#

The name of this DataContainer-derived object.

property parent#

The associated parent of this container or group

property path: str#

The path to get to this container or group from some root path

pop(k[, d]) v, remove specified key and return the corresponding value.#

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() (k, v), remove and return some (key, value) pair#

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D#
update([E, ]**F) None.  Update D from mapping/iterable E and F.#

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values()#

Returns an iterator over the data’s values.

class StringContainer(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)[source]#

Bases: dantro.mixins.base.CollectionMixin, dantro.containers.general.PassthroughContainer

A data container to store string-like data.

DATA_EXPECTED_TYPES = (<class 'str'>,)#
DATA_ALLOW_PROXY = False#
DATA_UNEXPECTED_ACTION = 'raise'#
FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#

Attributes to not forward. Evaluated after FORWARD_ATTR_ONLY

FORWARD_ATTR_ONLY: Sequence[str] = None#

If set, the only attributes to be forwarded

FORWARD_ATTR_TO: str = 'data'#

The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if __getattr__ was not called.

_ATTRS_CLS#

alias of dantro.base.BaseDataAttrs

__contains__(key) bool#

Whether the given key is contained in the items.

__delitem__(key)#

Deletes an item

__eq__(other) bool#

Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the _data and _attrs attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.

If types do not match exactly, NotImplemented is returned, thus referring the comparison to the other side of the ==.

__format__(spec_str: str) str#

Creates a formatted string from the given specification.

Invokes further methods which are prefixed by _format_.

__getattr__(attr_name: str)#

Forward attributes that were not available in this class to some other attribute of the group or container.

Parameters

attr_name (str) – The name of the attribute that was tried to be accessed but was not available in self.

Returns

The attribute attr_name of getattr(self, self.FORWARD_ATTR_TO)

__getitem__(key)#

Returns an item.

__getstate__() dict#

Returns the object’s __dict__

__init__(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)#

Initialize a BaseDataContainer, which can store data and attributes.

Parameters
  • name (str) – The name of this data container

  • data (Any) – The data to store in this container

  • attrs (Dict[str, Any], optional) – A mapping that is stored as data attributes.

  • parent (AbstractDataGroup, optional) – If known, the parent group, which can be used to extract information during initialization. Note that linking occurs only after the container was added to the parent group using the add() method. The child object is not responsible of linking or adding itself to the group.

__iter__()#

Iterates over the items.

__len__() int#

The number of items.

__repr__() str#

Same as __str__

__setitem__(key, val)#

Sets an item.

__setstate__(d: dict)#

Sets the object’s __dict__ to the given one

__sizeof__() int#

Returns the size of the data (in bytes) stored in this container’s data and its attributes.

Note that this value is approximate. It is computed by calling the sys.getsizeof() function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.

Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.

__str__() str#

An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.

_abc_impl = <_abc._abc_data object>#
_attrs = None#

The attribute that data attributes will be stored to

_check_data(data: Any) None#

This method can be used to check the data provided to this container

It is called before the data is stored in the __init__ method and should raise an exception or create a warning if the data is not as desired.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Note

The CheckDataMixin provides a generalised implementation of this method to perform some type checks and react to unexpected types.

Parameters

data (Any) – The data to check

_check_name(new_name: str) None#

Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Parameters

new_name (str) – The new name, which is to be checked.

_format_cls_name() str#

A __format__ helper function: returns the class name

_format_info() str#

A __format__ helper function: returns info about the stored data

_format_logstr() str#

A __format__ helper function: returns the log string, a combination of class name and name

_format_name() str#

A __format__ helper function: returns the name

_format_path() str#

A __format__ helper function: returns the path to this container

_forward_attr_get_forwarding_target()#

Get the object that the attribute call is to be forwarded to

_forward_attr_post_hook(attr)#

Invoked before attribute forwarding occurs

_forward_attr_pre_hook(attr_name: Optional[str] = None)#

Invoked before attribute forwarding occurs

_item_access_convert_list_key(key)#

If given something that is not a list, just return that key

property attrs#

The container attributes.

property classname: str#

Returns the name of this DataContainer-derived class

property data: Any#

The stored data.

property logstr: str#

Returns the classname and name of this object

property name: str#

The name of this DataContainer-derived object.

property parent#

The associated parent of this container or group

property path: str#

The path to get to this container or group from some root path

dantro.containers.numeric module#

This module implements specializations of the BaseDataContainer class that focus on holding numerical, array-like data.

class NumpyDataContainer(*, name: str, data: ndarray, **dc_kwargs)[source]#

Bases: dantro.mixins.general.ForwardAttrsToDataMixin, dantro.mixins.numeric.NumbersMixin, dantro.mixins.numeric.ComparisonMixin, dantro.mixins.base.CheckDataMixin, dantro.mixins.base.ItemAccessMixin, dantro.base.BaseDataContainer

The NumpyDataContainer stores numerical array-shaped data.

Specifically: it is made for use with the numpy.ndarray class.

DATA_EXPECTED_TYPES: tuple = (<class 'numpy.ndarray'>,)#

Which types to allow. If None, all types are allowed.

DATA_ALLOW_PROXY: bool = False#

Whether to allow all proxy types, i.e. classes derived from AbstractDataProxy.

DATA_UNEXPECTED_ACTION = 'raise'#

The action to take when an unexpected type was supplied. Can be: raise, warn, ignore.

__init__(*, name: str, data: ndarray, **dc_kwargs)[source]#

Initialize a NumpyDataContainer, storing data that behaves mostly like a numpy.ndarray.

Parameters
  • name (str) – The name of this container

  • data (ndarray) – The numpy data to store

  • **dc_kwargs – Additional arguments for container initialization, passed on to parent method

_format_info() str[source]#

A __format__ helper function: returns info about the item

In this case, the dtype and shape of the stored data is returned. Note that this relies on the ForwardAttrsToDataMixin.

__len__() int[source]#

Length of the underlying data, i.e. first entry in shape

copy()[source]#

Return a copy of this NumpyDataContainer.

NOTE that this will create copies of the stored data.

save(path: str, **save_kwargs)[source]#

Saves the NumpyDataContainer to a file by invoking the numpy.save() function on the underlying data.

The file extension should be .npy, which is compatible with the numpy-based data loader. If another file extension is given, the numpy method will _append_ .npy!

Warning

This does NOT store container attributes!

Parameters
  • path (str) – The path to save the file at

  • **save_kwargs – Passed to the numpy.save() function

FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#

Attributes to not forward. Evaluated after FORWARD_ATTR_ONLY

FORWARD_ATTR_ONLY: Sequence[str] = None#

If set, the only attributes to be forwarded

FORWARD_ATTR_TO: str = 'data'#

The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if __getattr__ was not called.

_ATTRS_CLS#

alias of dantro.base.BaseDataAttrs

__abs__()#

Absolute value

Returns

A new object with the absolute value of the elements

__add__(other)#

Add two objects

Returns

A new object containing the summed data

__bool__()#

Truth value

__ceil__()#

Smallest integer

Returns

A new object containing the smallest integer

__delitem__(key)#

Deletes an item

__divmod__(other)#

Calculate the floor division and modulo of two objects

Returns

A new object containing the floor divided data and its modulo

__eq__(other)#

Equality

__floor__()#

Largest integer

Returns

A new object containing the largest element

__floordiv__(other)#

Floor divide two objects

Returns

A new object containing the floor divided data

__format__(spec_str: str) str#

Creates a formatted string from the given specification.

Invokes further methods which are prefixed by _format_.

__ge__(other)#

Greater than or equal

__getattr__(attr_name: str)#

Forward attributes that were not available in this class to some other attribute of the group or container.

Parameters

attr_name (str) – The name of the attribute that was tried to be accessed but was not available in self.

Returns

The attribute attr_name of getattr(self, self.FORWARD_ATTR_TO)

__getitem__(key)#

Returns an item.

__getstate__() dict#

Returns the object’s __dict__

__gt__(other)#

Greater than

__iadd__(other)#

Add two objects

Returns

Self with modified data

__ifloordiv__(other)#

Floor divide two objects

Returns

Self with modified data

__imod__(other)#

Calculate the modulo of two objects

Returns

Self with modified data

__imul__(other)#

Multiply two objects

Returns

Self with modified data

__invert__()#

Inverse value

Returns

A new object with the inverted values of the elements

__ipow__(other)#

Calculate the self data to the power of other data

Returns

Self with modified data

__isub__(other)#

Subtract two objects

Returns

Self with modified data

__itruediv__(other)#

Divide two objects

Returns

Self with modified data

__le__(other)#

Less than or equal

__lt__(other)#

Less than

__mod__(other)#

Calculate the modulo of two objects

Returns

A new object containing the summed data

__mul__(other)#

Multiply two objects

Returns

A object containing the multiplied data

__ne__(other)#

Inequality

__neg__()#

Make negative

Returns

A new object with negative elements

__pos__()#

Make positive

Returns

A new object with negative elements

__pow__(other)#

Calculate the self data to the power of other data

Returns

A new object containing the result

__repr__() str#

Same as __str__

__round__()#

Rounds number to nearest integer

Returns

A new object as rounded number to nearest integer

__setitem__(key, val)#

Sets an item.

__setstate__(d: dict)#

Sets the object’s __dict__ to the given one

__sizeof__() int#

Returns the size of the data (in bytes) stored in this container’s data and its attributes.

Note that this value is approximate. It is computed by calling the sys.getsizeof() function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.

Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.

__str__() str#

An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.

__sub__(other)#

Subtract two objects

Returns

A new object containing the subtracted data

__truediv__(other)#

Divide two objects

Returns

A new object containing the divided data

__trunc__()#

Truncated to the nearest integer toward 0

Returns

A new object containing the truncated element

_abc_impl = <_abc._abc_data object>#
_attrs = None#

The attribute that data attributes will be stored to

_check_data(data) None#

A general method to check the received data for its type

Parameters

data – The data to check

Raises
  • TypeError – If the type was unexpected and the action was ‘raise’

  • ValueError – Illegal value for DATA_UNEXPECTED_ACTION class variable

Returns

None

_check_name(new_name: str) None#

Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Parameters

new_name (str) – The new name, which is to be checked.

_format_cls_name() str#

A __format__ helper function: returns the class name

_format_logstr() str#

A __format__ helper function: returns the log string, a combination of class name and name

_format_name() str#

A __format__ helper function: returns the name

_format_path() str#

A __format__ helper function: returns the path to this container

_forward_attr_get_forwarding_target()#

Get the object that the attribute call is to be forwarded to

_forward_attr_post_hook(attr)#

Invoked before attribute forwarding occurs

_forward_attr_pre_hook(attr_name: Optional[str] = None)#

Invoked before attribute forwarding occurs

_item_access_convert_list_key(key)#

If given something that is not a list, just return that key

property attrs#

The container attributes.

property classname: str#

Returns the name of this DataContainer-derived class

property data: Any#

The stored data.

property logstr: str#

Returns the classname and name of this object

property name: str#

The name of this DataContainer-derived object.

property parent#

The associated parent of this container or group

property path: str#

The path to get to this container or group from some root path

dantro.containers.path module#

This module implements a container that holds the path to a file as data

class PathContainer(*args, name: str, data: Union[str, Path] = None, parent: DirectoryGroup = None, **kwargs)[source]#

Bases: dantro.mixins.general.ForwardAttrsToDataMixin, dantro.base.BaseDataContainer

A container that maps to a file system path.

It uses pathlib.Path to represent the given path and allow easy access and manipulation. To have direct access to the underlying pathlib.Path object, use the fs_path() property.

Note

The paths can also be paths to directories. However, it’s worth considering using a DirectoryGroup if it is desired to carry over the directory tree structure into the data tree.

Unlike in DirectoryGroup, the local file system path is set via the data argument during initialization and not via a data attribute.

__init__(*args, name: str, data: Union[str, Path] = None, parent: DirectoryGroup = None, **kwargs)[source]#

Sets up a container that holds a filesystem path as data.

Note

The filesystem path need not necessarily exist and it also need not be equivalent to the path within the data tree.

Parameters
  • *args – Passed to parent class init

  • name (str) – The name of this container

  • data (Union[str, Path], optional) – The filesystem path this object is meant to represent. Can be empty if this container is initialized from a parent directory group, in which case the information stored therein and the name of this container will be used to generate the path.

  • parent (DirectoryGroup, optional) – If data was not given, the path of this parent group and the name of this container will be used to generate a path.

  • **kwargs – Passed to parent class init

property fs_path: Path#

Returns the filesystem path associated with this container. This property is identical to the data property.

FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#

Attributes to not forward. Evaluated after FORWARD_ATTR_ONLY

FORWARD_ATTR_ONLY: Sequence[str] = None#

If set, the only attributes to be forwarded

FORWARD_ATTR_TO: str = 'data'#

The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if __getattr__ was not called.

_ATTRS_CLS#

alias of dantro.base.BaseDataAttrs

__eq__(other) bool#

Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the _data and _attrs attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.

If types do not match exactly, NotImplemented is returned, thus referring the comparison to the other side of the ==.

__format__(spec_str: str) str#

Creates a formatted string from the given specification.

Invokes further methods which are prefixed by _format_.

__getattr__(attr_name: str)#

Forward attributes that were not available in this class to some other attribute of the group or container.

Parameters

attr_name (str) – The name of the attribute that was tried to be accessed but was not available in self.

Returns

The attribute attr_name of getattr(self, self.FORWARD_ATTR_TO)

__getstate__() dict#

Returns the object’s __dict__

__repr__() str#

Same as __str__

__setstate__(d: dict)#

Sets the object’s __dict__ to the given one

__sizeof__() int#

Returns the size of the data (in bytes) stored in this container’s data and its attributes.

Note that this value is approximate. It is computed by calling the sys.getsizeof() function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.

Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.

__str__() str#

An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.

_abc_impl = <_abc._abc_data object>#
_attrs = None#

The attribute that data attributes will be stored to

_check_data(data: Any) None#

This method can be used to check the data provided to this container

It is called before the data is stored in the __init__ method and should raise an exception or create a warning if the data is not as desired.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Note

The CheckDataMixin provides a generalised implementation of this method to perform some type checks and react to unexpected types.

Parameters

data (Any) – The data to check

_check_name(new_name: str) None#

Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Parameters

new_name (str) – The new name, which is to be checked.

_format_cls_name() str#

A __format__ helper function: returns the class name

_format_info() str#

A __format__ helper function: returns info about the content of this data container.

_format_logstr() str#

A __format__ helper function: returns the log string, a combination of class name and name

_format_name() str#

A __format__ helper function: returns the name

_format_path() str#

A __format__ helper function: returns the path to this container

_forward_attr_get_forwarding_target()#

Get the object that the attribute call is to be forwarded to

_forward_attr_post_hook(attr)#

Invoked before attribute forwarding occurs

_forward_attr_pre_hook(attr_name: Optional[str] = None)#

Invoked before attribute forwarding occurs

property attrs#

The container attributes.

property classname: str#

Returns the name of this DataContainer-derived class

property data: Any#

The stored data.

property logstr: str#

Returns the classname and name of this object

property name: str#

The name of this DataContainer-derived object.

property parent#

The associated parent of this container or group

property path: str#

The path to get to this container or group from some root path

dantro.containers.xr module#

This module implements specializations of the BaseDataContainer class that make use of the xarray package to represent the underlying data.

class XrDataContainer(*, name: str, data: Union[ndarray, DataArray], dims: Sequence[str] = None, coords: dict = None, extract_metadata: bool = True, apply_metadata: bool = True, **dc_kwargs)[source]#

Bases: dantro.mixins.general.ForwardAttrsToDataMixin, dantro.mixins.numeric.NumbersMixin, dantro.mixins.numeric.ComparisonMixin, dantro.mixins.base.CheckDataMixin, dantro.mixins.base.ItemAccessMixin, dantro.base.BaseDataContainer

The XrDataContainer stores numerical xarray.DataArray data associated with dimensions, coordinates, and attributes.

DATA_EXPECTED_TYPES: tuple = ('xarray.DataArray', <class 'numpy.ndarray'>)#

Which types to allow. If None, all types are allowed.

DATA_ALLOW_PROXY: bool = False#

Whether to allow all proxy types, i.e. classes derived from AbstractDataProxy.

DATA_UNEXPECTED_ACTION = 'raise'#

The action to take when an unexpected type was supplied. Can be: raise, warn, ignore.

_XRC_DIMS_ATTR = 'dims'#

Define as class variable the name of the attribute that determines the dimensions of the xarray.DataArray

_XRC_DIM_NAME_PREFIX = 'dim_name__'#

Attributes prefixed with this string can be used to set names for specific dimensions. The prefix should be followed by an integer-parsable string, e.g. dim_name__0 would be the dimension name for the 0th dim.

_XRC_COORDS_ATTR_PREFIX = 'coords__'#

Attributes prefixed with this string determine the coordinate values for a specific dimension. The prefix should be followed by the name of the dimension, e.g. coord__time. The values are interpreted according to the default coordinate mode or, if given, the coord_mode__* attribute.

_XRC_COORDS_MODE_DEFAULT = 'values'#

The default mode by which coordinates are interpreted

_XRC_COORDS_MODE_ATTR_PREFIX = 'coords_mode__'#

Prefix for the coordinate mode if a custom mode is to be used

_XRC_INHERIT_CONTAINER_ATTRIBUTES = True#

Whether to inherit the other container attributes

_XRC_STRICT_ATTR_CHECKING = True#

Whether to use strict attribute checking; throws errors if there are container attributes available that match the prefix but don’t match a valid dimension name. Can be disabled for speed improvements.

__init__(*, name: str, data: Union[ndarray, DataArray], dims: Sequence[str] = None, coords: dict = None, extract_metadata: bool = True, apply_metadata: bool = True, **dc_kwargs)[source]#

Initialize a XrDataContainer and extract dimension and coordinate labels.

Parameters
  • name (str) – which name to give to the XrDataContainer

  • data (Union[ndarray, DataArray]) – The data to store; anything that an xarray.DataArray can take.

  • dims (Sequence[str], optional) – The dimension names.

  • coords (dict, optional) – The coordinates. The keys of this dict have to correspond to the dimension names.

  • extract_metadata (bool, optional) – If True, missing dims or coords arguments are tried to be populated from the container attributes.

  • apply_metadata (bool, optional) – Whether to apply the extracted or passed dims and coords to the underlying data. This might not be desired in cases where the given data already is a labelled xarray.DataArray or where the data is a proxy and the labelling should be postponed.

  • **dc_kwargs – passed to parent

_format_info() str[source]#

A __format__ helper function: returns info about the item.

In this case, the dtype and sizes of the stored data is returned. Depending on whether metadata is available, the shape information is shown or the dimension names and the length of the dimensions are used.

_format_shape() str[source]#

A __format__ helper for parsing shape information

__len__() int[source]#

Length of the underlying data, i.e. first entry in shape

FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#

Attributes to not forward. Evaluated after FORWARD_ATTR_ONLY

FORWARD_ATTR_ONLY: Sequence[str] = None#

If set, the only attributes to be forwarded

FORWARD_ATTR_TO: str = 'data'#

The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if __getattr__ was not called.

_ATTRS_CLS#

alias of dantro.base.BaseDataAttrs

__abs__()#

Absolute value

Returns

A new object with the absolute value of the elements

__add__(other)#

Add two objects

Returns

A new object containing the summed data

__bool__()#

Truth value

__ceil__()#

Smallest integer

Returns

A new object containing the smallest integer

__delitem__(key)#

Deletes an item

__divmod__(other)#

Calculate the floor division and modulo of two objects

Returns

A new object containing the floor divided data and its modulo

__eq__(other)#

Equality

__floor__()#

Largest integer

Returns

A new object containing the largest element

__floordiv__(other)#

Floor divide two objects

Returns

A new object containing the floor divided data

__format__(spec_str: str) str#

Creates a formatted string from the given specification.

Invokes further methods which are prefixed by _format_.

__ge__(other)#

Greater than or equal

__getattr__(attr_name: str)#

Forward attributes that were not available in this class to some other attribute of the group or container.

Parameters

attr_name (str) – The name of the attribute that was tried to be accessed but was not available in self.

Returns

The attribute attr_name of getattr(self, self.FORWARD_ATTR_TO)

__getitem__(key)#

Returns an item.

__getstate__() dict#

Returns the object’s __dict__

__gt__(other)#

Greater than

__iadd__(other)#

Add two objects

Returns

Self with modified data

__ifloordiv__(other)#

Floor divide two objects

Returns

Self with modified data

__imod__(other)#

Calculate the modulo of two objects

Returns

Self with modified data

__imul__(other)#

Multiply two objects

Returns

Self with modified data

__invert__()#

Inverse value

Returns

A new object with the inverted values of the elements

__ipow__(other)#

Calculate the self data to the power of other data

Returns

Self with modified data

__isub__(other)#

Subtract two objects

Returns

Self with modified data

__itruediv__(other)#

Divide two objects

Returns

Self with modified data

__le__(other)#

Less than or equal

__lt__(other)#

Less than

__mod__(other)#

Calculate the modulo of two objects

Returns

A new object containing the summed data

__mul__(other)#

Multiply two objects

Returns

A object containing the multiplied data

__ne__(other)#

Inequality

__neg__()#

Make negative

Returns

A new object with negative elements

__pos__()#

Make positive

Returns

A new object with negative elements

__pow__(other)#

Calculate the self data to the power of other data

Returns

A new object containing the result

__repr__() str#

Same as __str__

__round__()#

Rounds number to nearest integer

Returns

A new object as rounded number to nearest integer

__setitem__(key, val)#

Sets an item.

__setstate__(d: dict)#

Sets the object’s __dict__ to the given one

__sizeof__() int#

Returns the size of the data (in bytes) stored in this container’s data and its attributes.

Note that this value is approximate. It is computed by calling the sys.getsizeof() function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.

Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.

__str__() str#

An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.

__sub__(other)#

Subtract two objects

Returns

A new object containing the subtracted data

__truediv__(other)#

Divide two objects

Returns

A new object containing the divided data

__trunc__()#

Truncated to the nearest integer toward 0

Returns

A new object containing the truncated element

_abc_impl = <_abc._abc_data object>#
_attrs = None#

The attribute that data attributes will be stored to

_check_data(data) None#

A general method to check the received data for its type

Parameters

data – The data to check

Raises
  • TypeError – If the type was unexpected and the action was ‘raise’

  • ValueError – Illegal value for DATA_UNEXPECTED_ACTION class variable

Returns

None

_check_name(new_name: str) None#

Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.

This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().

Parameters

new_name (str) – The new name, which is to be checked.

_format_cls_name() str#

A __format__ helper function: returns the class name

_format_logstr() str#

A __format__ helper function: returns the log string, a combination of class name and name

_format_name() str#

A __format__ helper function: returns the name

_format_path() str#

A __format__ helper function: returns the path to this container

_forward_attr_get_forwarding_target()#

Get the object that the attribute call is to be forwarded to

_forward_attr_post_hook(attr)#

Invoked before attribute forwarding occurs

_forward_attr_pre_hook(attr_name: Optional[str] = None)#

Invoked before attribute forwarding occurs

_item_access_convert_list_key(key)#

If given something that is not a list, just return that key

property attrs#

The container attributes.

property classname: str#

Returns the name of this DataContainer-derived class

copy(*, deep: bool = True)[source]#

Return a new object with a copy of the data. The copy is deep if not specified otherwise.

Parameters

deep (bool, optional) – Whether the copy is deep

Returns

A (deep) copy of this object.

Return type

XrDataContainer

property data: Any#

The stored data.

property logstr: str#

Returns the classname and name of this object

property name: str#

The name of this DataContainer-derived object.

property parent#

The associated parent of this container or group

property path: str#

The path to get to this container or group from some root path

save(path: str, **save_kwargs)[source]#

Saves the XrDataContainer to a file by invoking the .to_netcdf method of the underlying data.

The recommended file extension is .xrdc or .nc_da, which are compatible with the xarray-based data loader.

Warning

This does NOT store container attributes!

Parameters
  • path (str) – The path to save the file at

  • **save_kwargs – Passed to .no_netcdf method call

_extract_metadata()[source]#

Extracts metadata from the container attributes and stores them in the _dim_names and _dim_to_coords_map cache attributes.

_inherit_attrs()[source]#

Carry over container attributes to the data array attributes.

This does not include container attributes that are used for extracting metadata; it makes no sense to have them in the attributes of the already labelled xarray.DataArray.

_apply_metadata()[source]#

Applies the cached metadata to the underlying xarray.DataArray

_postprocess_proxy_resolution()[source]#

Only invoked from ProxySupportMixin, which have to be added to the class specifically. This function takes care to apply the potentially existing metadata after the proxy was resolved.

_parse_sizes_from_metadata() Sequence[Tuple[str, int]][source]#

Invoked from _format_shape when no metadata was applied but the dimension names are available. Should return data in the same form as xr.DataArray.sizes.items() does.