dantro.containers package
Contents
dantro.containers package#
Implements BaseDataContainer
specializations.
isort:skip_file
Submodules#
dantro.containers._registry module#
Implements a registry for dantro container types based on
ObjectRegistry
.
- class ContainerRegistry[source]#
Bases:
dantro._registry.ObjectRegistry
- _EXPECTED_TYPE: Optional[Union[tuple, type]] = (<class 'type'>,)#
If set, will check for expected types
- _register_via_decorator(obj: type, name: Optional[str] = None, **kws)[source]#
Performs the registration operations when the decorator is used to register an object.
- __contains__(obj_or_key: Union[Any, str]) bool #
Whether the given argument is part of the keys or values of this registry.
- _decorator(arg: Optional[Union[Any, str]] = None, /, **kws)#
Method that can be used as a decorator for registering objects with this registry.
- Parameters
arg (Union[Any, str], optional) – The name that should be used or the object that is to be added. If not a string, this refers to the
@is_container
call syntax**kws – Passed to
register()
- _determine_name(obj: Any, *, name: Optional[str]) str #
Determines the object name, using a potentially given name
- items()#
- keys()#
- register(obj: Any, name: Optional[str] = None, *, skip_existing: Optional[bool] = None, overwrite_existing: Optional[bool] = None) str #
Adds an entry to the registry.
- Parameters
obj (Any) – The object to add to the registry.
name (Optional[str], optional) – The name to use. If not given, will deduce a name from the given object.
skip_existing (bool, optional) – Whether to skip registration if an object of that name already exists. If None, the classes default behavior (see
_SKIP
) is used.overwrite_existing (bool, optional) – Whether to overwrite an entry if an object with that name already exists. If None, the classes default behavior (see
_OVERWRITE
) is used.
- values()#
- CONTAINERS = <dantro.containers._registry.ContainerRegistry object>#
The dantro data container registry object.
- register_container(Cls: type, name: str, *, skip_existing: bool = False, overwrite_existing: bool = True) None [source]#
Adds an entry to the shared container registry.
- Parameters
Cls (type) – The class that is to be registered as a container.
name (str) – The name to use for registration.
skip_existing (bool, optional) – Whether to skip registration if the container name is already registered. This suppresses the ValueError raised on existing container name.
overwrite_existing (bool, optional) – Whether to overwrite a potentially already existing container of the same name. If set, this takes precedence over
skip_existing
.
- is_container(arg: Optional[Union[type, str]] = None, /, **kws)[source]#
Decorator for registering containers with the container type registry.
As an alternative to
register_container()
, this decorator can be used to register a container right where its defined:from dantro.containers import BaseDataContainer, is_container # Container name deduced from class name @is_container class MyDataContainer(BaseDataContainer): # ... do stuff here ... pass # Custom container name @is_container("my_container") class MyDataContainer(BaseDataContainer): # ... do stuff here ... pass # Overwriting a registered container of the same name @is_container("my_container", overwrite_existing=True) class MyDataContainer(BaseDataContainer): # ... do stuff here ... pass
dantro.containers.general module#
This module implements general specialisations of the
BaseDataContainer
- class ObjectContainer(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)[source]#
Bases:
dantro.mixins.base.ItemAccessMixin
,dantro.base.BaseDataContainer
Generically stores any Python object
This allows item access, but not more.
- _ATTRS_CLS#
alias of
dantro.base.BaseDataAttrs
- __delitem__(key)#
Deletes an item
- __eq__(other) bool #
Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the
_data
and_attrs
attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.If types do not match exactly,
NotImplemented
is returned, thus referring the comparison to the other side of the==
.
- __format__(spec_str: str) str #
Creates a formatted string from the given specification.
Invokes further methods which are prefixed by
_format_
.
- __getitem__(key)#
Returns an item.
- __init__(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)#
Initialize a BaseDataContainer, which can store data and attributes.
- Parameters
name (str) – The name of this data container
data (Any) – The data to store in this container
attrs (Dict[str, Any], optional) – A mapping that is stored as data attributes.
parent (AbstractDataGroup, optional) – If known, the parent group, which can be used to extract information during initialization. Note that linking occurs only after the container was added to the parent group using the
add()
method. The child object is not responsible of linking or adding itself to the group.
- __setitem__(key, val)#
Sets an item.
- __sizeof__() int #
Returns the size of the data (in bytes) stored in this container’s data and its attributes.
Note that this value is approximate. It is computed by calling the
sys.getsizeof()
function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.
- __str__() str #
An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.
- _abc_impl = <_abc_data object>#
- _attrs = None#
The attribute that data attributes will be stored to
- _check_data(data: Any) None #
This method can be used to check the data provided to this container
It is called before the data is stored in the
__init__
method and should raise an exception or create a warning if the data is not as desired.This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using
super()
.Note
The
CheckDataMixin
provides a generalised implementation of this method to perform some type checks and react to unexpected types.- Parameters
data (Any) – The data to check
- _check_name(new_name: str) None #
Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.
This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().
- Parameters
new_name (str) – The new name, which is to be checked.
- _format_logstr() str #
A __format__ helper function: returns the log string, a combination of class name and name
- _item_access_convert_list_key(key)#
If given something that is not a list, just return that key
- property attrs#
The container attributes.
- property data: Any#
The stored data.
- property parent#
The associated parent of this container or group
- class PassthroughContainer(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)[source]#
Bases:
dantro.mixins.general.ForwardAttrsToDataMixin
,dantro.containers.general.ObjectContainer
An object container that forwards all attribute calls to .data
- FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#
Attributes to not forward. Evaluated after
FORWARD_ATTR_ONLY
- FORWARD_ATTR_TO: str = 'data'#
The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if
__getattr__
was not called.
- _ATTRS_CLS#
alias of
dantro.base.BaseDataAttrs
- __delitem__(key)#
Deletes an item
- __eq__(other) bool #
Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the
_data
and_attrs
attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.If types do not match exactly,
NotImplemented
is returned, thus referring the comparison to the other side of the==
.
- __format__(spec_str: str) str #
Creates a formatted string from the given specification.
Invokes further methods which are prefixed by
_format_
.
- __getattr__(attr_name: str)#
Forward attributes that were not available in this class to some other attribute of the group or container.
- Parameters
attr_name (str) – The name of the attribute that was tried to be accessed but was not available in
self
.- Returns
The attribute
attr_name
ofgetattr(self, self.FORWARD_ATTR_TO)
- __getitem__(key)#
Returns an item.
- __init__(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)#
Initialize a BaseDataContainer, which can store data and attributes.
- Parameters
name (str) – The name of this data container
data (Any) – The data to store in this container
attrs (Dict[str, Any], optional) – A mapping that is stored as data attributes.
parent (AbstractDataGroup, optional) – If known, the parent group, which can be used to extract information during initialization. Note that linking occurs only after the container was added to the parent group using the
add()
method. The child object is not responsible of linking or adding itself to the group.
- __setitem__(key, val)#
Sets an item.
- __sizeof__() int #
Returns the size of the data (in bytes) stored in this container’s data and its attributes.
Note that this value is approximate. It is computed by calling the
sys.getsizeof()
function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.
- __str__() str #
An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.
- _abc_impl = <_abc_data object>#
- _attrs = None#
The attribute that data attributes will be stored to
- _check_data(data: Any) None #
This method can be used to check the data provided to this container
It is called before the data is stored in the
__init__
method and should raise an exception or create a warning if the data is not as desired.This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using
super()
.Note
The
CheckDataMixin
provides a generalised implementation of this method to perform some type checks and react to unexpected types.- Parameters
data (Any) – The data to check
- _check_name(new_name: str) None #
Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.
This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().
- Parameters
new_name (str) – The new name, which is to be checked.
- _format_logstr() str #
A __format__ helper function: returns the log string, a combination of class name and name
- _forward_attr_get_forwarding_target()#
Get the object that the attribute call is to be forwarded to
- _forward_attr_post_hook(attr)#
Invoked before attribute forwarding occurs
- _item_access_convert_list_key(key)#
If given something that is not a list, just return that key
- property attrs#
The container attributes.
- property data: Any#
The stored data.
- property parent#
The associated parent of this container or group
- class MutableSequenceContainer(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)[source]#
Bases:
dantro.mixins.base.CheckDataMixin
,dantro.mixins.base.ItemAccessMixin
,dantro.mixins.base.CollectionMixin
,dantro.base.BaseDataContainer
,collections.abc.MutableSequence
The MutableSequenceContainer stores data that is sequence-like
- DATA_EXPECTED_TYPES: tuple = (<class 'collections.abc.MutableSequence'>, <class 'list'>)#
Which types to allow. If None, all types are allowed.
- DATA_ALLOW_PROXY: bool = False#
Whether to allow all proxy types, i.e. classes derived from
AbstractDataProxy
.
- DATA_UNEXPECTED_ACTION = 'warn'#
The action to take when an unexpected type was supplied. Can be:
raise
,warn
,ignore
.
- insert(idx: int, val) None [source]#
Insert an item at a given position. The first argument is the index of the element before which to insert, so a.insert(0, x) inserts at the front of the list, and a.insert(len(a), x) is equivalent to a.append(x).
- Parameters
idx (int) – The index before which to insert
val – The value to insert
- _ATTRS_CLS#
alias of
dantro.base.BaseDataAttrs
- __delitem__(key)#
Deletes an item
- __eq__(other) bool #
Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the
_data
and_attrs
attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.If types do not match exactly,
NotImplemented
is returned, thus referring the comparison to the other side of the==
.
- __format__(spec_str: str) str #
Creates a formatted string from the given specification.
Invokes further methods which are prefixed by
_format_
.
- __getitem__(key)#
Returns an item.
- __init__(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)#
Initialize a BaseDataContainer, which can store data and attributes.
- Parameters
name (str) – The name of this data container
data (Any) – The data to store in this container
attrs (Dict[str, Any], optional) – A mapping that is stored as data attributes.
parent (AbstractDataGroup, optional) – If known, the parent group, which can be used to extract information during initialization. Note that linking occurs only after the container was added to the parent group using the
add()
method. The child object is not responsible of linking or adding itself to the group.
- __iter__()#
Iterates over the items.
- __setitem__(key, val)#
Sets an item.
- __sizeof__() int #
Returns the size of the data (in bytes) stored in this container’s data and its attributes.
Note that this value is approximate. It is computed by calling the
sys.getsizeof()
function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.
- __str__() str #
An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.
- _abc_impl = <_abc_data object>#
- _attrs = None#
The attribute that data attributes will be stored to
- _check_data(data) None #
A general method to check the received data for its type
- Parameters
data – The data to check
- Raises
TypeError – If the type was unexpected and the action was ‘raise’
ValueError – Illegal value for
DATA_UNEXPECTED_ACTION
class variable
- Returns
None
- _check_name(new_name: str) None #
Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.
This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().
- Parameters
new_name (str) – The new name, which is to be checked.
- _format_info() str #
A __format__ helper function: returns info about the content of this data container.
- _format_logstr() str #
A __format__ helper function: returns the log string, a combination of class name and name
- _item_access_convert_list_key(key)#
If given something that is not a list, just return that key
- append(value)#
S.append(value) – append value to the end of the sequence
- property attrs#
The container attributes.
- clear() None -- remove all items from S #
- count(value) integer -- return number of occurrences of value #
- property data: Any#
The stored data.
- extend(values)#
S.extend(iterable) – extend sequence by appending elements from the iterable
- index(value[, start[, stop]]) integer -- return first index of value. #
Raises ValueError if the value is not present.
Supporting start and stop arguments is optional, but recommended.
- property parent#
The associated parent of this container or group
- pop([index]) item -- remove and return item at index (default last). #
Raise IndexError if list is empty or index is out of range.
- remove(value)#
S.remove(value) – remove first occurrence of value. Raise ValueError if the value is not present.
- reverse()#
S.reverse() – reverse IN PLACE
- class MutableMappingContainer(*, name: str, data=None, **dc_kwargs)[source]#
Bases:
dantro.mixins.base.CheckDataMixin
,dantro.mixins.base.MappingAccessMixin
,dantro.base.BaseDataContainer
,collections.abc.MutableMapping
The MutableMappingContainer stores mutable mapping data, e.g. dicts
- DATA_EXPECTED_TYPES: tuple = (<class 'collections.abc.MutableMapping'>, <class 'dict'>)#
Which types to allow. If None, all types are allowed.
- DATA_ALLOW_PROXY: bool = False#
Whether to allow all proxy types, i.e. classes derived from
AbstractDataProxy
.
- DATA_UNEXPECTED_ACTION = 'warn'#
The action to take when an unexpected type was supplied. Can be:
raise
,warn
,ignore
.
- __init__(*, name: str, data=None, **dc_kwargs)[source]#
Initialize a MutableMappingContainer, storing mapping data.
- Parameters
name (str) – The name of this container
data – The mapping-like data to store. If not given, an empty dict is created
**dc_kwargs – Additional arguments for container initialization
- _ATTRS_CLS#
alias of
dantro.base.BaseDataAttrs
- __delitem__(key)#
Deletes an item
- __eq__(other) bool #
Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the
_data
and_attrs
attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.If types do not match exactly,
NotImplemented
is returned, thus referring the comparison to the other side of the==
.
- __format__(spec_str: str) str #
Creates a formatted string from the given specification.
Invokes further methods which are prefixed by
_format_
.
- __getitem__(key)#
Returns an item.
- __iter__()#
Iterates over the items.
- __setitem__(key, val)#
Sets an item.
- __sizeof__() int #
Returns the size of the data (in bytes) stored in this container’s data and its attributes.
Note that this value is approximate. It is computed by calling the
sys.getsizeof()
function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.
- __str__() str #
An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.
- _abc_impl = <_abc_data object>#
- _attrs = None#
The attribute that data attributes will be stored to
- _check_data(data) None #
A general method to check the received data for its type
- Parameters
data – The data to check
- Raises
TypeError – If the type was unexpected and the action was ‘raise’
ValueError – Illegal value for
DATA_UNEXPECTED_ACTION
class variable
- Returns
None
- _check_name(new_name: str) None #
Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.
This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().
- Parameters
new_name (str) – The new name, which is to be checked.
- _format_info() str #
A __format__ helper function: returns info about the content of this data container.
- _format_logstr() str #
A __format__ helper function: returns the log string, a combination of class name and name
- _item_access_convert_list_key(key)#
If given something that is not a list, just return that key
- property attrs#
The container attributes.
- clear() None. Remove all items from D. #
- property data: Any#
The stored data.
- get(key, default=None)#
Return the value at
key
, ordefault
ifkey
is not available.
- items()#
Returns an iterator over data’s
(key, value)
tuples
- keys()#
Returns an iterator over the data’s keys.
- property parent#
The associated parent of this container or group
- pop(k[, d]) v, remove specified key and return the corresponding value. #
If key is not found, d is returned if given, otherwise KeyError is raised.
- popitem() (k, v), remove and return some (key, value) pair #
as a 2-tuple; but raise KeyError if D is empty.
- setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D #
- update([E, ]**F) None. Update D from mapping/iterable E and F. #
If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
- values()#
Returns an iterator over the data’s values.
- class StringContainer(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)[source]#
Bases:
dantro.mixins.base.CollectionMixin
,dantro.containers.general.PassthroughContainer
A data container to store string-like data.
- DATA_EXPECTED_TYPES = (<class 'str'>,)#
- DATA_ALLOW_PROXY = False#
- DATA_UNEXPECTED_ACTION = 'raise'#
- FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#
Attributes to not forward. Evaluated after
FORWARD_ATTR_ONLY
- FORWARD_ATTR_TO: str = 'data'#
The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if
__getattr__
was not called.
- _ATTRS_CLS#
alias of
dantro.base.BaseDataAttrs
- __delitem__(key)#
Deletes an item
- __eq__(other) bool #
Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the
_data
and_attrs
attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.If types do not match exactly,
NotImplemented
is returned, thus referring the comparison to the other side of the==
.
- __format__(spec_str: str) str #
Creates a formatted string from the given specification.
Invokes further methods which are prefixed by
_format_
.
- __getattr__(attr_name: str)#
Forward attributes that were not available in this class to some other attribute of the group or container.
- Parameters
attr_name (str) – The name of the attribute that was tried to be accessed but was not available in
self
.- Returns
The attribute
attr_name
ofgetattr(self, self.FORWARD_ATTR_TO)
- __getitem__(key)#
Returns an item.
- __init__(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)#
Initialize a BaseDataContainer, which can store data and attributes.
- Parameters
name (str) – The name of this data container
data (Any) – The data to store in this container
attrs (Dict[str, Any], optional) – A mapping that is stored as data attributes.
parent (AbstractDataGroup, optional) – If known, the parent group, which can be used to extract information during initialization. Note that linking occurs only after the container was added to the parent group using the
add()
method. The child object is not responsible of linking or adding itself to the group.
- __iter__()#
Iterates over the items.
- __setitem__(key, val)#
Sets an item.
- __sizeof__() int #
Returns the size of the data (in bytes) stored in this container’s data and its attributes.
Note that this value is approximate. It is computed by calling the
sys.getsizeof()
function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.
- __str__() str #
An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.
- _abc_impl = <_abc_data object>#
- _attrs = None#
The attribute that data attributes will be stored to
- _check_data(data: Any) None #
This method can be used to check the data provided to this container
It is called before the data is stored in the
__init__
method and should raise an exception or create a warning if the data is not as desired.This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using
super()
.Note
The
CheckDataMixin
provides a generalised implementation of this method to perform some type checks and react to unexpected types.- Parameters
data (Any) – The data to check
- _check_name(new_name: str) None #
Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.
This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().
- Parameters
new_name (str) – The new name, which is to be checked.
- _format_logstr() str #
A __format__ helper function: returns the log string, a combination of class name and name
- _forward_attr_get_forwarding_target()#
Get the object that the attribute call is to be forwarded to
- _forward_attr_post_hook(attr)#
Invoked before attribute forwarding occurs
- _item_access_convert_list_key(key)#
If given something that is not a list, just return that key
- property attrs#
The container attributes.
- property data: Any#
The stored data.
- property parent#
The associated parent of this container or group
dantro.containers.link module#
Implements the dantro.containers.link.LinkContainer
which holds
a Link
object and can be used to link to
another position in the data tree.
- class LinkContainer(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)[source]#
Bases:
dantro.mixins.base.CheckDataMixin
,dantro.containers.general.PassthroughContainer
A LinkContainer is a container containing a
Link
object.It forwards all attribute calls to the
Link
object, which in turn forwards all attribute calls to the linked object, thereby emulating the behaviour of the linked object.- DATA_EXPECTED_TYPES: tuple = (<class 'dantro.utils.link.Link'>,)#
Which types to allow. If None, all types are allowed.
- _format_info() str [source]#
A __format__ helper function: returns info about the item.
In this case, the anchor and relative path of the associated link is returned.
- DATA_ALLOW_PROXY: bool = False#
Whether to allow all proxy types, i.e. classes derived from
AbstractDataProxy
.
- DATA_UNEXPECTED_ACTION = 'warn'#
The action to take when an unexpected type was supplied. Can be:
raise
,warn
,ignore
.
- FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#
Attributes to not forward. Evaluated after
FORWARD_ATTR_ONLY
- FORWARD_ATTR_TO: str = 'data'#
The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if
__getattr__
was not called.
- _ATTRS_CLS#
alias of
dantro.base.BaseDataAttrs
- __delitem__(key)#
Deletes an item
- __eq__(other) bool #
Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the
_data
and_attrs
attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.If types do not match exactly,
NotImplemented
is returned, thus referring the comparison to the other side of the==
.
- __format__(spec_str: str) str #
Creates a formatted string from the given specification.
Invokes further methods which are prefixed by
_format_
.
- __getattr__(attr_name: str)#
Forward attributes that were not available in this class to some other attribute of the group or container.
- Parameters
attr_name (str) – The name of the attribute that was tried to be accessed but was not available in
self
.- Returns
The attribute
attr_name
ofgetattr(self, self.FORWARD_ATTR_TO)
- __getitem__(key)#
Returns an item.
- __init__(*, name: str, data: Any, attrs: Optional[Dict[str, Any]] = None, parent: Optional[AbstractDataGroup] = None)#
Initialize a BaseDataContainer, which can store data and attributes.
- Parameters
name (str) – The name of this data container
data (Any) – The data to store in this container
attrs (Dict[str, Any], optional) – A mapping that is stored as data attributes.
parent (AbstractDataGroup, optional) – If known, the parent group, which can be used to extract information during initialization. Note that linking occurs only after the container was added to the parent group using the
add()
method. The child object is not responsible of linking or adding itself to the group.
- __setitem__(key, val)#
Sets an item.
- __sizeof__() int #
Returns the size of the data (in bytes) stored in this container’s data and its attributes.
Note that this value is approximate. It is computed by calling the
sys.getsizeof()
function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.
- __str__() str #
An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.
- _abc_impl = <_abc_data object>#
- _attrs = None#
The attribute that data attributes will be stored to
- _check_data(data) None #
A general method to check the received data for its type
- Parameters
data – The data to check
- Raises
TypeError – If the type was unexpected and the action was ‘raise’
ValueError – Illegal value for
DATA_UNEXPECTED_ACTION
class variable
- Returns
None
- _check_name(new_name: str) None #
Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.
This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().
- Parameters
new_name (str) – The new name, which is to be checked.
- _format_logstr() str #
A __format__ helper function: returns the log string, a combination of class name and name
- _forward_attr_get_forwarding_target()#
Get the object that the attribute call is to be forwarded to
- _forward_attr_post_hook(attr)#
Invoked before attribute forwarding occurs
- _item_access_convert_list_key(key)#
If given something that is not a list, just return that key
- property attrs#
The container attributes.
- property data: Any#
The stored data.
- property parent#
The associated parent of this container or group
dantro.containers.numeric module#
This module implements specializations of the
BaseDataContainer
class that focus on holding
numerical, array-like data.
- class NumpyDataContainer(*, name: str, data: ndarray, **dc_kwargs)[source]#
Bases:
dantro.mixins.general.ForwardAttrsToDataMixin
,dantro.mixins.numeric.NumbersMixin
,dantro.mixins.numeric.ComparisonMixin
,dantro.mixins.base.CheckDataMixin
,dantro.mixins.base.ItemAccessMixin
,dantro.base.BaseDataContainer
The NumpyDataContainer stores numerical array-shaped data.
Specifically: it is made for use with the
numpy.ndarray
class.- DATA_EXPECTED_TYPES: tuple = (<class 'numpy.ndarray'>,)#
Which types to allow. If None, all types are allowed.
- DATA_ALLOW_PROXY: bool = False#
Whether to allow all proxy types, i.e. classes derived from
AbstractDataProxy
.
- DATA_UNEXPECTED_ACTION = 'raise'#
The action to take when an unexpected type was supplied. Can be:
raise
,warn
,ignore
.
- __init__(*, name: str, data: ndarray, **dc_kwargs)[source]#
Initialize a NumpyDataContainer, storing data that behaves mostly like a
numpy.ndarray
.
- _format_info() str [source]#
A __format__ helper function: returns info about the item
In this case, the dtype and shape of the stored data is returned. Note that this relies on the ForwardAttrsToDataMixin.
- copy()[source]#
Return a copy of this NumpyDataContainer.
NOTE that this will create copies of the stored data.
- save(path: str, **save_kwargs)[source]#
Saves the NumpyDataContainer to a file by invoking the
numpy.save()
function on the underlying data.The file extension should be
.npy
, which is compatible with the numpy-based data loader. If another file extension is given, the numpy method will _append_.npy
!Warning
This does NOT store container attributes!
- Parameters
path (str) – The path to save the file at
**save_kwargs – Passed to the
numpy.save()
function
- FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#
Attributes to not forward. Evaluated after
FORWARD_ATTR_ONLY
- FORWARD_ATTR_TO: str = 'data'#
The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if
__getattr__
was not called.
- _ATTRS_CLS#
alias of
dantro.base.BaseDataAttrs
- __abs__()#
Absolute value
- Returns
A new object with the absolute value of the elements
- __add__(other)#
Add two objects
- Returns
A new object containing the summed data
- __bool__()#
Truth value
- __ceil__()#
Smallest integer
- Returns
A new object containing the smallest integer
- __delitem__(key)#
Deletes an item
- __divmod__(other)#
Calculate the floor division and modulo of two objects
- Returns
A new object containing the floor divided data and its modulo
- __eq__(other)#
Equality
- __floor__()#
Largest integer
- Returns
A new object containing the largest element
- __floordiv__(other)#
Floor divide two objects
- Returns
A new object containing the floor divided data
- __format__(spec_str: str) str #
Creates a formatted string from the given specification.
Invokes further methods which are prefixed by
_format_
.
- __ge__(other)#
Greater than or equal
- __getattr__(attr_name: str)#
Forward attributes that were not available in this class to some other attribute of the group or container.
- Parameters
attr_name (str) – The name of the attribute that was tried to be accessed but was not available in
self
.- Returns
The attribute
attr_name
ofgetattr(self, self.FORWARD_ATTR_TO)
- __getitem__(key)#
Returns an item.
- __gt__(other)#
Greater than
- __iadd__(other)#
Add two objects
- Returns
Self with modified data
- __ifloordiv__(other)#
Floor divide two objects
- Returns
Self with modified data
- __imod__(other)#
Calculate the modulo of two objects
- Returns
Self with modified data
- __imul__(other)#
Multiply two objects
- Returns
Self with modified data
- __invert__()#
Inverse value
- Returns
A new object with the inverted values of the elements
- __ipow__(other)#
Calculate the self data to the power of other data
- Returns
Self with modified data
- __isub__(other)#
Subtract two objects
- Returns
Self with modified data
- __itruediv__(other)#
Divide two objects
- Returns
Self with modified data
- __le__(other)#
Less than or equal
- __lt__(other)#
Less than
- __mod__(other)#
Calculate the modulo of two objects
- Returns
A new object containing the summed data
- __mul__(other)#
Multiply two objects
- Returns
A object containing the multiplied data
- __ne__(other)#
Inequality
- __neg__()#
Make negative
- Returns
A new object with negative elements
- __pos__()#
Make positive
- Returns
A new object with negative elements
- __pow__(other)#
Calculate the self data to the power of other data
- Returns
A new object containing the result
- __round__()#
Rounds number to nearest integer
- Returns
A new object as rounded number to nearest integer
- __setitem__(key, val)#
Sets an item.
- __sizeof__() int #
Returns the size of the data (in bytes) stored in this container’s data and its attributes.
Note that this value is approximate. It is computed by calling the
sys.getsizeof()
function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.
- __str__() str #
An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.
- __sub__(other)#
Subtract two objects
- Returns
A new object containing the subtracted data
- __truediv__(other)#
Divide two objects
- Returns
A new object containing the divided data
- __trunc__()#
Truncated to the nearest integer toward 0
- Returns
A new object containing the truncated element
- _abc_impl = <_abc_data object>#
- _attrs = None#
The attribute that data attributes will be stored to
- _check_data(data) None #
A general method to check the received data for its type
- Parameters
data – The data to check
- Raises
TypeError – If the type was unexpected and the action was ‘raise’
ValueError – Illegal value for
DATA_UNEXPECTED_ACTION
class variable
- Returns
None
- _check_name(new_name: str) None #
Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.
This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().
- Parameters
new_name (str) – The new name, which is to be checked.
- _format_logstr() str #
A __format__ helper function: returns the log string, a combination of class name and name
- _forward_attr_get_forwarding_target()#
Get the object that the attribute call is to be forwarded to
- _forward_attr_post_hook(attr)#
Invoked before attribute forwarding occurs
- _item_access_convert_list_key(key)#
If given something that is not a list, just return that key
- property attrs#
The container attributes.
- property data: Any#
The stored data.
- property parent#
The associated parent of this container or group
dantro.containers.path module#
This module implements a container that holds the path to a file as data
- class PathContainer(*args, name: str, data: Union[str, Path] = None, parent: DirectoryGroup = None, **kwargs)[source]#
Bases:
dantro.mixins.general.ForwardAttrsToDataMixin
,dantro.base.BaseDataContainer
A container that maps to a file system path.
It uses
pathlib.Path
to represent the given path and allow easy access and manipulation. To have direct access to the underlyingpathlib.Path
object, use thefs_path()
property.Note
The paths can also be paths to directories. However, it’s worth considering using a
DirectoryGroup
if it is desired to carry over the directory tree structure into the data tree.Unlike in
DirectoryGroup
, the local file system path is set via thedata
argument during initialization and not via a data attribute.- __init__(*args, name: str, data: Union[str, Path] = None, parent: DirectoryGroup = None, **kwargs)[source]#
Sets up a container that holds a filesystem path as data.
Note
The filesystem path need not necessarily exist and it also need not be equivalent to the path within the data tree.
- Parameters
*args – Passed to parent class init
name (str) – The name of this container
data (Union[str, Path], optional) – The filesystem path this object is meant to represent. Can be empty if this container is initialized from a
parent
directory group, in which case the information stored therein and thename
of this container will be used to generate the path.parent (DirectoryGroup, optional) – If
data
was not given, the path of this parent group and thename
of this container will be used to generate a path.**kwargs – Passed to parent class init
- property fs_path: Path#
Returns the filesystem path associated with this container. This property is identical to the
data
property.
- FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#
Attributes to not forward. Evaluated after
FORWARD_ATTR_ONLY
- FORWARD_ATTR_TO: str = 'data'#
The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if
__getattr__
was not called.
- _ATTRS_CLS#
alias of
dantro.base.BaseDataAttrs
- __eq__(other) bool #
Evaluates equality by making the following comparisons: identity, strict type equality, and finally: equality of the
_data
and_attrs
attributes, i.e. the private attribute. This ensures that comparison does not trigger any downstream effects like resolution of proxies.If types do not match exactly,
NotImplemented
is returned, thus referring the comparison to the other side of the==
.
- __format__(spec_str: str) str #
Creates a formatted string from the given specification.
Invokes further methods which are prefixed by
_format_
.
- __getattr__(attr_name: str)#
Forward attributes that were not available in this class to some other attribute of the group or container.
- Parameters
attr_name (str) – The name of the attribute that was tried to be accessed but was not available in
self
.- Returns
The attribute
attr_name
ofgetattr(self, self.FORWARD_ATTR_TO)
- __sizeof__() int #
Returns the size of the data (in bytes) stored in this container’s data and its attributes.
Note that this value is approximate. It is computed by calling the
sys.getsizeof()
function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.
- __str__() str #
An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.
- _abc_impl = <_abc_data object>#
- _attrs = None#
The attribute that data attributes will be stored to
- _check_data(data: Any) None #
This method can be used to check the data provided to this container
It is called before the data is stored in the
__init__
method and should raise an exception or create a warning if the data is not as desired.This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using
super()
.Note
The
CheckDataMixin
provides a generalised implementation of this method to perform some type checks and react to unexpected types.- Parameters
data (Any) – The data to check
- _check_name(new_name: str) None #
Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.
This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().
- Parameters
new_name (str) – The new name, which is to be checked.
- _format_info() str #
A __format__ helper function: returns info about the content of this data container.
- _format_logstr() str #
A __format__ helper function: returns the log string, a combination of class name and name
- _forward_attr_get_forwarding_target()#
Get the object that the attribute call is to be forwarded to
- _forward_attr_post_hook(attr)#
Invoked before attribute forwarding occurs
- property attrs#
The container attributes.
- property data: Any#
The stored data.
- property parent#
The associated parent of this container or group
dantro.containers.xr module#
This module implements specializations of the
BaseDataContainer
class that make use of the xarray
package to represent the underlying data.
- class XrDataContainer(*, name: str, data: Union[ndarray, DataArray], dims: Sequence[str] = None, coords: dict = None, extract_metadata: bool = True, apply_metadata: bool = True, **dc_kwargs)[source]#
Bases:
dantro.mixins.general.ForwardAttrsToDataMixin
,dantro.mixins.numeric.NumbersMixin
,dantro.mixins.numeric.ComparisonMixin
,dantro.mixins.base.CheckDataMixin
,dantro.mixins.base.ItemAccessMixin
,dantro.base.BaseDataContainer
The XrDataContainer stores numerical
xarray.DataArray
data associated with dimensions, coordinates, and attributes.- DATA_EXPECTED_TYPES: tuple = ('xarray.DataArray', <class 'numpy.ndarray'>)#
Which types to allow. If None, all types are allowed.
- DATA_ALLOW_PROXY: bool = False#
Whether to allow all proxy types, i.e. classes derived from
AbstractDataProxy
.
- DATA_UNEXPECTED_ACTION = 'raise'#
The action to take when an unexpected type was supplied. Can be:
raise
,warn
,ignore
.
- _XRC_DIMS_ATTR = 'dims'#
Define as class variable the name of the attribute that determines the dimensions of the
xarray.DataArray
- _XRC_DIM_NAME_PREFIX = 'dim_name__'#
Attributes prefixed with this string can be used to set names for specific dimensions. The prefix should be followed by an integer-parsable string, e.g.
dim_name__0
would be the dimension name for the 0th dim.
- _XRC_COORDS_ATTR_PREFIX = 'coords__'#
Attributes prefixed with this string determine the coordinate values for a specific dimension. The prefix should be followed by the name of the dimension, e.g.
coord__time
. The values are interpreted according to the default coordinate mode or, if given, thecoord_mode__*
attribute.
- _XRC_COORDS_MODE_DEFAULT = 'values'#
The default mode by which coordinates are interpreted
- _XRC_COORDS_MODE_ATTR_PREFIX = 'coords_mode__'#
Prefix for the coordinate mode if a custom mode is to be used
- _XRC_INHERIT_CONTAINER_ATTRIBUTES = True#
Whether to inherit the other container attributes
- _XRC_STRICT_ATTR_CHECKING = True#
Whether to use strict attribute checking; throws errors if there are container attributes available that match the prefix but don’t match a valid dimension name. Can be disabled for speed improvements.
- __init__(*, name: str, data: Union[ndarray, DataArray], dims: Sequence[str] = None, coords: dict = None, extract_metadata: bool = True, apply_metadata: bool = True, **dc_kwargs)[source]#
Initialize a XrDataContainer and extract dimension and coordinate labels.
- Parameters
name (str) – which name to give to the XrDataContainer
data (Union[ndarray, DataArray]) – The data to store; anything that an
xarray.DataArray
can take.dims (Sequence[str], optional) – The dimension names.
coords (dict, optional) – The coordinates. The keys of this dict have to correspond to the dimension names.
extract_metadata (bool, optional) – If True, missing
dims
orcoords
arguments are tried to be populated from the container attributes.apply_metadata (bool, optional) – Whether to apply the extracted or passed
dims
andcoords
to the underlying data. This might not be desired in cases where the givendata
already is a labelledxarray.DataArray
or where the data is a proxy and the labelling should be postponed.**dc_kwargs – passed to parent
- _format_info() str [source]#
A __format__ helper function: returns info about the item.
In this case, the dtype and sizes of the stored data is returned. Depending on whether metadata is available, the shape information is shown or the dimension names and the length of the dimensions are used.
- FORWARD_ATTR_EXCLUDE: Sequence[str] = ()#
Attributes to not forward. Evaluated after
FORWARD_ATTR_ONLY
- FORWARD_ATTR_TO: str = 'data'#
The name of the existing attribute to forward to. For None, this behaves as if no forwarding would occur, i.e. as if
__getattr__
was not called.
- _ATTRS_CLS#
alias of
dantro.base.BaseDataAttrs
- __abs__()#
Absolute value
- Returns
A new object with the absolute value of the elements
- __add__(other)#
Add two objects
- Returns
A new object containing the summed data
- __bool__()#
Truth value
- __ceil__()#
Smallest integer
- Returns
A new object containing the smallest integer
- __delitem__(key)#
Deletes an item
- __divmod__(other)#
Calculate the floor division and modulo of two objects
- Returns
A new object containing the floor divided data and its modulo
- __eq__(other)#
Equality
- __floor__()#
Largest integer
- Returns
A new object containing the largest element
- __floordiv__(other)#
Floor divide two objects
- Returns
A new object containing the floor divided data
- __format__(spec_str: str) str #
Creates a formatted string from the given specification.
Invokes further methods which are prefixed by
_format_
.
- __ge__(other)#
Greater than or equal
- __getattr__(attr_name: str)#
Forward attributes that were not available in this class to some other attribute of the group or container.
- Parameters
attr_name (str) – The name of the attribute that was tried to be accessed but was not available in
self
.- Returns
The attribute
attr_name
ofgetattr(self, self.FORWARD_ATTR_TO)
- __getitem__(key)#
Returns an item.
- __gt__(other)#
Greater than
- __iadd__(other)#
Add two objects
- Returns
Self with modified data
- __ifloordiv__(other)#
Floor divide two objects
- Returns
Self with modified data
- __imod__(other)#
Calculate the modulo of two objects
- Returns
Self with modified data
- __imul__(other)#
Multiply two objects
- Returns
Self with modified data
- __invert__()#
Inverse value
- Returns
A new object with the inverted values of the elements
- __ipow__(other)#
Calculate the self data to the power of other data
- Returns
Self with modified data
- __isub__(other)#
Subtract two objects
- Returns
Self with modified data
- __itruediv__(other)#
Divide two objects
- Returns
Self with modified data
- __le__(other)#
Less than or equal
- __lt__(other)#
Less than
- __mod__(other)#
Calculate the modulo of two objects
- Returns
A new object containing the summed data
- __mul__(other)#
Multiply two objects
- Returns
A object containing the multiplied data
- __ne__(other)#
Inequality
- __neg__()#
Make negative
- Returns
A new object with negative elements
- __pos__()#
Make positive
- Returns
A new object with negative elements
- __pow__(other)#
Calculate the self data to the power of other data
- Returns
A new object containing the result
- __round__()#
Rounds number to nearest integer
- Returns
A new object as rounded number to nearest integer
- __setitem__(key, val)#
Sets an item.
- __sizeof__() int #
Returns the size of the data (in bytes) stored in this container’s data and its attributes.
Note that this value is approximate. It is computed by calling the
sys.getsizeof()
function on the data, the attributes, the name and some caching attributes that each dantro data tree class contains. Importantly, this is not a recursive algorithm.Also, derived classes might implement further attributes that are not taken into account either. To be more precise in a subclass, create a specific __sizeof__ method and invoke this parent method additionally.
- __str__() str #
An info string, that describes the object. This invokes the formatting helpers to show the log string (type and name) as well as the info string of this object.
- __sub__(other)#
Subtract two objects
- Returns
A new object containing the subtracted data
- __truediv__(other)#
Divide two objects
- Returns
A new object containing the divided data
- __trunc__()#
Truncated to the nearest integer toward 0
- Returns
A new object containing the truncated element
- _abc_impl = <_abc_data object>#
- _attrs = None#
The attribute that data attributes will be stored to
- _check_data(data) None #
A general method to check the received data for its type
- Parameters
data – The data to check
- Raises
TypeError – If the type was unexpected and the action was ‘raise’
ValueError – Illegal value for
DATA_UNEXPECTED_ACTION
class variable
- Returns
None
- _check_name(new_name: str) None #
Called from name.setter and can be used to check the name that the container is supposed to have. On invalid name, this should raise.
This method can be subclassed to implement more specific behaviour. To propagate the parent classes’ behaviour the subclassed method should always call its parent method using super().
- Parameters
new_name (str) – The new name, which is to be checked.
- _format_logstr() str #
A __format__ helper function: returns the log string, a combination of class name and name
- _forward_attr_get_forwarding_target()#
Get the object that the attribute call is to be forwarded to
- _forward_attr_post_hook(attr)#
Invoked before attribute forwarding occurs
- _item_access_convert_list_key(key)#
If given something that is not a list, just return that key
- property attrs#
The container attributes.
- copy(*, deep: bool = True)[source]#
Return a new object with a copy of the data. The copy is deep if not specified otherwise.
- Parameters
deep (bool, optional) – Whether the copy is deep
- Returns
A (deep) copy of this object.
- Return type
- property data: Any#
The stored data.
- property parent#
The associated parent of this container or group
- save(path: str, **save_kwargs)[source]#
Saves the XrDataContainer to a file by invoking the
.to_netcdf
method of the underlying data.The recommended file extension is
.xrdc
or.nc_da
, which are compatible with the xarray-based data loader.Warning
This does NOT store container attributes!
- Parameters
path (str) – The path to save the file at
**save_kwargs – Passed to
.no_netcdf
method call
- _extract_metadata()[source]#
Extracts metadata from the container attributes and stores them in the
_dim_names
and_dim_to_coords_map
cache attributes.
- _inherit_attrs()[source]#
Carry over container attributes to the data array attributes.
This does not include container attributes that are used for extracting metadata; it makes no sense to have them in the attributes of the already labelled
xarray.DataArray
.
- _apply_metadata()[source]#
Applies the cached metadata to the underlying
xarray.DataArray
- _postprocess_proxy_resolution()[source]#
Only invoked from
ProxySupportMixin
, which have to be added to the class specifically. This function takes care to apply the potentially existing metadata after the proxy was resolved.