Specializing `dantro` Classes¶

This page shows a few examples of how to specialize dantro classes to your liking. This step is an important aspect of adapting dantro to work with the data structures that you are frequently using, which is beneficial for good integration into your workflow.

Specializing a data container
- Using a specialized data container
- Configuring mixins
Specializing the DataManager

Specializing a data container ¶

As an example, let’s look at the implementation of the MutableSequenceContainer, a container that is meant to store mutable sequences:

# Import the python abstract base class we want to adhere to
from collections.abc import MutableSequence

# Import base container class and the mixins we would like to use
from dantro.base import BaseDataContainer
from dantro.mixins import ItemAccessMixin, CollectionMixin, CheckDataMixin

class MutableSequenceContainer(CheckDataMixin,
                               ItemAccessMixin,
                               CollectionMixin,
                               BaseDataContainer,
                               MutableSequence):
    """The MutableSequenceContainer stores sequence-like mutable data"""

The steps to arrive at this point are as follows:

The collections.abc python module is also used by python to specify the interfaces for python-internal classes. In the documentation it says that the MutableSequence inherits from Sequence and has the following abstract methods: __getitem__, __setitem__, __delitem__, __len__, and insert.

As we want the resulting container to adhere to this interface, we set MutableSequence as the first class to inherit from. The BaseDataContainer is what makes this object a dantro data container. It implements some of the required methods to concur with the MutableSequence interface but leaves others abstract.

Now, we need to supply implementations of these abstract methods. That is the job of the following two (reading from right to left) mixin classes. In this case, the Sequence interface has to be fulfilled. As a Sequence is nothing more than a Collection with item access, we can fulfill this by inheriting from the CollectionMixin and the ItemAccessMixin.

The CheckDataMixin is an example of how functionality can be added to the container while still adhering to the interface. This mixin checks the provided data before storing it and allows specifying whether unexpected data should lead to warnings or exceptions; for an example, see below

Some methods will remain abstract, in this case: insert. These need to be manually defined; the MutableSequenceContainer’s insert() method does exactly that, thus becoming a fully non-abstract class:

def insert(self, idx: int, val) -> None:
    """Insert an item at a given position.

    Args:
        idx (int): The index before which to insert
        val: The value to insert
    """
    self.data.insert(idx, val)

Using a specialized data container ¶

Once defined, instantiation of a custom container works the same way as for other data containers:

dc = MutableSequenceContainer(name="my_mutable_sequence",
                              data=[4, 8, 16])

# Insert values
dc.insert(0, 2)
dc.insert(0, 1)

# Item access and collection interface
assert 16 in dc
assert 32 not in dc
assert dc[0] == 1

for num in dc:
    print(num, end=", ")
# prints:  1, 2, 4, 8, 16,

Configuring mixins ¶

Many mixins allow some form of configuration. This typically happens via class variables.

Let’s define a new container that strictly requires its stored data to be a list, i.e. an often-used mutable sequence type. We can use the already-included CheckDataMixin such that it checks a type. To do so, we set the DATA_EXPECTED_TYPES to only allow list and we set DATA_UNEXPECTED_ACTION to raise an exception if this is not the case.

class StrictlyListContainer(MutableSequenceContainer):
    """A MutableSequenceContainer that allows only a list as data"""
    DATA_EXPECTED_TYPES = (list,)     # as tuple or None (allow all)
    DATA_UNEXPECTED_ACTION = 'raise'  # can be: raise, warn, ignore

# This will work
some_list = StrictlyListContainer(name="some_list", data=["foo", "bar"])

# The following will fail
with pytest.raises(TypeError):
    StrictlyListContainer(name="some_tuple", data=("foo", "bar"))

with pytest.raises(TypeError):
    StrictlyListContainer(name="some_tuple", data="just some string")

Other mixins provide other class variables for specializing behavior. Consult the documentation or the source code to find out which ones.

Note

The class variables typically define the default behavior for a certain specialized type. However, depending on the mixin, its behavior might also depend on runtime information, e.g. specified in __init__.

Warning

We advise against overwriting class variables during the lifetime of an object.

Specializing the `DataManager`¶

This works in essentially the same way: A DataManager is specialized by adding data_loaders mixin classes.

import dantro
from dantro.data_loaders import YamlLoaderMixin, PickleLoaderMixin

class MyDataManager(PickleLoaderMixin,
                    YamlLoaderMixin,
                    dantro.DataManager):
    """A DataManager specialization that can load pickle and yaml data"""

That’s all.

For more information, see The DataManager.

Note

As an example, you can have a look at data manager used in utopya.

Specializing dantro Classes¶

Specializing a data container¶

Using a specialized data container¶

Configuring mixins¶

Specializing the DataManager¶