.. _welcome:

Welcome to dantro's documentation!
==================================

:py:mod:`dantro` – from *data* and *dentro* (Greek for *tree*) – is a Python package that provides a uniform interface for hierarchically structured and semantically heterogeneous data.
It is built around three main features:

-  **data handling:** loading heterogeneous data into a tree-like data structure, providing a uniform interface to it
-  **data transformation:** performing arbitrary operations on the data, if necessary using lazy evaluation
-  **data visualization:** creating a visual representation of the processed data

Together, these stages constitute a **data processing pipeline**:
an automated sequence of predefined, configurable operations.
Akin to a Continuous Integration pipeline, a data processing pipeline provides a uniform, consistent, and easily extensible infrastructure that contributes to more efficient and reproducible workflows.
This can be beneficial especially in a scientific context, for instance when handling data that was generated by computer simulations.

:py:mod:`dantro` is meant to be *integrated* into projects and be used to set up such a data processing pipeline, customized to the needs of the project.
It is designed to be **easily customizable** to the requirements of the project it is integrated in, even if the involved data is hierachically structured or semantically heterogeneous.
Furthermore, it allows a **configuration-based specification** of all operations via `YAML <https://en.wikipedia.org/wiki/YAML>`_ configuration files; the resulting pipeline can then be controlled entirely via these configuration files and without requiring code changes.

The :py:mod:`dantro` package is **open source software** released under the `LGPLv3+ <https://www.gnu.org/licenses/lgpl-3.0.html>`_ license.
It was developed alongside the `Utopia project <https://gitlab.com/utopia-project/utopia>`_ (a modelling framework for complex and adaptive systems), but is an independent package.

.. hint::

    A description paper about the **motivation and scope of dantro** can be found in `in the Journal of Open Source Software <https://doi.org/10.21105/joss.02316>`_.

    For a real-world example of how dantro is used, make sure to check out `the Utopia project <https://utopia-project.org/>`_, where the dantro-based data processing pipeline is fed with the output of complex systems models.

.. note::

    If you find any errors in this documentation or would like to contribute to the project, we are happy about your visit to the `project page <https://gitlab.com/utopia-project/dantro>`_.


.. toctree::
    :hidden:

    usage
    integrating
    specializing
    philosophy

.. toctree::
    :hidden:

    Repository <https://gitlab.com/utopia-project/dantro>
    Changelog <https://gitlab.com/utopia-project/dantro/-/blob/main/CHANGELOG.md>
    JOSS Publication <https://doi.org/10.21105/joss.02316>
    how-to-cite
    Utopia Project <https://utopia-project.org/>

.. toctree::
    :caption: Data Handling and Transformation
    :maxdepth: 3
    :hidden:

    data_io/data_mngr
    data_io/data_ops
    data_io/transform
    data_io/large_data
    data_io/faq

.. toctree::
    :caption: Data Structures
    :maxdepth: 3
    :hidden:

    data_structures/index

.. toctree::
    :caption: Plotting
    :maxdepth: 3
    :hidden:

    plotting/plot_manager
    plotting/plot_creators
    plotting/plot_helper
    plotting/plot_data_selection
    plotting/plot_functions
    plotting/color_mngr
    plotting/faq
    plotting/examples

.. toctree::
    :caption: Reference
    :maxdepth: 2
    :hidden:

    API Reference <api/dantro>
    data_io/data_ops_ref
    plotting/plot_cfg_ref
    plotting/base_plots
    LGPLv3 License <https://www.gnu.org/licenses/lgpl-3.0.html>
    index_pages