dantro.plot_mngr module

This module implements the PlotManager class, which handles the configuration of multiple plots and prepares the data and configuration to pass to the PlotCreator.

dantro.plot_mngr.fmt_time(seconds)
class dantro.plot_mngr.PlotManager(*, dm: dantro.data_mngr.DataManager, base_cfg_pools: Sequence[Tuple[str, Union[dict, str]]] = (), plots_cfg: Union[dict, str] = None, default_plots_cfg: Union[dict, str] = None, base_cfg: Union[str, dict] = None, update_base_cfg: Union[str, dict] = None, out_dir: Optional[str] = '{timestamp:}/', out_fstrs: dict = None, creator_init_kwargs: Dict[str, dict] = None, default_creator: str = None, auto_detect_creator: bool = False, save_plot_cfg: bool = True, raise_exc: bool = False, cfg_exists_action: str = 'raise')[source]

Bases: object

The PlotManager takes care of configuring plots and calling the configured PlotCreator classes that then carry out the plots.

CREATORS

The mapping of creator names to classes. When it is desired to subclass PlotManager and extend the creator mapping, use dict(**pcr.ALL) to inherit the default creator mapping.

Type

dict

DEFAULT_OUT_FSTRS

The default values for the output format strings.

Type

dict

SPECIAL_BASE_CFG_POOL_LABELS

Special keys that may not be used as labels for the base configuration pools.

Type

Tuple[str]

CREATORS = {'declarative': <class 'dantro.plot_creators.pcr_decl.DeclarativePlotCreator'>, 'external': <class 'dantro.plot_creators.pcr_ext.ExternalPlotCreator'>, 'multiverse': <class 'dantro.plot_creators.pcr_psp.MultiversePlotCreator'>, 'universe': <class 'dantro.plot_creators.pcr_psp.UniversePlotCreator'>, 'vega': <class 'dantro.plot_creators.pcr_vega.VegaPlotCreator'>}
DEFAULT_OUT_FSTRS = {'path': '{name:}{ext:}', 'plot_cfg': '{basename:}_cfg.yml', 'plot_cfg_sweep': '{name:}/sweep_cfg.yml', 'state': '{name:}_{val:}', 'state_join_char': '__', 'state_name_replace_chars': [], 'state_no': '{no:0{digits:d}d}', 'state_val_replace_chars': [('/', '-')], 'state_vector_join_char': '-', 'sweep': '{name:}/{state_no:}__{state:}{ext:}', 'timestamp': '%y%m%d-%H%M%S'}
SPECIAL_BASE_CFG_POOL_LABELS = ('plot', 'plot_from_cfg', 'plot_from_cfg_unused', 'plot_pspace')
__init__(*, dm: dantro.data_mngr.DataManager, base_cfg_pools: Sequence[Tuple[str, Union[dict, str]]] = (), plots_cfg: Union[dict, str] = None, default_plots_cfg: Union[dict, str] = None, base_cfg: Union[str, dict] = None, update_base_cfg: Union[str, dict] = None, out_dir: Optional[str] = '{timestamp:}/', out_fstrs: dict = None, creator_init_kwargs: Dict[str, dict] = None, default_creator: str = None, auto_detect_creator: bool = False, save_plot_cfg: bool = True, raise_exc: bool = False, cfg_exists_action: str = 'raise')[source]

Initialize a PlotManager, which provides a uniform configuration interface for creating plots and passes tasks on to the respective plot creators.

To avoid copy-paste of plot configurations, the PlotManager comes with versatile capabilities to define default plots and re-use other plots.

  • The default_plots_cfg specifies plot configurations that are to be carried out by default when calling the plotting method plot_from_cfg().

  • When calling any of the plot methods plot_from_cfg() or plot(), there is the possibility to update the existing configuration dict with new entries.

  • At each stage, the based_on feature allows to make a plot configuration inherit entries from an existing configuration. These are looked up from the base_cfg_pools following the rules described in resolve_based_on().

For more information on how the plot configuration can be defined, see Plot Configuration Inheritance.

Parameters
  • dm (DataManager) – The DataManager-derived object to read the plot data from.

  • base_cfg_pools (Sequence[Tuple[str, Union[dict, str]]], optional) – The base configuration pools are used to perform the lookups of based_on entries. The tuples in these sequence consist of (label, plots_cfg) pairs and are fed to add_base_cfg_pool(). This argument can also be provided as an OrderedDict.

  • default_plots_cfg (Union[dict, str], optional) – The default plots config or a path to a YAML file to import. Used as defaults when calling plot_from_cfg()

  • base_cfg (Union[str, dict], optional) – Deprecated! Legacy interface for defining base plot configurations. During the deprecation period, these entries are automatically added to base_cfg_pools under the label base.

  • update_base_cfg (Union[str, dict], optional) – Deprecated! Used to update the base_cfg before adding ot to base_cfg_pools.

  • out_dir (Union[str, None], optional) – If given, will use this output directory as basis for the output path for each plot. The path can be a format-string; it is evaluated upon call to the plot command. Available keys: timestamp, name, … For a relative path, this will be relative to the DataManager’s output directory. Absolute paths remain absolute. If this argument evaluates to False, the DataManager’s output directory will be the output directory.

  • out_fstrs (dict, optional) –

    Format strings that define how the output path is generated. The dict given here updates the DEFAULT_OUT_FSTRS class variable which holds the default values.

    Keys: timestamp (%-style), path, sweep, state, plot_cfg, state, state_no, state_join_char, state_vector_join_char.

    Available keys for path: name, timestamp, ext.

    Additionally, for sweep: state_no, state_vector,

    state.

  • creator_init_kwargs (Dict[str, dict], optional) – If given, these kwargs are passed to the initialization calls of the respective creator classes.

  • default_creator (str, optional) – If given, a plot without explicit creator declaration will use this creator as default.

  • auto_detect_creator (bool, optional) – If true, and no default creator is given, will try to automatically deduce the creator using the given plot arguments. All creators registered with this PlotManager instance are candidates.

  • save_plot_cfg (bool, optional) – If True, the plot configuration is saved to a yaml file alongside the created plot.

  • raise_exc (bool, optional) – Whether to raise exceptions if there are errors raised from the plot creator or errors in the plot configuration. If False, the errors will only be logged.

  • cfg_exists_action (str, optional) – Behaviour when a config file already exists. Can be: raise (default), skip, append, overwrite, or overwrite_nowarn.

Raises

InvalidCreator – When trying to set an invalid default_creator

property out_fstrs

The dict of output format strings

property plot_info

A list of dicts with info on all plots carried out so far

property base_cfg_pools

The base plot configuration pools, used for lookup the based_on entry in plot configurations.

The order of the entries in the pool is relevant, with later entries taking precedence over previous ones. See Plot Configuration Inheritance for a more detailed description.

property default_creator

The name of the default creator

add_base_cfg_pool(*, label: str, plots_cfg: Union[str, dict])[source]

Adds a base configuration pool entry, allowing for the plots_cfg to be a path to a YAML configuration file which is then loaded.

The new pool is used for based_on lookups and takes precedence over existing entries. For more information on lookup rules, see resolve_based_on() and Plot Configuration Inheritance.

Parameters
  • label (str) – A label of the pool that is used for identifying it.

  • plots_cfg (Union[str, dict]) – Description

Raises

ValueError – If label already exists or is a special label.

static _prepare_cfg(s: Union[str, dict]) → dict[source]

Prepares a plots configuration by either loading it from a YAML file if the given argument is a string or returning a deep copy of the given dict-like object.

_handle_exception(exc: Exception, *, pc: dantro.plot_creators.pcr_base.BasePlotCreator, debug: bool = None, ExcCls: type = <class 'dantro.exceptions.PlottingError'>)[source]

Helper for handling exceptions from the plot creator

_parse_out_dir(fstr: str, *, name: str) → str[source]

Evaluates the format string to create an output directory path.

Note that the directories are _not_ created; this is outsourced to the plot creator such that it happens as late as possible.

Parameters
  • fstr (str) – The format string to evaluate and create a directory at

  • name (str) – Name of the plot

  • timestamp (float, optional) – Description

Returns

The path of the created directory

Return type

str

_parse_out_path(creator: dantro.plot_creators.pcr_base.BasePlotCreator, *, name: str, out_dir: str, file_ext: str = None, state_no: int = None, state_no_max: int = None, state_vector: Tuple[int] = None, dims: dict = None) → str[source]

Given a creator and (optionally) parameter sweep information, a full and absolute output path is generated, including the file extension.

Note that the directories are _not_ created; this is outsourced to the plot creator such that it happens as late as possible.

Parameters
  • creator (BasePlotCreator) – The creator instance, used to extract information on the file extension.

  • name (str) – The name of the plot

  • out_dir (str) – The absolute output directory, prepended to all generated paths

  • file_ext (str, optional) – The file extension to use

  • state_no (int, optional) – The state number, starting with 0

  • state_no_max (int, optional) – The maximum state number

  • state_vector (Tuple[int], optional) – The state vector with info on how far each state dimension has progressed in the sweep

  • dims (dict, optional) – The dict of parameter dimensions of the sweep that is carried out.

Returns

The fully parsed output path for this plot

Return type

str

_check_plot_name(name: str) → None[source]

Raises if a plot name contains bad characters

_get_plot_creator(creator: Optional[str], *, name: str, init_kwargs: dict, from_pspace: paramspace.paramspace.ParamSpace = None, plot_cfg: dict, auto_detect: bool = None) → dantro.plot_creators.pcr_base.BasePlotCreator[source]

Determines which plot creator to use by looking at the given arguments. If set, tries to auto-detect from the arguments, which creator is to be used.

Then, sets up the corresponding creator and returns it.

This method is called from the plot() method.

Parameters
  • creator (Union[str, None]) – The name of the creator to be found. Can be None, if no argument was given to the plot method.

  • name (str) – The name of the plot

  • init_kwargs (dict) – Additional creator initialization parameters

  • from_pspace (ParamSpace, optional) – If the plot is to be creatd from a parameter space, that parameter space.

  • plot_cfg (dict) – The plot configuration

  • auto_detect (bool, optional) – Whether to auto-detect the creator. If none, the value given at initialization is used.

Returns

The selected creator object, fully initialized.

Return type

BasePlotCreator

Raises

InvalidCreator – If the creator argument was invalid or auto- detection failed.

_invoke_creator(plot_creator: dantro.plot_creators.pcr_base.BasePlotCreator, *, out_path: str, debug: bool = None, **plot_cfg) → Union[bool, str][source]

This method wraps the plot creator’s __call__ and is the last PlotManager method that is called prior to handing over to the selected plot creator. It takes care of invoking the plot creator’s __call__ method and handling potential error messages and return values.

Parameters
  • plot_creator (BasePlotCreator) – The currently used creator object

  • out_path (str) – The plot output path

  • debug (bool, optional) – If given, this overwrites the raise_exc option specified during initialization.

  • **plot_cfg – The plot configuration

Returns

Whether the plot was carried out successfully.

Returns the string 'skipped' if the plot was skipped via a SkipPlot exception.

Return type

Union[bool, str]

Raises

PlotCreatorError – On error within the plot creator. This is only raised if either debug is True or debug is None and self.raise_exc. Otherwise, the error message is merely logged.

_store_plot_info(name: str, *, plot_cfg: dict, creator_name: str, save: bool, target_dir: str, **info)[source]

Stores all plot information in the plot_info list and, if save is set, also saves it using the _save_plot_cfg method.

_save_plot_cfg(cfg: dict, *, name: str, creator_name: str, target_dir: str, exists_action: str = None, is_sweep: bool = False) → str[source]

Saves the given configuration under the top-level entry name to a yaml file.

Parameters
  • cfg (dict) – The plot configuration to save

  • name (str) – The name of the plot

  • creator_name (str) – The name of the creator

  • target_dir (str) – The directory path to store the file in

  • exists_action (str, optional) – What to do if a plot configuration already exists. Can be: overwrite, overwrite_nowarn, skip, append, raise. If None, uses the value of the cfg_exists_action argument given during initialization.

  • is_sweep (bool, optional) – Set if the configuration refers to a plot in sweep mode, for which a different format string is used

Returns

The path the config was saved at (mainly used for testing)

Return type

str

Raises

ValueError – For invalid exists_action argument

plot_from_cfg(*, plots_cfg: Union[dict, str] = None, plot_only: List[str] = None, out_dir: str = None, resolve_based_on: bool = True, **update_plots_cfg) → None[source]

Create multiple plots from a configuration, either a given one or the one passed during initialization.

This is mostly a wrapper around the plot function, allowing additional ways of how to configure and create plots.

Parameters
  • plots_cfg (Union[dict, str], optional) – The plots configuration to use. If not given, the default_plots_cfg specified during initialization is used. If a string is given, will assume it is a path and load the file.

  • plot_only (List[str], optional) – If given, create only those plots from the resulting configuration that match these names. This will lead to the enabled key being ignored, regardless of its value. The strings given here may also include Unix shell-like wildcards like * and ? ``, which are matched using the Python ``fnmatch module.

  • out_dir (str, optional) – A different output directory; will use the one passed at initialization if the given argument evaluates to False.

  • resolve_based_on (bool, optional) – Whether to resolve the based_on entries in plots_cfg here. If false, will postpone this to plot(), thus not including the rest of the plots_cfg in the base configuration pool for name resolution. Lookups happen from base_cfg_pools following the rules described in resolve_based_on().

  • **update_plots_cfg – If given, it is used to update the plots_cfg recursively. Note that on the top level the _names_ of the plots are placed; this cannot be used to make all plots have a common property. Furthermore, this update happens before the based_on entries are resolved.

Raises
  • PlotConfigError – Empty or invalid plot configuration

  • ValueError – Bad plot_only argument, e.g. not matching any of the available plot names.

plot(name: str, *, based_on: Union[str, Tuple[str]] = None, from_pspace: Union[dict, paramspace.paramspace.ParamSpace] = None, **plot_cfg) → dantro.plot_creators.pcr_base.BasePlotCreator[source]

Create plot(s) from a single configuration entry.

A call to this function resolves the based_on feature and passes the derived plot configuration to self._plot(), which actually carries out the plots.

Note that more than one plot can result from a single configuration entry, e.g. when plots were configured that have more dimensions than representable in a single file.

Parameters
  • name (str) – The name of this plot. This will be used for generating an output file path later on. Some characters are not allowed, e.g. * and ?, but a / can be used to store the plot output in a subdirectory.

  • based_on (Union[str, Tuple[str]], optional) – A key or a sequence of keys of entries in the base pool that should be used as the basis of this plot. The given plot configuration is then used to recursively update (a copy of) those base configuration entries. Lookups happen from base_cfg_pools following the rules described in resolve_based_on().

  • from_pspace (Union[dict, ParamSpace], optional) – If given, execute a parameter sweep over these parameters, re-using the same creator instance. If this is a dict, a ParamSpace is created from it.

  • **plot_cfg – The plot configuration, including some parameters that the plot manager will evaluate (and consequently: does not pass on to the plot creator). If using from_pspace, parameters given here will recursively update those given in from_pspace.

Returns

The PlotCreator used for these plots

Return type

BasePlotCreator

_plot(name: str, *, creator: str = None, out_dir: str = None, default_out_dir: str = None, file_ext: str = None, save_plot_cfg: bool = None, auto_detect_creator: bool = None, creator_init_kwargs: dict = None, from_pspace: dict = None, **plot_cfg) → dantro.plot_creators.pcr_base.BasePlotCreator[source]

Create plot(s) from a single configuration entry.

A call to this function creates a single PlotCreator, which is also returned after all plots are finished.

Note that more than one plot can result from a single configuration entry, e.g. when plots were configured that have more dimensions than representable in a single file or when using from_pspace.

Parameters
  • name (str) – The name of this plot

  • creator (str, optional) – The name of the creator to use. Has to be part of the CREATORS class variable. If not given, the argument default_creator given at initialization will be used.

  • out_dir (str, optional) – If given, will use this directory as out directory. If not, will use the default value given by default_out_dir or that given at initialization.

  • default_out_dir (str, optional) – An output directory that was determined in the calling context and which should be used as default if no out_dir was given explicitly.

  • file_ext (str, optional) – The file extension to use, including the leading dot!

  • save_plot_cfg (bool, optional) – Whether to save the plot config. If not given, uses the default value from initialization.

  • auto_detect_creator (bool, optional) – Whether to attempt auto- detection of the creator argument. If given, this argument overwrites the value given at PlotManager initialization.

  • creator_init_kwargs (dict, optional) – Passed to the plot creator during initialization. Note that the arguments given at initialization of the PlotManager are updated by this.

  • from_pspace (dict, optional) – If given, execute a parameter sweep over this parameter space, re-using the same creator instance. Each point in parameter space will end up calling this method with arguments unpacked to the plot_cfg argument.

  • **plot_cfg – The plot configuration to pass on to the plot creator. This may be completely empty if from_pspace is used!

Returns

The PlotCreator used for these plots. This will

also be returned in case the plot failed!

Return type

BasePlotCreator

Raises
  • PlotConfigError – If no out directory was specified here or at initialization.

  • PlotCreatorError – In case the preparation or execution of the plot failed for whatever reason. Not raised if not in debug mode.