Search paths

Search paths#

It is possible to extend DLite with new datamodels, code generation templates and plugins, by appending to corresponding search path. The table below lists the different types of search paths that are available in DLite.

Search path type	Description
storages	Storage URLs or directory paths to datamodels
templates	Directory paths to code generator templates
storage_plugins	Directory paths to storage plugins (drivers) written in C
mapping-plugins	Directory paths to mapping plugins written in C
python-storage-plugins	Directory paths to storage plugins (drivers) written in Python
python-mapping-plugins	Directory paths to mapping plugins written in Python
python-protocol-plugins	Directory paths to protocol plugins written in Python

Search paths can be extended in different three ways:

setting environment variables
appending to DLite path variables (from Python)
using entry points (only in user-defined Python packages)

The table below lists the lists the name of the environment variables and Python path variables corresponding to the different types of search paths.

Search path type	Environment variable name	Python variable name
storages	DLITE_STORAGES	dlite.storage_path
templates	DLITE_TEMPLATE_DIRS	dlite.template_path
storage_plugins	DLITE_STORAGE_PLUGIN_DIRS	dlite.storage_plugin_path
mapping-plugins	DLITE_MAPPING_PLUGIN_DIRS	dlite.mapping-plugin_path
python-storage-plugins	DLITE_PYTHON_STORAGE_PLUGIN_DIRS	dlite.python-storage-plugin_path
python-mapping-plugins	DLITE_PYTHON_MAPPING_PLUGIN_DIRS	dlite.python-mapping-plugin_path
python-protocol-plugins	DLITE_PYTHON_PROTOCOL_PLUGIN_DIRS	dlite.python-protocol-plugin_path

Setting environment variables#

This is typically done where you set up your environment, like in a virtualenv activate script or the users ~/.bashrc file.

Note

All the path variables, except for DLITE_STORAGES, uses (:) colon as path separator. However, since colon may appears in URLs, DLITE_STORAGES uses instead the pipe symbol (|) as path separator.

See environment variables for more details.

Appending to DLite path variables#

The dlite Python module defines the path variables listed in the table above. A Python script or application can configure new datamodels and plugins by appending to these variables.

Using entry points#

How to use entry points is easiest described with an example. Let us assume you have a package with the following directory layout:

project_root
├── mypackage
│   ├── __init__.py
│   ├── mymodule.py
│   ├── data
│   │   ├── python_storage_plugins
│   │   │   ├── myplugin.py
│   │   │   └── anotherplugin.py
│   │   └── datamodels
│   │       ├── mydatamodel.yaml
│   │       └── anotherdatamodel.yaml
├── pyproject.toml
├── README.md
└── LICENSE

To make your datamodels and Python storage plugins available for users of your package, you can add the following section to your pyproject.toml:

[tool.setuptools.package-data]
"mypackages.data.datamodels" = ["*.json", "*.yaml"]
"mypackages.data.python_storage_plugins" = ["*.py"]

# Note the quotes around dlite.python_storage_plugins to escape the embedded dot
[project.entry-points."dlite.python_storage_plugins"]
storage_plugin_path = "mypackage:data/python_storage_plugins"

[project.entry-points."dlite.storages"]
datamodel_path = "mypackage:data/datamodels"

See the Setuptools documentation for how to this can be done with setup.py or setup.cfg.