dlite-storage-plugins#

Common API for all storage plugins (internal).

A DLite storage plugin should be a shared library that defines the function

const DLiteStoragePlugin *get_dlite_storage_api(const char *name);

that returns a pointer to a struct with pointers to all functions provided by the plugin. The name is just a hint that plugins are free to ignore. It is used by storage plugins that supports several different drivers, to select which api that should be returned.

The storage plugin search path is initialised from the environment variable DLITE_STORAGE_PLUGIN_DIRS.

Two APIs

Storage plugins may choose to implement either the datamodel API or the instance API.

The datamodel API is the original API resembling SOFT5. It provides a generic abstract layer between the data representation in the storage and the DLite instances.

Since DLite now offers an abstract datamodel-like API for all instances, the need for the datamodel layer seems to be gone. DLite therefore offers an alternative and simpler API for storage plugins that works directly on DLite instances and only contains two functions; LoadInstance() and SaveInstance().

Plugin frontend

typedef const DLiteStoragePlugin *(*GetDLiteStorageAPI)(DLiteGlobals *g, int *iter)#

Returns a pointer to a DLiteStoragePlugin or NULL on error.

The iter argument is normally ignored. It is provided to support plugins exposing several APIs. If the plugin has more APIs to expose, it should increase the integer pointed to by iter by one.

const DLiteStoragePlugin *dlite_storage_plugin_get(const char *name)#

Returns a storage plugin with the given name, or NULL if it cannot be found.

If a plugin with the given name is registered, it is returned.

Otherwise the plugin search path is checked for shared libraries matching name.EXT where EXT is the extension for shared library on the current platform (“dll” on Windows and “so” on Unix/Linux). If a plugin with the provided name is found, it is loaded, registered and returned.

Otherwise the plugin search path is checked again, but this time for any shared library. If a plugin with the provided name is found, it is loaded, registered and returned.

Otherwise NULL is returned.

int dlite_storage_plugin_load_all()#: Load all plugins that can be found in the plugin search path.

void dlite_storage_plugin_unload_all()#: Unloads and unregisters all storage plugins.

DLiteStoragePluginIter *dlite_storage_plugin_iter_create()#: Returns a pointer to a new plugin iterator or NULL on error. It should be free’ed with dlite_storage_plugin_iter_free().

const DLiteStoragePlugin *dlite_storage_plugin_iter_next(DLiteStoragePluginIter *iter)#: Returns pointer the next plugin or NULL if there a re no more plugins. iter is the iterator returned by dlite_storage_plugin_iter_create().

void dlite_storage_plugin_iter_free(DLiteStoragePluginIter *iter)#: Frees plugin iterator iter created with dlite_storage_plugin_iter_create().

int dlite_storage_plugin_unload(const char *name)#: Unloads and unregisters storage plugin with the given name. Returns non-zero on error.

FUPaths *dlite_storage_plugin_paths_get(void)#: Returns a pointer to the underlying FUPaths object for storage plugins or NULL on error.

const char **dlite_storage_plugin_paths(void)#

Returns a pointer to the current storage plugin search path. It is initialised from the environment variable DLITE_STORAGE_PLUGIN_DIRS.

Use dlite_storage_plugin_path_insert(), dlite_storage_plugin_path_append() and dlite_storage_plugin_path_remove() to modify it.

char *dlite_storage_plugin_path_string(void)#

Returns an allocated string with the content of paths formatted according to the current platform. See dlite_set_platform().

Returns NULL on error.

int dlite_storage_plugin_path_insert(int n, const char *path)#

Inserts path into the current search path at index n. If n is negative, it counts from the end of the search path (like Python).

If n is out of range, it is clipped.

Returns non-zero on error.

int dlite_storage_plugin_path_append(const char *path)#

Appends path into the current search path.

Returns the index of the newly appended element or -1 on error.

int dlite_storage_plugin_path_appendn(const char *path, size_t n)#

Like dlite_storage_plugin_path_append(), but appends at most the first n bytes of path to the current search path.

Returns the index of the newly appended element or -1 on error.

int dlite_storage_plugin_path_remove_index(int index)#

Removes path number n from current search path.

Returns non-zero on error.

int dlite_storage_plugin_path_remove(const char *path)#

Removes path path from current search path.

Returns non-zero if there is no such path.

Basic API

Signatures of basic functions. Open() and Close() must always be defined. Flush() is optional.

typedef DLiteStorage *(*Open)(const DLiteStoragePlugin *api, const char *uri, const char *options)#

Opens uri and returns a newly created storage for it.

The api argument can normally be ignored (it is needed for the Python storage backend).

The options argument provies additional input to the driver. Which options that are supported varies between the plugins. It should be a valid URL query string of the form:

key1=value1;key2=value2...

An ampersand (&) may be used instead of the semicolon (;).

Typical options supported by most drivers include:

mode : append | r | w Valid values are:
- append Append to existing file or create new file (default)
- r Open existing file for read-only
- w Truncate existing file or create new file

Returns NULL on error.

typedef int (*Close)(DLiteStorage *s)#: Closes storage s. Returns non-zero on error.

typedef int (*Flush)(DLiteStorage *s)#: Flush storage s. Optional. Returns non-zero on error.

typedef char *(*Help)(const DLiteStoragePlugin *api)#: Returns a malloc’ed string with plugin documentation or NULL on error. Optional.

Queue API

Function for querying the content of a plugin. The new IterCreate(), IterNext() and IterFree() are preferred in front of the old GetUUIDs() function.

Any of these functions are fully optional for the storage plugin to implement.

typedef void *(*IterCreate)(const DLiteStorage *s, const char *pattern)#

Returns a new iterator over all instances in storage s who’s metadata URI matches pattern.

Returns NULL on error.

typedef int (*IterNext)(void *iter, char *buf)#

Writes the uuid of the next instance to buf, where iter is an iterator returned by IterCreate().

Returns zero on success, 1 if there are no more UUIDs to iterate over and a negative number on other errors.

typedef void (*IterFree)(void *iter)#: Free’s iterator created with IterCreate().

typedef char **(*GetUUIDs)(const DLiteStorage *s)#

Returns a newly malloc’ed NULL-terminated array of (malloc’ed) string pointers to the UUID’s of all instances in storage s.

The caller is responsible to free the returned array.

Returns NULL on error.

Deprecated:: Will most likely be deprecated in favour for IterCreate(), IterNext() and IterFree().

Instance API

New API for loading, saving and deleting instances that works direct on instances themselves.

These are all optional for the plugin to implement. If LoadInstance() or SaveInstance() are not implemented, the old datamodel API will be attempted.

typedef DLiteInstance *(*LoadInstance)(const DLiteStorage *s, const char *id)#: Returns a new instance from id in storage s. NULL is returned on error.

typedef int (*SaveInstance)(DLiteStorage *s, const DLiteInstance *inst)#: Stores instance inst to storage s. Returns non-zero on error.

typedef int (*DeleteInstance)(DLiteStorage *s, const char *id)#: Delete instance uuid from storage s. Returns non-zero on error.

In-memory API

These optional functions does not operate on the storage, but rather on a pointer to a memory buffer.

typedef DLiteInstance *(*MemLoadInstance)(const DLiteStoragePlugin *api, const unsigned char *buf, size_t size, const char *id)#: Returns a new instance from memory buffer buf of size size. NULL is returned on error.

typedef int (*MemSaveInstance)(const DLiteStoragePlugin *api, unsigned char *buf, size_t size, const DLiteInstance *inst)#

Stores instance inst to memory buffer buf of size size.

Returns number of bytes written to buf (or would have been written to buf if buf is not large enough). Returns a negative error code on error.

Datamodel API

Old datamodel-based API for loading instances from and saving them to a storage.

The functions in the datamodel API may be omitted if the corresponding functions in the instance API are implemented.

Deprecated:: Will most likely be deprecated in favour for the new and simpler Instance API.

typedef DLiteDataModel *(*DataModel)(const DLiteStorage *s, const char *uuid)#

Creates a new datamodel for storage s.

If uuid exists in the root of s, the datamodel should describe the corresponding instance.

Otherwise (if s is writable), a new instance described by the datamodel should be created in s.

Returns the new datamodel or NULL on error.

typedef int (*DataModelFree)(DLiteDataModel *d)#: Frees all memory associated with datamodel d.

typedef char *(*GetMetaURI)(const DLiteDataModel *d)#: Returns a pointer to newly malloc’ed metadata uri for datamodel d or NULL on error.

typedef void (*ResolveDimensions)(DLiteDataModel *d, const DLiteMeta *meta)#: Resolve the dimensions from the properties (JSON or YAML storage)

typedef int (*GetDimensionSize)(const DLiteDataModel *d, const char *name)#: Returns the size of dimension name or -1 on error.

typedef int (*GetProperty)(const DLiteDataModel *d, const char *name, void *ptr, DLiteType type, size_t size, size_t ndims, const size_t *shape)#

Copies property name to memory pointed to by ptr.

The expected type, size, number of dimensions and size of each dimension of the memory is described by type, size, ndims and shape, respectively.

Returns non-zero on error.

Optional part of the DataModel API

Signatures of function that are optional for the plugins to define.

typedef int (*SetMetaURI)(DLiteDataModel *d, const char *uri)#: Sets the metadata uri in datamodel d to uri. Returns non-zero on error.

typedef int (*SetDimensionSize)(DLiteDataModel *d, const char *name, size_t size)#: Sets the size of dimension name. Returns non-zero on error.

typedef int (*SetProperty)(DLiteDataModel *d, const char *name, const void *ptr, DLiteType type, size_t size, size_t ndims, const size_t *shape)#

Sets property name to the memory pointed to by ptr.

The expected type, size, number of dimensions and size of each dimension of the memory is described by type, size, ndims and shape, respectively.

Returns non-zero on error.

typedef int (*HasDimension)(const DLiteDataModel *d, const char *name)#: Returns a positive value if dimension name is defined, zero if it isn’t and a negative value on error.

typedef int (*HasProperty)(const DLiteDataModel *d, const char *name)#: Returns a positive value if property name is defined, zero if it isn’t and a negative value on error.

typedef char *(*GetDataName)(const DLiteDataModel *d)#: If the uuid was generated from a unique name, return a pointer to a newly malloc’ed string with this name. Otherwise NULL is returned.

typedef int (*SetDataName)(DLiteDataModel *d, const char *name)#: Gives the instance a name. This function should only be called if the uuid was generated from name. Returns non-zero on error.

Internal data

Internal data used by the driver. Optional.

typedef void (*DriverFreer)(DLiteStoragePlugin *api)#: Releases internal resources associated with api.

Defines

DLiteStorage_HEAD#: Initial segment of all DLiteStorage plugin data structures. Number of references to this storage

DLiteDataModel_HEAD#: Initial segment of all DLiteDataModel plugin data structures. UUID for the stored data

Typedefs

typedef struct _DLiteStoragePlugin DLiteStoragePlugin#: A struct with function pointers to all functions provided by a plugin.

typedef struct _DLiteStoragePluginIter DLiteStoragePluginIter#

typedef enum _DLiteStorageFlags DLiteStorageFlags#: Storage flags

Enums

enum _DLiteStorageFlags#

Storage flags

Values:

enumerator dliteReadable#: Whether storage is readable

enumerator dliteWritable#: Whether storage is writable

enumerator dliteGeneric#: Whether storage may hold both data and metadata

enumerator dliteTransaction#: Whether storage supports transactions

struct DLiteMapInstance#

#include <dlite-storage-plugins.h>

Map to instance pointer

Public Members

map_base_t base#

DLiteInstance **ref#

DLiteInstance *tmp#

struct _DLiteStorage#: #include <dlite-storage-plugins.h>

Base definition of a DLite storage, that all plugin storage objects can be cast to. Is never actually instansiated.

struct _DLiteDataModel#: #include <dlite-storage-plugins.h>

Base definition of a DLite data model, that all plugin data model objects can be cast to. Is never actually instansiated.

struct _DLiteStoragePlugin#

#include <dlite-storage-plugins.h>

Struct with the name and pointers to function for a plugin. All plugins should define themselves by defining an intance of DLiteStoragePlugin.

Public Members

Open open#: Open storage

Close close#: Close storage

Flush flush#: Flush storage

Help help#: Returns plugin documentation

IterCreate iterCreate#: Creates iterator over storage

IterNext iterNext#: Returns next UUID

IterFree iterFree#: Free’s iterator

LoadInstance loadInstance#: Returns new instance from storage

SaveInstance saveInstance#: Stores an instance

DeleteInstance deleteInstance#: Delete an instance

MemLoadInstance memLoadInstance#: Load instance from memory

MemSaveInstance memSaveInstance#: Save instance to memory

GetUUIDs getUUIDs#: Returns all UUIDs in storage

DataModel dataModel#: Creates new data model

DataModelFree dataModelFree#: Frees a data model

GetMetaURI getMetaURI#: Returns uri to metadata

ResolveDimensions resolveDimensions#: Resolves dimensions from properties

GetDimensionSize getDimensionSize#: Returns size of dimension

GetProperty getProperty#: Gets value of property

SetMetaURI setMetaURI#: Sets metadata uri

SetDimensionSize setDimensionSize#: Sets size of dimension

SetProperty setProperty#: Sets value of property

HasDimension hasDimension#: Checks for dimension name

HasProperty hasProperty#: Checks for property name

GetDataName getDataName#: Returns name of instance

SetDataName setDataName#: Assigns name to instance

void *data#: Internal data used by the driver