Use OTEAPI-OPTIMADE with OTElib¶

OTElib is a Python client library for the OTEAPI system. It has two usable backends:

1. Python-based. This uses the OTEAPI Core library directly to exectue pipelines. This backend should mainly be used for development purposes.
2. HTTP requests-based. This uses the OTEAPI Services library to execute pipelines via HTTP requests to a running OTEAPI server. This backend should be used for production purposes.

This notebook demonstrates how to use OTEAPI-OPTIMADE with OTElib.

Example¶

Using OTElib we will do different OPTIMADE queries. First, we will search through the Materials Project database for all structures with the formula Al2O3. Then, we will search through the Materials Project database for all structures that include Al and O in their chemical formula. Finally, we will search through the Materials Cloud MC3D - Materials Cloud three-dimensional crystals database for all structures that include Al and O in their chemical formula.

HTTP requests-based backend¶

This backend is equivalent to running in production. It requires a running OTEAPI server.

Setup¶

First, we need to have a running OTEAPI server. If one is not available, we can start one using Docker. See the overview for instructions on how to start a server.

Further documentation about the OTEAPI Service is available as a README on GitHub.

Note

It is advisable to run the OTEAPI server in a separate terminal window, so that we can see the logs from the server. Furthermore, we can stop the server by pressing Ctrl+C in the terminal window.

After following the instructions, we should have a running OTEAPI server at localhost:80.

Create a client¶

In [1]:

from otelib import OTEClient

client = OTEClient("http://localhost:80")

from otelib import OTEClient client = OTEClient("http://localhost:80")

Search through Materials Project for all structures with the formula Al2O3¶

To search through Materials Project for all structures with the formula Al2O3, we need to create the OTE strategies that we want to use for creating the OTE pipeline:

To create the strategies, we need to know how to configure them. This information is available in the OTEAPI Core documentation, specifically for Data Resource strategies and for Filter strategies.

For the data resource strategy, i.e., the OPTIMADE DB strategy, we need to know the base URL of the OPTIMADE API for Materials Project. OPTIMADE has a useful providers dashboard that lists all known OPTIMADE providers and their (sub-)databases. Here we can find the base URL for the Materials Project: https://optimade.materialsproject.org.

In [2]:

data_resource_strategy = client.create_dataresource(
    accessService="OPTIMADE",
    accessUrl="https://optimade.materialsproject.org/",
)

data_resource_strategy = client.create_dataresource( accessService="OPTIMADE", accessUrl="https://optimade.materialsproject.org/", )

For the filter strategy, we need to know the OPTIMADE query that we want to execute.

For retrieving all structures with the formula Al2O3, we can use the following OPTIMADE filter query:

chemical_formula_descriptive = "Al2O3" OR chemical_formula_reduced = "Al2O3" OR chemical_formula_hill = "Al2O3"

In [3]:

filter_strategy = client.create_filter(
    filterType="OPTIMADE",
    query='chemical_formula_descriptive = "Al2O3" OR chemical_formula_reduced = "Al2O3" OR chemical_formula_hill = "Al2O3"',
)

filter_strategy = client.create_filter( filterType="OPTIMADE", query='chemical_formula_descriptive = "Al2O3" OR chemical_formula_reduced = "Al2O3" OR chemical_formula_hill = "Al2O3"', )

Now we can create the OTE pipeline shown above and execute it.

In [4]:

pipeline = filter_strategy >> data_resource_strategy
session = pipeline.get()

pipeline = filter_strategy >> data_resource_strategy session = pipeline.get()

The returned session is a JSON object we can parse and investigate.

In [5]:

import json

parsed_session: dict = json.loads(session)
print(parsed_session.keys())

import json parsed_session: dict = json.loads(session) print(parsed_session.keys())

dict_keys(['optimade_resources', 'optimade_config', 'optimade_resource_model'])

As can be seen, there are three keys in the returned session. optimade_config summarizes the query that has been performed to Materials Project. The OPTIMADE structures are listed under the optimade_resources. It is named as such due to there being different OPTIMADE resources, e.g., structures, references, links, etc. The OPTIMADE Python tools has a useful OPTIMADE Structure model class that can be used to parse the OPTIMADE structures into Python objects as well as validating them according to the OPTIMADE specification. Again, since one can query for different OPTIMADE resources, the specific Python class to use is given in optimade_resource_model.

In [6]:

from importlib import import_module

import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1)
ResourceClass = getattr(import_module(import_path), class_name)

parsed_structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]]
print(f"The query resulted in {len(parsed_structures)} structures found (on page 1) of the returned data.")
print(f"Their Materials Project IDs are: {[structure.id for structure in parsed_structures]}")

from importlib import import_module import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1) ResourceClass = getattr(import_module(import_path), class_name) parsed_structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]] print(f"The query resulted in {len(parsed_structures)} structures found (on page 1) of the returned data.") print(f"Their Materials Project IDs are: {[structure.id for structure in parsed_structures]}")

The query resulted in 20 structures found (on page 1) of the returned data.
Their Materials Project IDs are: ['mp-1228448', 'mp-755483', 'mp-1245081', 'mp-1244878', 'mp-1105018', 'mp-754401', 'mp-1245063', 'mp-1245265', 'mp-684591', 'mp-1244898', 'mp-1245211', 'mp-1245056', 'mp-642363', 'mp-1244930', 'mp-1244937', 'mp-1244967', 'mp-1244954', 'mp-1244874', 'mp-1245008', 'mp-1245023']

To find them on the Materials Project website, go to materialsproject.org/materials/<ID>, for example: materialsproject.org/materials/mp-1228448.

What is more, we can investigate the structure according to the well-defined OPTIMADE structure model attributes. For example, so assert the chemical formula is what we expected, we can check the different chemical formula attributes:

In [7]:

structure = parsed_structures[0]
print(structure.id)
for attribute in ("descriptive", "reduced", "hill", "anonymous"):
    print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

structure = parsed_structures[0] print(structure.id) for attribute in ("descriptive", "reduced", "hill", "anonymous"): print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

mp-1228448
chemical_formula_descriptive: Al2O3
chemical_formula_reduced: Al2O3
chemical_formula_hill: Al2O3
chemical_formula_anonymous: A3B2

Search through Materials Project for all structures that include Al and O in their chemical formula¶

We must use a different OPTIMADE filter query for this search:

elements HAS ALL "Al","O"

elements is a structure attribute that lists all elements in the structure. The HAS ALL operator matches if, for each value, there is at least one element in elements equal to that value. (This implements the set operator >=.)

To do this search, we can reuse the OTE pipeline from the previous search, but change the filter strategy.

In [8]:

filter_strategy = client.create_filter(
    filterType="OPTIMADE",
    query='elements HAS ALL "Al","O"',
    limit=5,
)

filter_strategy = client.create_filter( filterType="OPTIMADE", query='elements HAS ALL "Al","O"', limit=5, )

For this query, we have added the limit parameter to the filter configuration, which will pass a page_limit query parameter to the OPTIMADE API ensuring that we only retrieve the first 5 structures (it limits each result's page to 5 resources).

Let us investigate the result again, checking the list of elements and the chemical formula attributes:

In [9]:

pipeline = filter_strategy >> data_resource_strategy
parsed_session = json.loads(pipeline.get())
print(f"The query resulted in {len(parsed_session['optimade_resources'])} structures found (on page 1) of the returned data.")

import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1)
ResourceClass = getattr(import_module(import_path), class_name)
structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]]
print(f"Their Materials Project IDs are: {[structure.id for structure in structures]}")

pipeline = filter_strategy >> data_resource_strategy parsed_session = json.loads(pipeline.get()) print(f"The query resulted in {len(parsed_session['optimade_resources'])} structures found (on page 1) of the returned data.") import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1) ResourceClass = getattr(import_module(import_path), class_name) structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]] print(f"Their Materials Project IDs are: {[structure.id for structure in structures]}")

The query resulted in 5 structures found (on page 1) of the returned data.
Their Materials Project IDs are: ['mp-1038042', 'mp-1182891', 'mp-1208627', 'mp-1247835', 'mp-1521059']

Let us also check the query parameters used for the request to the OPTIMADE API to ensure that the page_limit query parameter was passed:

In [10]:

parsed_session["optimade_config"]

parsed_session["optimade_config"]

Out[10]:

{'query_parameters': {'filter': 'elements HAS ALL "Al","O"', 'page_limit': 5}}

In [11]:

structure = structures[0]
print(structure.id)
print(f"elements: {structure.elements}")
for attribute in ("descriptive", "reduced", "hill", "anonymous"):
    print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

structure = structures[0] print(structure.id) print(f"elements: {structure.elements}") for attribute in ("descriptive", "reduced", "hill", "anonymous"): print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

mp-1038042
elements: ['Al', 'Cr', 'Mg', 'O']
chemical_formula_descriptive: AlCrMg30O32
chemical_formula_reduced: AlCrMg30O32
chemical_formula_hill: AlCrMg30O32
chemical_formula_anonymous: A32B30CD

Search through MC3D database in Materials Cloud for all structures that include Al and O in their chemical formula¶

Finally, let us reuse the same filter strategy we used for the previous search, but change the data resource strategy to point to the MC3D database in Materials Cloud.

The base URL for the MC3D database is https://aiida.materialscloud.org/mc3d/optimade as found in the providers dashboard.

In [12]:

data_resource_strategy = client.create_dataresource(
    accessService="OPTIMADE",
    accessUrl="https://aiida.materialscloud.org/mc3d/optimade",
)

data_resource_strategy = client.create_dataresource( accessService="OPTIMADE", accessUrl="https://aiida.materialscloud.org/mc3d/optimade", )

In [13]:

pipeline = filter_strategy >> data_resource_strategy
parsed_session = json.loads(pipeline.get())
print(f"The query resulted in {len(parsed_session['optimade_resources'])} structures found (on page 1) of the returned data.")

import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1)
ResourceClass = getattr(import_module(import_path), class_name)
structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]]
print(f"Their Materials Cloud (AiiDA) IDs are: {[structure.id for structure in structures]}")

pipeline = filter_strategy >> data_resource_strategy parsed_session = json.loads(pipeline.get()) print(f"The query resulted in {len(parsed_session['optimade_resources'])} structures found (on page 1) of the returned data.") import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1) ResourceClass = getattr(import_module(import_path), class_name) structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]] print(f"Their Materials Cloud (AiiDA) IDs are: {[structure.id for structure in structures]}")

The query resulted in 5 structures found (on page 1) of the returned data.
Their Materials Cloud (AiiDA) IDs are: ['13952', '43703', '43800', '56101', '56270']

Again, let's check the query parameters used for the request to the OPTIMADE API to ensure it is equivalent to the previous search:

In [14]:

parsed_session["optimade_config"]

parsed_session["optimade_config"]

Out[14]:

{'query_parameters': {'filter': 'elements HAS ALL "Al","O"', 'page_limit': 5}}

In [15]:

structure = structures[0]
print(structure.id)
print(f"elements: {structure.elements}")
for attribute in ("descriptive", "reduced", "hill", "anonymous"):
    print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

structure = structures[0] print(structure.id) print(f"elements: {structure.elements}") for attribute in ("descriptive", "reduced", "hill", "anonymous"): print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

13952
elements: ['Al', 'O']
chemical_formula_descriptive: Al2O6
chemical_formula_reduced: AlO3
chemical_formula_hill: Al2O6
chemical_formula_anonymous: A3B

The MC3D structures can, unfortunately, not be found easily in the Materials Cloud website, as the ID in the DISCOVER section does not match the ID in the OPTIMADE structure. However, the full OPTIMADE structure can be found at <OPTIMADE_BASE_URL>/structures/<ID>, for example: aiida.materialscloud.org/mc3d/optimade/structures/13952.

Note

The structure with the OPTIMADE ID 13952 is found with the Materials Cloud DISCOVER ID mc3d-76896 and can be found here.

Python-based backend¶

With this backend, we do not need a running OTEAPI server.

Create a client¶

In [16]:

client = OTEClient("python")

client = OTEClient("python")

Now we can go through the same searches as we did with the HTTP requests-based backend. The result should not change.

Search through Materials Project for all structures with the formula Al2O3¶

In [17]:

data_resource_strategy = client.create_dataresource(
    accessService="OPTIMADE",
    accessUrl="https://optimade.materialsproject.org/",
)

filter_strategy = client.create_filter(
    filterType="OPTIMADE",
    query='chemical_formula_descriptive = "Al2O3" OR chemical_formula_reduced = "Al2O3" OR chemical_formula_hill = "Al2O3"',
)

pipeline = filter_strategy >> data_resource_strategy
session = pipeline.get()

data_resource_strategy = client.create_dataresource( accessService="OPTIMADE", accessUrl="https://optimade.materialsproject.org/", ) filter_strategy = client.create_filter( filterType="OPTIMADE", query='chemical_formula_descriptive = "Al2O3" OR chemical_formula_reduced = "Al2O3" OR chemical_formula_hill = "Al2O3"', ) pipeline = filter_strategy >> data_resource_strategy session = pipeline.get()

Setting filter from query.
Setting filter from query.
resource_config: OPTIMADEResourceConfig(user=None, password=None, token=None, client_id=None, client_secret=None, configuration=OPTIMADEConfig(version='v1', endpoint='structures', query_parameters=OPTIMADEQueryParameters(filter='chemical_formula_descriptive = "Al2O3" OR chemical_formula_reduced = "Al2O3" OR chemical_formula_hill = "Al2O3"', response_format='json', email_address='', response_fields='', sort='', page_limit=20, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint=''), datacache_config=DataCacheConfig(cacheDir=PosixPath('oteapi'), accessKey=None, hashType='md5', expireTime=86400, tag='optimade'), return_object=False, use_dlite=False), description='Resource Strategy Data Configuration.\n\n    Important:\n        Either of the pairs of attributes `downloadUrl`/`mediaType` or\n        `accessUrl`/`accessService` MUST be specified.\n\n    ', downloadUrl=None, mediaType=None, accessUrl=OPTIMADEUrl('https://optimade.materialsproject.org/', base_url='https://optimade.materialsproject.org', scheme='https', tld='org', host_type='domain'), accessService='OPTIMADE', license=None, accessRights=None, publisher=None)
resource_config: OPTIMADEResourceConfig(user=None, password=None, token=None, client_id=None, client_secret=None, configuration=OPTIMADEConfig(version='v1', endpoint='structures', query_parameters=OPTIMADEQueryParameters(filter='chemical_formula_descriptive = "Al2O3" OR chemical_formula_reduced = "Al2O3" OR chemical_formula_hill = "Al2O3"', response_format='json', email_address='', response_fields='', sort='', page_limit=20, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint=''), datacache_config=DataCacheConfig(cacheDir=PosixPath('oteapi'), accessKey=None, hashType='md5', expireTime=86400, tag='optimade'), return_object=False, use_dlite=False), description='Resource Strategy Data Configuration.\n\n    Important:\n        Either of the pairs of attributes `downloadUrl`/`mediaType` or\n        `accessUrl`/`accessService` MUST be specified.\n\n    ', downloadUrl=None, mediaType=None, accessUrl=OPTIMADEUrl('https://optimade.materialsproject.org/', base_url='https://optimade.materialsproject.org', scheme='https', tld='org', host_type='domain'), accessService='OPTIMADE', license=None, accessRights=None, publisher=None)
optimade_query after update: OPTIMADEQueryParameters(filter='chemical_formula_descriptive = "Al2O3" OR chemical_formula_reduced = "Al2O3" OR chemical_formula_hill = "Al2O3"', response_format='json', email_address='', response_fields='', sort='', page_limit=20, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint='')
optimade_query after update: OPTIMADEQueryParameters(filter='chemical_formula_descriptive = "Al2O3" OR chemical_formula_reduced = "Al2O3" OR chemical_formula_hill = "Al2O3"', response_format='json', email_address='', response_fields='', sort='', page_limit=20, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint='')
OPTIMADE URL to be requested: https://optimade.materialsproject.org/v1/structures?filter=chemical_formula_descriptive%20%3D%20%22Al2O3%22%20OR%20chemical_formula_reduced%20%3D%20%22Al2O3%22%20OR%20chemical_formula_hill%20%3D%20%22Al2O3%22&response_format=json&page_limit=20&include=references
OPTIMADE URL to be requested: https://optimade.materialsproject.org/v1/structures?filter=chemical_formula_descriptive%20%3D%20%22Al2O3%22%20OR%20chemical_formula_reduced%20%3D%20%22Al2O3%22%20OR%20chemical_formula_hill%20%3D%20%22Al2O3%22&response_format=json&page_limit=20&include=references

/home/cwa/.venvs/oteapi-optimade/lib/python3.9/site-packages/optimade/server/config.py:113: UserWarning: Unable to find config file at /home/cwa/.optimade.json, using the default settings instead.
  warnings.warn(

As one can see, this backend runs locally within the same Python environment as the notebook. This is useful for development purposes, where the logging messages are shown directly in the output.

In [18]:

parsed_session: dict = json.loads(session)
print(parsed_session.keys())

parsed_session: dict = json.loads(session) print(parsed_session.keys())

dict_keys(['optimade_config', 'optimade_resources', 'optimade_resource_model'])

We get the same keys in the returned session as we did with the HTTP requests-based backend. If we again import the OPTIMADE Structure model class from the OPTIMADE Python tools, we can parse the OPTIMADE structures into Python objects as well as validating them according to the OPTIMADE specification, just as we did with the HTTP requests-based backend.

In [19]:

import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1)
ResourceClass = getattr(import_module(import_path), class_name)

parsed_structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]]
print(f"The query resulted in {len(parsed_structures)} structures found (on page 1) of the returned data.")
print(f"Their Materials Project IDs are: {[structure.id for structure in parsed_structures]}")

structure = parsed_structures[0]
print(structure.id)
for attribute in ("descriptive", "reduced", "hill", "anonymous"):
    print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1) ResourceClass = getattr(import_module(import_path), class_name) parsed_structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]] print(f"The query resulted in {len(parsed_structures)} structures found (on page 1) of the returned data.") print(f"Their Materials Project IDs are: {[structure.id for structure in parsed_structures]}") structure = parsed_structures[0] print(structure.id) for attribute in ("descriptive", "reduced", "hill", "anonymous"): print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

The query resulted in 20 structures found (on page 1) of the returned data.
Their Materials Project IDs are: ['mp-1228448', 'mp-755483', 'mp-1245081', 'mp-1244878', 'mp-1105018', 'mp-754401', 'mp-1245063', 'mp-1245265', 'mp-684591', 'mp-1244898', 'mp-1245211', 'mp-1245056', 'mp-642363', 'mp-1244930', 'mp-1244937', 'mp-1244967', 'mp-1244954', 'mp-1244874', 'mp-1245008', 'mp-1245023']
mp-1228448
chemical_formula_descriptive: Al2O3
chemical_formula_reduced: Al2O3
chemical_formula_hill: Al2O3
chemical_formula_anonymous: A3B2

Search through Materials Project for all structures that include Al and O in their chemical formula¶

In [20]:

filter_strategy = client.create_filter(
    filterType="OPTIMADE",
    query='elements HAS ALL "Al","O"',
    limit=5,
)

pipeline = filter_strategy >> data_resource_strategy
parsed_session = json.loads(pipeline.get())
print(f"The query resulted in {len(parsed_session['optimade_resources'])} structures found (on page 1) of the returned data.")

import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1)
ResourceClass = getattr(import_module(import_path), class_name)
structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]]
print(f"Their Materials Project IDs are: {[structure.id for structure in structures]}")

filter_strategy = client.create_filter( filterType="OPTIMADE", query='elements HAS ALL "Al","O"', limit=5, ) pipeline = filter_strategy >> data_resource_strategy parsed_session = json.loads(pipeline.get()) print(f"The query resulted in {len(parsed_session['optimade_resources'])} structures found (on page 1) of the returned data.") import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1) ResourceClass = getattr(import_module(import_path), class_name) structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]] print(f"Their Materials Project IDs are: {[structure.id for structure in structures]}")

Setting filter from query.
Setting filter from query.
Setting filter from query.
Setting page_limit from limit.
Setting page_limit from limit.
Setting page_limit from limit.
resource_config: OPTIMADEResourceConfig(user=None, password=None, token=None, client_id=None, client_secret=None, configuration=OPTIMADEConfig(version='v1', endpoint='structures', query_parameters=OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint=''), datacache_config=DataCacheConfig(cacheDir=PosixPath('oteapi'), accessKey=None, hashType='md5', expireTime=86400, tag='optimade'), return_object=False, use_dlite=False), description='Resource Strategy Data Configuration.\n\n    Important:\n        Either of the pairs of attributes `downloadUrl`/`mediaType` or\n        `accessUrl`/`accessService` MUST be specified.\n\n    ', downloadUrl=None, mediaType=None, accessUrl=OPTIMADEUrl('https://optimade.materialsproject.org/', base_url='https://optimade.materialsproject.org', scheme='https', tld='org', host_type='domain'), accessService='OPTIMADE', license=None, accessRights=None, publisher=None)
resource_config: OPTIMADEResourceConfig(user=None, password=None, token=None, client_id=None, client_secret=None, configuration=OPTIMADEConfig(version='v1', endpoint='structures', query_parameters=OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint=''), datacache_config=DataCacheConfig(cacheDir=PosixPath('oteapi'), accessKey=None, hashType='md5', expireTime=86400, tag='optimade'), return_object=False, use_dlite=False), description='Resource Strategy Data Configuration.\n\n    Important:\n        Either of the pairs of attributes `downloadUrl`/`mediaType` or\n        `accessUrl`/`accessService` MUST be specified.\n\n    ', downloadUrl=None, mediaType=None, accessUrl=OPTIMADEUrl('https://optimade.materialsproject.org/', base_url='https://optimade.materialsproject.org', scheme='https', tld='org', host_type='domain'), accessService='OPTIMADE', license=None, accessRights=None, publisher=None)
resource_config: OPTIMADEResourceConfig(user=None, password=None, token=None, client_id=None, client_secret=None, configuration=OPTIMADEConfig(version='v1', endpoint='structures', query_parameters=OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint=''), datacache_config=DataCacheConfig(cacheDir=PosixPath('oteapi'), accessKey=None, hashType='md5', expireTime=86400, tag='optimade'), return_object=False, use_dlite=False), description='Resource Strategy Data Configuration.\n\n    Important:\n        Either of the pairs of attributes `downloadUrl`/`mediaType` or\n        `accessUrl`/`accessService` MUST be specified.\n\n    ', downloadUrl=None, mediaType=None, accessUrl=OPTIMADEUrl('https://optimade.materialsproject.org/', base_url='https://optimade.materialsproject.org', scheme='https', tld='org', host_type='domain'), accessService='OPTIMADE', license=None, accessRights=None, publisher=None)
optimade_query after update: OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint='')
optimade_query after update: OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint='')
optimade_query after update: OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint='')
OPTIMADE URL to be requested: https://optimade.materialsproject.org/v1/structures?filter=elements%20HAS%20ALL%20%22Al%22%2C%22O%22&response_format=json&page_limit=5&include=references
OPTIMADE URL to be requested: https://optimade.materialsproject.org/v1/structures?filter=elements%20HAS%20ALL%20%22Al%22%2C%22O%22&response_format=json&page_limit=5&include=references
OPTIMADE URL to be requested: https://optimade.materialsproject.org/v1/structures?filter=elements%20HAS%20ALL%20%22Al%22%2C%22O%22&response_format=json&page_limit=5&include=references
The query resulted in 5 structures found (on page 1) of the returned data.
Their Materials Project IDs are: ['mp-1038042', 'mp-1182891', 'mp-1208627', 'mp-1247835', 'mp-1521059']

The configuration in optimade_config is quite more extensive than with the HTTP requests-based backend, as it includes more information, automatically setting the default values so they are shown in the config:

In [21]:

parsed_session["optimade_config"]

parsed_session["optimade_config"]

Out[21]:

{'version': 'v1',
 'endpoint': 'structures',
 'query_parameters': {'filter': 'elements HAS ALL "Al","O"', 'page_limit': 5},
 'datacache_config': {'cacheDir': 'oteapi',
  'accessKey': None,
  'hashType': 'md5',
  'expireTime': 86400,
  'tag': 'optimade'},
 'return_object': False,
 'use_dlite': False}

Let's look at the first structure again:

In [22]:

structure = structures[0]
print(structure.id)
print(f"elements: {structure.elements}")
for attribute in ("descriptive", "reduced", "hill", "anonymous"):
    print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

structure = structures[0] print(structure.id) print(f"elements: {structure.elements}") for attribute in ("descriptive", "reduced", "hill", "anonymous"): print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

mp-1038042
elements: ['Al', 'Cr', 'Mg', 'O']
chemical_formula_descriptive: AlCrMg30O32
chemical_formula_reduced: AlCrMg30O32
chemical_formula_hill: AlCrMg30O32
chemical_formula_anonymous: A32B30CD

Search through MC3D database in Materials Cloud for all structures that include Al and O in their chemical formula¶

In [23]:

data_resource_strategy = client.create_dataresource(
    accessService="OPTIMADE",
    accessUrl="https://aiida.materialscloud.org/mc3d/optimade",
)

pipeline = filter_strategy >> data_resource_strategy
parsed_session = json.loads(pipeline.get())
print(f"The query resulted in {len(parsed_session['optimade_resources'])} structures found (on page 1) of the returned data.")

import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1)
ResourceClass = getattr(import_module(import_path), class_name)
structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]]
print(f"Their Materials Cloud (AiiDA) IDs are: {[structure.id for structure in structures]}")

print(parsed_session["optimade_config"])

structure = structures[0]
print(structure.id)
print(f"elements: {structure.elements}")
for attribute in ("descriptive", "reduced", "hill", "anonymous"):
    print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

data_resource_strategy = client.create_dataresource( accessService="OPTIMADE", accessUrl="https://aiida.materialscloud.org/mc3d/optimade", ) pipeline = filter_strategy >> data_resource_strategy parsed_session = json.loads(pipeline.get()) print(f"The query resulted in {len(parsed_session['optimade_resources'])} structures found (on page 1) of the returned data.") import_path, class_name = parsed_session["optimade_resource_model"].split(":", maxsplit=1) ResourceClass = getattr(import_module(import_path), class_name) structures = [ResourceClass(structure) for structure in parsed_session["optimade_resources"]] print(f"Their Materials Cloud (AiiDA) IDs are: {[structure.id for structure in structures]}") print(parsed_session["optimade_config"]) structure = structures[0] print(structure.id) print(f"elements: {structure.elements}") for attribute in ("descriptive", "reduced", "hill", "anonymous"): print(f"chemical_formula_{attribute}: {getattr(structure, f'chemical_formula_{attribute}', '(not defined)')}")

Setting filter from query.
Setting filter from query.
Setting filter from query.
Setting page_limit from limit.
Setting page_limit from limit.
Setting page_limit from limit.
resource_config: OPTIMADEResourceConfig(user=None, password=None, token=None, client_id=None, client_secret=None, configuration=OPTIMADEConfig(version='v1', endpoint='structures', query_parameters=OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint=''), datacache_config=DataCacheConfig(cacheDir=PosixPath('oteapi'), accessKey=None, hashType='md5', expireTime=86400, tag='optimade'), return_object=False, use_dlite=False), description='Resource Strategy Data Configuration.\n\n    Important:\n        Either of the pairs of attributes `downloadUrl`/`mediaType` or\n        `accessUrl`/`accessService` MUST be specified.\n\n    ', downloadUrl=None, mediaType=None, accessUrl=OPTIMADEUrl('https://aiida.materialscloud.org/mc3d/optimade', base_url='https://aiida.materialscloud.org/mc3d/optimade', scheme='https', tld='org', host_type='domain'), accessService='OPTIMADE', license=None, accessRights=None, publisher=None)
resource_config: OPTIMADEResourceConfig(user=None, password=None, token=None, client_id=None, client_secret=None, configuration=OPTIMADEConfig(version='v1', endpoint='structures', query_parameters=OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint=''), datacache_config=DataCacheConfig(cacheDir=PosixPath('oteapi'), accessKey=None, hashType='md5', expireTime=86400, tag='optimade'), return_object=False, use_dlite=False), description='Resource Strategy Data Configuration.\n\n    Important:\n        Either of the pairs of attributes `downloadUrl`/`mediaType` or\n        `accessUrl`/`accessService` MUST be specified.\n\n    ', downloadUrl=None, mediaType=None, accessUrl=OPTIMADEUrl('https://aiida.materialscloud.org/mc3d/optimade', base_url='https://aiida.materialscloud.org/mc3d/optimade', scheme='https', tld='org', host_type='domain'), accessService='OPTIMADE', license=None, accessRights=None, publisher=None)
resource_config: OPTIMADEResourceConfig(user=None, password=None, token=None, client_id=None, client_secret=None, configuration=OPTIMADEConfig(version='v1', endpoint='structures', query_parameters=OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint=''), datacache_config=DataCacheConfig(cacheDir=PosixPath('oteapi'), accessKey=None, hashType='md5', expireTime=86400, tag='optimade'), return_object=False, use_dlite=False), description='Resource Strategy Data Configuration.\n\n    Important:\n        Either of the pairs of attributes `downloadUrl`/`mediaType` or\n        `accessUrl`/`accessService` MUST be specified.\n\n    ', downloadUrl=None, mediaType=None, accessUrl=OPTIMADEUrl('https://aiida.materialscloud.org/mc3d/optimade', base_url='https://aiida.materialscloud.org/mc3d/optimade', scheme='https', tld='org', host_type='domain'), accessService='OPTIMADE', license=None, accessRights=None, publisher=None)
optimade_query after update: OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint='')
optimade_query after update: OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint='')
optimade_query after update: OPTIMADEQueryParameters(filter='elements HAS ALL "Al","O"', response_format='json', email_address='', response_fields='', sort='', page_limit=5, page_offset=0, page_number=None, page_cursor=0, page_above=None, page_below=None, include='references', api_hint='')
OPTIMADE URL to be requested: https://aiida.materialscloud.org/mc3d/optimade/v1/structures?filter=elements%20HAS%20ALL%20%22Al%22%2C%22O%22&response_format=json&page_limit=5&include=references
OPTIMADE URL to be requested: https://aiida.materialscloud.org/mc3d/optimade/v1/structures?filter=elements%20HAS%20ALL%20%22Al%22%2C%22O%22&response_format=json&page_limit=5&include=references
OPTIMADE URL to be requested: https://aiida.materialscloud.org/mc3d/optimade/v1/structures?filter=elements%20HAS%20ALL%20%22Al%22%2C%22O%22&response_format=json&page_limit=5&include=references
The query resulted in 5 structures found (on page 1) of the returned data.
Their Materials Cloud (AiiDA) IDs are: ['13952', '43703', '43800', '56101', '56270']
{'version': 'v1', 'endpoint': 'structures', 'query_parameters': {'filter': 'elements HAS ALL "Al","O"', 'page_limit': 5}, 'datacache_config': {'cacheDir': 'oteapi', 'accessKey': None, 'hashType': 'md5', 'expireTime': 86400, 'tag': 'optimade'}, 'return_object': False, 'use_dlite': False}
13952
elements: ['Al', 'O']
chemical_formula_descriptive: Al2O6
chemical_formula_reduced: AlO3
chemical_formula_hill: Al2O6
chemical_formula_anonymous: A3B