DMF Resources

Data items stored in the DMF, whether they be property data, property models, flowsheets, or other user-defined objects, are called “resources”. They are represented in the Python API by a class, Resource, with attributes for metadata and a generic “data” section where resource-type-specific information can be held. You store Resources in the DMF with the add method on an instance of the DMF class. See the DMF API Examples documentation for all the available operations.

Resource classes documented on this page: Resource, PropertyDataResource, FlowsheetResource.

A “special” type of resource is an “experiment”. The idea of experiments is that they anchor a set of resources that were the result of one coherent set of activities. Experiments are initialized with their associated DMF instance and have some convenience methods to make them a handy way to track a set of resources. See the experiment documentation page <experiment> for more details.

Here is a diagram of the resource structure:

../_images/resource-structure.png

Resources are populated by simply setting their attributes. The attributes are actually traits, using the Traitlets library, to make them “smarter” and easier to use. Some of the attributes, such as creator or sources, are lists of containers with more attributes. All the attributes are documented in the API Reference section.

Example

Below is an example of creating and then setting all (well, most) of the attributes for a resource that is a plot of the standard ‘iris’ dataset.

##############################################################################
# Institute for the Design of Advanced Energy Systems Process Systems
# Engineering Framework (IDAES PSE Framework) Copyright (c) 2018, by the
# software owners: The Regents of the University of California, through
# Lawrence Berkeley National Laboratory,  National Technology & Engineering
# Solutions of Sandia, LLC, Carnegie Mellon University, West Virginia
# University Research Corporation, et al. All rights reserved.
# 
# Please see the files COPYRIGHT.txt and LICENSE.txt for full copyright and
# license information, respectively. Both files are also available online
# at the URL "https://github.com/IDAES/idaes".
##############################################################################
from idaes.dmf import resource_old

# Create an empty resource
rsrc = resource_old.Resource(type='plot')

# Populate the resource. All the attributes are optional.
# This is set automatically to current time
rsrc.created = '2017-10-31'
# This is set automatically to current time
rsrc.modified = '2017-11-23'
# Description of the resource
rsrc.desc = 'Plots of the Iris dataset'
# Version of the resource
rsrc.version = resource_old.Version(revision='1.0.3-a7',
                                    name='november-release')
# Name and contact email of creator of the resource
rsrc.creator = resource_old.Contact(name='Dan Gunter',
                                    email='dang@science-lab.gov')
# Name and contact email of other people involved
rsrc.collaborators = [resource_old.Contact(name='John Eslick',
                                           email='john@science-lab.gov'),
                      resource_old.Contact(name='David Miller',
                                           email='david@science-lab.gov')]
# Provenance: sources, such as publications -- but
# really anything is allowed -- for the resource.
rsrc.sources = [resource_old.Source(
             source='R. A. Fisher. "The use of multiple measurements '
                    'in taxonomic problems". '
                    'Annals of Eugenics. 7 (2): 179-188.',
             doi='10.1111/j.1469-1809.1936.tb02137.x', date='1936'),
           resource_old.Source(source='Edgar Anderson. '
                                  '"The species problem in Iris". '
                                  'Annals of the Missouri Botanical Garden. '
                                  '23 (3): 457-509. '
                                  'JSTOR 2394164',
                               date='1936')]
# Associated code files, including Jupyter notebooks.
# If these codes are worth making into their own Resources,
# you should not put them here, but instead link to them
# through the `relations` attribute.
rsrc.codes = [resource_old.Code(type='notebook',
                                desc='Python plotting code',
                                name='plot_iris_dataset.ipynb',
                                language='Python',
                                location='http://scikit-learn.org/stable/_downloads/'
                                'plot_iris_dataset.ipynb')]
# Associated data files. Same comment as for `codes` re: linking.
rsrc.datafiles = [resource_old.FilePath(path='iris-data.csv',
                                        desc='Iris data')]
# Short names that make it easier to find this resource.
rsrc.aliases = ['iris', 'iris-plots']
# Tags that make it easier to find this resource later.
rsrc.tags = ['iris', 'anderson', 'fisher']
# Arbitrary additional data to add into the resource.
# Note that larger data units should be made into files and
# added into the `datafiles` attributes.
rsrc.data = {
    'features': ['sepal length (cm)', 'sepal width (cm)',
                  'petal length (cm)', 'petal width (cm)'],
    # the data is already defined in the datafile,
    # but we could also put it here, inline
    'array': [[5.1, 3.5, 1.4, 0.2], [4.9, 3.0, 1.4, 0.2],
              [4.7, 3.2, 1.3, 0.2], [4.6, 3.1, 1.5, 0.2],
              [5.0, 3.6, 1.4, 0.2], [5.4, 3.9, 1.7, 0.4],
              # ... you get the idea..
             ]
}

API Reference

This section shows the documentation from the source code for the Resource and ResourceType classes, as well as Python classes that inherit from Resource in order to add custom functionality: PropertyDataResource and FlowsheetResource.

class idaes.dmf.resource.Resource(value=None, type_=None)[source]

Core object for the Data Management Framework.

ID_FIELD = 'id_'

Identifier field name constant

TYPE_FIELD = 'type'

Resource type field name constant

id

Get resource identifier.

type

Get resource type.

data

Get JSON data for this resource.

get_datafiles(mode='r')[source]

Generate readable file objects for ‘datafiles’ in resource.

Parameters:mode (str) – Mode for open()
Returns:Generates `file`s.
Return type:generator

Resource schema

Below are HTML and raw JSON versions of the resource “schema”. This describes the structure of a resource. Having a schema is useful for other programs to be able to independently manipulate the resource representation.

Resource Schema (HTML)

schema
  • aliases
    List of:
    string
  • codes
    List of:
    • desc string
    • name string
    • language string
    • idhash string
    • location string
    • version
      List of:
      0 integer
      1 integer
      2 integer
      3 string
  • collaborators
    List of:
    • email string
    • name string
  • created number
  • creator
    • email string
    • name string
  • data
  • datafiles
    List of:
    • desc string
    • metadata
    • mimetype string
    • path string
    • subdir string
  • datafiles_dir string
  • desc string
  • id_ integer
  • modified number
  • relations
    List of:
    • predicate string
      Possible values:
      • WasGeneratedBy
      • Used
      • WasDerivedFrom
      • WasTriggeredBy
      • WasControlledBy
      • WasRevisionOf
    • identifier string
    • role string
      Possible values:
      • subject
      • object
  • sources
    List of:
    • date number
    • doi string
    • isbn string
    • language string
    • source string
  • tags
    List of:
    string
  • version
    • created number
    • name string
    • revision
      List of:
      0 integer
      1 integer
      2 integer
      3 string

Resouce Schema (JSON)

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "http://idaes.org",
  "definitions": {
    "SemanticVersion": {
      "type": "array",
      "items": [
        {
          "type": "integer"
        },
        {
          "type": "integer"
        },
        {
          "type": "integer"
        },
        {
          "type": "string"
        }
      ],
      "minItems": 4
    }
  },
  "type": "object",
  "properties": {
    "aliases": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "codes": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "type": {
            "type": "string",
            "enum": [
              "method",
              "function",
              "module",
              "class",
              "file",
              "package",
              "repository",
              "notebook"
            ]
          },
          "desc": {
            "type": "string"
          },
          "name": {
            "type": "string"
          },
          "language": {
            "type": "string"
          },
          "idhash": {
            "type": "string"
          },
          "location": {
            "type": "string"
          },
          "version": {
            "$ref": "#/definitions/SemanticVersion"
          }
        }
      }
    },
    "collaborators": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "email": {
            "type": "string",
            "format": "email"
          },
          "name": {
            "type": "string"
          }
        },
        "required": [
          "name"
        ]
      }
    },
    "created": {
      "type": "number"
    },
    "creator": {
      "type": "object",
      "properties": {
        "email": {
          "type": "string",
          "format": "email"
        },
        "name": {
          "type": "string"
        }
      },
      "required": [
        "name"
      ]
    },
    "data": {
      "type": "object"
    },
    "datafiles": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "desc": {
            "type": "string"
          },
          "metadata": {
            "type": "object"
          },
          "mimetype": {
            "type": "string"
          },
          "path": {
            "type": "string"
          },
          "subdir": {
            "type": "string"
          }
        },
        "required": [
          "desc",
          "metadata",
          "mimetype",
          "path",
          "subdir"
        ]
      }
    },
    "datafiles_dir": {
      "type": "string"
    },
    "desc": {
      "type": "string"
    },
    "id_": {
      "type": "integer"
    },
    "modified": {
      "type": "number"
    },
    "relations": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "predicate": {
            "type": "string",
            "enum": [
              "WasGeneratedBy",
              "Used",
              "WasDerivedFrom",
              "WasTriggeredBy",
              "WasControlledBy",
              "WasRevisionOf"
            ]
          },
          "identifier": {
            "type": "string"
          },
          "role": {
            "type": "string",
            "enum": [
              "subject",
              "object"
            ]
          }
        },
        "required": [
          "predicate",
          "identifier",
          "role"
        ]
      }
    },
    "sources": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "date": {
            "type": "number"
          },
          "doi": {
            "type": "string"
          },
          "isbn": {
            "type": "string"
          },
          "language": {
            "type": "string"
          },
          "source": {
            "type": "string"
          }
        }
      }
    },
    "tags": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "type": {
      "type": "string"
    },
    "version": {
      "type": "object",
      "properties": {
        "created": {
          "type": "number"
        },
        "name": {
          "type": "string"
        },
        "revision": {
          "$ref": "#/definitions/SemanticVersion"
        }
      }
    }
  },
  "required": [
    "id_"
  ],
  "additionalProperties": false
}