om.models

omegaml.models = OmegaStore(bucket=omegaml, prefix=models/)

the omegaml.store.base.OmegaStore store for models

Methods:

Mixins:

Backends:

Backends for tensorflow (loaded only if installed):

  • omegaml.backends.tensorflow.TensorflowKerasBackend

  • omegaml.backends.tensorflow.TensorflowKerasSavedModelBackend

  • omegaml.backends.tensorflow.TensorflowSavedModelBackend

  • omegaml.backends.tensorflow.TFEstimatorModelBackend

Backends for keras (loaded only if installed):

  • omegaml.backends.keras.KerasBackend

Backends for mlflow (loaded only if installed)

  • omegaml.backends.mlflow.models.MLFlowModelBackend

  • omegaml.backends.mlflow.registrymodels.MLFlowRegistryBackend

Backends for R (loaded if installed)

Backends

class omegaml.backends.scikitlearn.ScikitLearnBackend(model_store=None, data_store=None, tracking=None, **kwargs)
KIND = 'sklearn.joblib'
class omegaml.backends.experiment.ExperimentBackend(model_store=None, data_store=None, tracking=None, **kwargs)

ExperimentBackend provides storage of tracker configurations

Usage:

To log metrics and other data:

with om.runtime.experiment('myexp') as exp:
    om.runtime.model('mymodel').fit(X, Y)
    om.runtime.model('mymodel').score(X, Y) # automatically log score result
    exp.log_metric('mymetric', value)
    exp.log_param('myparam', value)
    exp.log_artifact(X, 'X')
    exp.log_artifact(Y, 'Y')
    exp.log_artifact(om.models.metadata('mymodel'), 'mymodel')

To log data and automatically profile system data:

with om.runtime.experiment('myexp', provider='profiling') as exp:
    om.runtime.model('mymodel').fit(X, Y)
    om.runtime.model('mymodel').score(X, Y) # automatically log score result
    exp.log_metric('mymetric', value)
    exp.log_param('myparam', value)
    exp.log_artifact(X, 'X')
    exp.log_artifact(Y, 'Y')
    exp.log_artifact(om.models.metadata('mymodel'), 'mymodel')

# profiling data contains metrics for cpu, memory and disk use
data = exp.data(event='profile')

To get back experiment data without running an experiment:

# recommended way
exp = om.runtime.experiment('myexp').use()
exp_df = exp.data()

# experiments exist in the models store
exp = om.models.get('experiments/myexp')
exp_df = exp.data()

See also

  • omegaml.backends.tracking.OmegaSimpleTracker

  • omegaml.backends.tracking.OmegaProfilingTracker

KIND = 'experiment.tracker'
get(name, raw=False, data_store=None, **kwargs)

retrieve a model

Parameters:

  • name – the name of the object

  • version – the version of the object (not supported)

put(obj, name, **kwargs)

store a model

Parameters:

  • obj – the model object to be stored

  • name – the name of the object

  • attributes – attributes for meta data

classmethod supports(obj, name, **kwargs)

test if this backend supports this obj

class omegaml.backends.rsystem.rmodels.RModelBackend(model_store=None, data_store=None, tracking=None, **kwargs)
KIND = 'model.r'
predict(modelname, Xname, rName=None, pure_python=True, **kwargs)

predict using data stored in Xname

Parameters:

  • modelname – the name of the model object

  • Xname – the name of the X data set

  • rName – the name of the result data object or None

  • pure_python – if True return a python object. If False return a dataframe. Defaults to True to support any client.

  • kwargs – kwargs passed to the model’s predict method

Returns:

return the predicted outcome

classmethod supports(obj, name, **kwargs)

test if this backend supports this obj

Mixins

class omegaml.mixins.store.virtualobj.VirtualObjectMixin

process virtual objects

This checks if an object is a VirtualObject and if so retrieves the handler and processes it.

class omegaml.mixins.store.promotion.PromotionMixin

Promote objects from one bucket to another

Promotion Methods:
  • getput - performs target.put(source.get()) and copies metadata

    attributes by merging target.metadata’s .attributes and .kind_meta

  • metadata - creates a new metadata data entry in target, copying

    metadata attributes and kind_meta. Does not get/put the object itself (i.e. no associated data is promoted).

  • data - like getput, but does not merge metadata

  • export - performs .to_archive() and .from_archive(), effectively

    copying metadata, the associated gridfile (if available) and collection data (if available). This is equivalent to om runtime export.

The default promotion method is getput(), or the object’s backend.PROMOTE method, if specified.

Some object backends provide a default promotion other than getput:

  • sqlalchemy.conx - uses the metadata promotion, effectively copying only

    metadata. Use promote(…, method=’metadata,data’) to also promote data

  • virtualobj.dill - uses the export promotion, effectively copying all

    metadata and the associated @virtualobj function. If the source object is a versioned model, this copies the current version and metadata. To copy a specific version, use promote(‘model@version’). To create a new version in the target bucket use promote(…, method=’getput’).

promote(name, other, asname=None, drop=None, method='default', get=None, put=None, **kwargs)

Promote object to another store.

This effectively copies the object. If the objects exists in the target it will be replaced.

Parameters:

  • name – The name of the object

  • other – the OmegaStore instance to promote to

  • asname – the name to use in other, defaults to .metadata(name).name

  • drop (bool) – if True calls other.drop(force=True) before promoting, defaults to False

  • method (str|list) – specify the method or multiple methods in sequence, available methods are ‘default’, ‘getput’, ‘metadata’, ‘data’. For ‘default’, the object backend’s .PROMOTE property is used, defaulting to ‘getput’

  • get (dict) – optional, specifies the store.get(**kwargs)

  • put (dict) – optional, specifies the other.put(**kwargs)

  • kwargs – additional kwargs are passed to the initial other.put(), for metadata promotion

Returns:

The Metadata of the new object

class omegaml.mixins.store.extdmeta.ModelSignatureMixin

link dataset information to this model

This sets the ‘dataset’ entry in metadata.attributes of a model. By default this method is called by any .fit() call initiated from the runtime. The ‘dataset’ entry is a dict that records the following information:

{
    'Xname': Xname,  # the X dataset
    'Yname': Yname,  # the Y dataset
    'features': features, # features for this model
    'labels': labels, # labels for this model
    'Xmeta': Xmeta, # Metadata for this model (at time of fit)
    'Ymeta': Ymeta, # Metadata for this model (at time of fit)
    'rName': rName, # the result dataset
    'kwargs': kwargs, # other kwargs used for fitting
}
Parameters:

  • name (str) – the name of the model or script

  • Xname (str) – the name of dataset, may include modifiers

  • Yname (str) – the name of dataset, may include modifiers

  • rName (str) – the result dataset

  • Xmeta (Metadata) – the Metadata for X (features), defaults to data_store.metadata(Xname)

  • Ymeta (Metadata) – the Metadata for Y (features), defaults to data_store.metadata(Yname)

  • features (list|dict) – a list of feature names or dict(name=info), where info is a dict with further information for each features

  • labels (list|dict) – a list of label names or dict(label=info), where info is a dict with futher information for each label

  • signature (bool) – if True the object’s signature will also be updated, defaults to True

  • meta (Metadata) – the metadata object, if provided .metadata() is not called again

  • data_store (OmegaStore) – the data store to retrieve Xmeta, Ymeta if not provided

  • **kwargs – passed to self.metadata(name, **kwargs) to retrieve the model metadata

Helpers

class omegaml.backends.rsystem.rmodels.RModelProxy(key)

a Python proxy to the R process that runs a model

This provides the model.predict() interface for R models so that we can use the same semantics for R and python scripts.

predict(X_or_name)

call $model.predict()

Parameters:

X_or_name (str) – the X object or name of the dataset