om.models

omegaml.models = OmegaStore(bucket=omegaml, prefix=models/)

the omegaml.store.base.OmegaStore store for models

Methods:

• omegaml.store.base.OmegaStore.list()

• omegaml.store.base.OmegaStore.get()

• omegaml.store.base.OmegaStore.getl()

• omegaml.store.base.OmegaStore.put()

• omegaml.store.base.OmegaStore.drop()

• omegaml.store.base.OmegaStore.exists()

• omegaml.store.base.OmegaStore.metadata()

• omegaml.store.base.OmegaStore.help()

Mixins:

• omegaml.mixins.store.virtualobj.VirtualObjectMixin

• omegaml.mixins.store.promotion.PromotionMixin

• omegaml.mixins.store.extdmeta.ExtendedMetadataMixin

• omegaml.mixins.store.extdmeta.ModelSignatureMixin

Backends:

• omegaml.backends.experiment.ExperimentBackend

• omegaml.backends.scikitlearn.ScikitLearnBackend

Backends for tensorflow (loaded only if installed):

• omegaml.backends.tensorflow.TensorflowKerasBackend

• omegaml.backends.tensorflow.TensorflowKerasSavedModelBackend

• omegaml.backends.tensorflow.TensorflowSavedModelBackend

• omegaml.backends.tensorflow.TFEstimatorModelBackend

Backends for keras (loaded only if installed):

• omegaml.backends.keras.KerasBackend

Backends for mlflow (loaded only if installed)

• omegaml.backends.mlflow.models.MLFlowModelBackend

• omegaml.backends.mlflow.registrymodels.MLFlowRegistryBackend

Backends for R (loaded if installed)

• omegaml.backends.rsystem.rmodels.RModelBackend

Backends for openai (loaded if installed)

Added in version 0.17.0.

• omegaml.backends.genai.GenAIBaseBackend

• omegaml.backends.genai.textmodel.TextModelBackend

Backends

class omegaml.backends.scikitlearn.ScikitLearnBackend(model_store=None, data_store=None, tracking=None, **kwargs)

KIND = 'sklearn.joblib'

class omegaml.backends.experiment.ExperimentBackend(model_store=None, data_store=None, tracking=None, **kwargs)

ExperimentBackend provides storage of tracker configurations

Usage:

To log metrics and other data:

with om.runtime.experiment('myexp') as exp:
    om.runtime.model('mymodel').fit(X, Y)
    om.runtime.model('mymodel').score(X, Y) # automatically log score result
    exp.log_metric('mymetric', value)
    exp.log_param('myparam', value)
    exp.log_artifact(X, 'X')
    exp.log_artifact(Y, 'Y')
    exp.log_artifact(om.models.metadata('mymodel'), 'mymodel')

To log data and automatically profile system data:

with om.runtime.experiment('myexp', provider='profiling') as exp:
    om.runtime.model('mymodel').fit(X, Y)
    om.runtime.model('mymodel').score(X, Y) # automatically log score result
    exp.log_metric('mymetric', value)
    exp.log_param('myparam', value)
    exp.log_artifact(X, 'X')
    exp.log_artifact(Y, 'Y')
    exp.log_artifact(om.models.metadata('mymodel'), 'mymodel')

# profiling data contains metrics for cpu, memory and disk use
data = exp.data(event='profile')

To get back experiment data without running an experiment:

# recommended way
exp = om.runtime.experiment('myexp').use()
exp_df = exp.data()

# experiments exist in the models store
exp = om.models.get('experiments/myexp')
exp_df = exp.data()

See also

• omegaml.backends.tracking.OmegaSimpleTracker

• omegaml.backends.tracking.OmegaProfilingTracker

KIND = 'experiment.tracker'

get(name, raw=False, data_store=None, **kwargs)

retrieve a model

Parameters:

• name – the name of the object

• uri – optional, /path/to/file, defaults to meta.gridfile, may use /path/{key} as placeholder for the file’s name

• version – the version of the object (not supported)

put(obj, name, **kwargs)

store a model

Parameters:

• obj – the model object to be stored

• name – the name of the object

• uri – optional, /path/to/file, defaults to meta.gridfile, may use /path/{key} as placeholder for the file’s name

• attributes – attributes for meta data

classmethod supports(obj, name, **kwargs)

test if this backend supports this obj

class omegaml.backends.tensorflow.TensorflowKerasBackend(model_store=None, data_store=None, tracking=None, **kwargs)

KIND = 'tfkeras.h5'

fit(modelname, Xname, Yname=None, pure_python=True, tpu_specs=None, **kwargs)

Fit a model

Parameters:

• modelname – the name of the model

• Xname – the name of the X dataset

• Yname – the name of the Y dataset

• pure_python – deprecated

• kwargs – any additional kwargs are passed on to model.fit()

Returns:

the meta data object of the model

classmethod supports(obj, name, **kwargs)

test if this backend supports this obj

class omegaml.backends.tensorflow.TensorflowKerasSavedModelBackend(model_store=None, data_store=None, tracking=None, **kwargs)

KIND = 'tfkeras.savedmodel'

fit(modelname, Xname, Yname=None, pure_python=True, tpu_specs=None, **kwargs)

fit the model with data

Parameters:

• modelname – the name of the model object

• Xname – the name of the X data set

• Yname – the name of the Y data set

• pure_python – if True return a python object. If False return a dataframe. Defaults to True to support any client.

• kwargs – kwargs passed to the model’s predict method

Returns:

return the meta data object of the model

classmethod supports(obj, name, **kwargs)

test if this backend supports this obj

class omegaml.backends.tensorflow.TensorflowSavedModelBackend(model_store=None, data_store=None, tracking=None, **kwargs)

KIND = 'tf.savedmodel'

fit(modelname, Xname, Yname=None, pure_python=True, tpu_specs=None, **kwargs)

fit the model with data

Parameters:

• modelname – the name of the model object

• Xname – the name of the X data set

• Yname – the name of the Y data set

• pure_python – if True return a python object. If False return a dataframe. Defaults to True to support any client.

• kwargs – kwargs passed to the model’s predict method

Returns:

return the meta data object of the model

predict(modelname, Xname, rName=None, pure_python=True, **kwargs)

Predict from a SavedModel

Parameters:

• modelname

• Xname

• rName

• pure_python

• kwargs

Returns:

classmethod supports(obj, name, **kwargs)

test if this backend supports this obj

class omegaml.backends.tensorflow.TFEstimatorModelBackend(model_store=None, data_store=None, tracking=None, **kwargs)

KIND = 'tfestimator.model'

fit(modelname, Xname, Yname=None, pure_python=True, **kwargs)

fit the model with data

Parameters:

• modelname – the name of the model object

• Xname – the name of the X data set

• Yname – the name of the Y data set

• pure_python – if True return a python object. If False return a dataframe. Defaults to True to support any client.

• kwargs – kwargs passed to the model’s predict method

Returns:

return the meta data object of the model

predict(modelname, Xname, rName=None, pure_python=True, **kwargs)

predict using data stored in Xname

Parameters:

• modelname – the name of the model object

• Xname – the name of the X data set

• rName – the name of the result data object or None

• pure_python – if True return a python object. If False return a dataframe. Defaults to True to support any client.

• kwargs – kwargs passed to the model’s predict method

Returns:

return the predicted outcome

score(modelname, Xname, Yname=None, rName=True, pure_python=True, **kwargs)

score using data

Parameters:

• modelname – the name of the model object

• Xname – the name of the X data set

• Yname – the name of the Y data set

• rName – the name of the transforms’s result data object or None

• pure_python – if True return a python object. If False return a dataframe. Defaults to True to support any client.

• kwargs – kwargs passed to the model’s predict method

Returns:

return the score result

classmethod supports(obj, name, **kwargs)

test if this backend supports this obj

class omegaml.backends.rsystem.rmodels.RModelBackend(model_store=None, data_store=None, tracking=None, **kwargs)

KIND = 'model.r'

predict(modelname, Xname, rName=None, pure_python=True, **kwargs)

predict using data stored in Xname

Parameters:

• modelname – the name of the model object

• Xname – the name of the X data set

• rName – the name of the result data object or None

• pure_python – if True return a python object. If False return a dataframe. Defaults to True to support any client.

• kwargs – kwargs passed to the model’s predict method

Returns:

return the predicted outcome

classmethod supports(obj, name, **kwargs)

test if this backend supports this obj

class omegaml.backends.genai.textmodel.TextModelBackend(model_store=None, data_store=None, tracking=None, **kwargs)

Backend for OpenAI models

Enables creating an OpenAI model via a connection string, e.g. om.models.put(‘openai://<base_url>;model=<model>’, ‘mymodel’). The connection string must be in the format openai://<base_url>;model=<model> where <base_url> is the base URL of the OpenAI-compatible model server’s REST API, and <model> is the model name as known to the model server. The connection string may also include an apikey, e.g. ‘openai://<apikey>@<base_url>;model=<model>’.

Usage:

# create a model om.models.put(‘openai://localhost:8000/mymodel’, ‘mymodel’) # get the model model = om.models.get(‘mymodel’) # use the model result = model.complete(‘hello, how are you?’)

Notes

• the actual implementation of the model handling logic is in OpenAIModel, this only provides the model store interface and acts as any VirtualObjectHandler

KIND = 'genai.text'

get(name, template=None, data_store=None, pipeline=None, tools=None, documents=None, strategy=None, tracking=None, secrets=None, **kwargs)

get an obj

Parameters:

name – the name of the object (str)

Returns:

the object as it was originally stored

put(obj, name, template=None, pipeline=None, provider=None, tools=None, documents=None, strategy=None, **kwargs)

put an obj

Parameters:

• obj – the object to store (object)

• name – the name of the object (str)

• attributes – the attributes dict (dict, optional)

• kwargs – other kwargs to be passed to the Metadata object

Returns:

the Metadata object

classmethod supports(obj, name, **kwargs)

test if this backend supports this obj

Mixins

class omegaml.mixins.store.virtualobj.VirtualObjectMixin

process virtual objects

This checks if an object is a VirtualObject and if so retrieves the handler and processes it.

class omegaml.mixins.store.promotion.PromotionMixin

Promote objects from one bucket to another

Promotion Methods:

• getput - performs target.put(source.get()) and copies metadata

  attributes by merging target.metadata’s .attributes and .kind_meta

• metadata - creates a new metadata data entry in target, copying

  metadata attributes and kind_meta. Does not get/put the object itself (i.e. no associated data is promoted).

• data - like getput, but does not merge metadata

• export - performs .to_archive() and .from_archive(), effectively

  copying metadata, the associated gridfile (if available) and collection data (if available). This is equivalent to om runtime export.

The default promotion method is getput(), or the object’s backend.PROMOTE method, if specified.

Some object backends provide a default promotion other than getput:

• sqlalchemy.conx - uses the metadata promotion, effectively copying only

  metadata. Use promote(…, method=’metadata,data’) to also promote data

• virtualobj.dill - uses the export promotion, effectively copying all

  metadata and the associated @virtualobj function. If the source object is a versioned model, this copies the current version and metadata. To copy a specific version, use promote(‘model@version’). To create a new version in the target bucket use promote(…, method=’getput’).

promote(name, other, asname=None, drop=None, method='default', get=None, put=None, **kwargs)

Promote object to another store.

This effectively copies the object. If the objects exists in the target it will be replaced.

Parameters:

• name – The name of the object

• other – the OmegaStore instance to promote to

• asname – the name to use in other, defaults to .metadata(name).name

• drop (bool) – if True calls other.drop(force=True) before promoting, defaults to False

• method (str|list) – specify the method or multiple methods in sequence, available methods are ‘default’, ‘getput’, ‘metadata’, ‘data’. For ‘default’, the object backend’s .PROMOTE property is used, defaulting to ‘getput’

• get (dict) – optional, specifies the store.get(**kwargs)

• put (dict) – optional, specifies the other.put(**kwargs)

• kwargs – additional kwargs are passed to the initial other.put(), for metadata promotion

Returns:

The Metadata of the new object

class omegaml.mixins.store.extdmeta.ModelSignatureMixin

link_dataset(name, Xname=None, Yname=None, Xmeta=None, Ymeta=None, rName=None, features=None, labels=None, data_store=None, meta=None, signature=True, actions=None, **kwargs)

link dataset information to this model

This sets the ‘dataset’ entry in metadata.attributes of a model. By default this method is called by any .fit() call initiated from the runtime. The ‘dataset’ entry is a dict that records the following information:

{
    'Xname': Xname,  # the X dataset
    'Yname': Yname,  # the Y dataset
    'features': features, # features for this model
    'labels': labels, # labels for this model
    'Xmeta': Xmeta, # Metadata for this model (at time of fit)
    'Ymeta': Ymeta, # Metadata for this model (at time of fit)
    'rName': rName, # the result dataset
    'kwargs': kwargs, # other kwargs used for fitting
}

Parameters:

• name (str) – the name of the model or script

• Xname (str) – the name of dataset, may include modifiers

• Yname (str) – the name of dataset, may include modifiers

• rName (str) – the result dataset

• Xmeta (Metadata) – the Metadata for X (features), defaults to data_store.metadata(Xname)

• Ymeta (Metadata) – the Metadata for Y (features), defaults to data_store.metadata(Yname)

• features (list|dict) – a list of feature names or dict(name=info), where info is a dict with further information for each features

• labels (list|dict) – a list of label names or dict(label=info), where info is a dict with futher information for each label

• signature (bool) – if True the object’s signature will also be updated, defaults to True

• meta (Metadata) – the metadata object, if provided .metadata() is not called again

• data_store (OmegaStore) – the data store to retrieve Xmeta, Ymeta if not provided

• **kwargs – passed to self.metadata(name, **kwargs) to retrieve the model metadata

Helpers

class omegaml.backends.rsystem.rmodels.RModelProxy(key)

a Python proxy to the R process that runs a model

This provides the model.predict() interface for R models so that we can use the same semantics for R and python scripts.

predict(X_or_name)

call $model.predict()

Parameters:

X_or_name (str) – the X object or name of the dataset