Custom development¶
Custom development for omega|ml is available for the following extensions:
storage backend & mixins - process and store new data types
model backend - integrate other machine learning frameworks or custom algorithms
runtime mixins & tasks - enable new tasks in the distributed compute-cluster
Backends provide the put,get semantics to store and retrieve objects
where as mixins provide overrides or extensions to existing implementations.
Think of a backend as the storage engine for objects a specific data type
(e.g. a Pandas Dataframe) while a mixin provide the pre- or post-processing
applied to these objects on specific method calls.
Semantics¶
Technically, storage and model backends, as well as storage mixins, extend the
capability of OmegaStore. Runtime mixins and tasks extend the
capability of OmegaRuntime. MDataFrame mixins extend the capability
of lazy-evaluation dataframes.
A data backend shall adhere to the protocol established by BaseDataBackend.
Similarly a model backend shall adhere to to the protocol established by
the BaseModelBackend.
Both backend types support the general storage put,get semantics to
store and retrieve objects, respectively. Model backends in addition provide
methods for specific model actions (e.g. fit and predict), following the
semantics of scikit-learn_.
Note
In principle a backend need not be a subclass of either of the two base
backends, however there is some default processing implemented in the base
backends __init__ methods so that sub-classing is the more practical
method.
Mixins are objects that implement arbitrary methods for their respective target.
For example, a mixin for OmegaStore may implement a get method,
extending the store’s default implementation. Mixins are applied to their target
the same way as a subclass would be.