Configuration ¶

Configuration hooks ¶

omega-ml is configured by constants defined in the omegaml.defaults module. Since this module is usually not directly changeable by the user or administrator, the following hooks are provided:

configuration file (`$HOME/.omegaml/config.yml`)¶

The configuration file in Yaml format is read automatically on omega-ml startup, if available.

Note

Commercial Edition A user-specific configuration file can be obtained by any authorized user by running the python -m omegacli init command. To override defaults, specify the corresponding variable in Yaml format.

system environment variables ¶

To change a particular default, set the environment variable of the same name as the respective configuration parameter. As an example, to change the OMEGA_MONGO_URL parameter, set the OMEGA_MONGO_URL environment variable. Note to change an entry in OMEGA_CELERY_CONFIG, add the OMEGA_CELERY_ prefix to the environment variable name. As an example, to change OMEGA_CELERY[BROKER_URL] use the env var OMEGA_CELERY_BROKER_URL.

in-code update ¶

If you integrate omega-ml into your application, the omegaml.defaults.update_from_obj provides a direct way to update the defaults from any object. The object needs to support getattr on the corresponding defaults parameter.

The hooks are applied in the above order, that is the defaults are overriden by configuration file, then operating system environment variables, then constance parameters and finally your own code. Note that parameter values specified in constance cannote be overridden by our own code except by changing the value in the constance Django table directly.

Basic configuration ¶

omegaml.defaults.OMEGA_TMP¶

the temp directory used by omegaml processes

Defaults to /tmp

omegaml.defaults.OMEGA_MONGO_URL¶

the fully qualified mongodb database URL, including the database name

Format mongodb://user:password@host:port/database

omegaml.defaults.OMEGA_MONGO_COLLECTION = 'omegaml'¶: the collection name in the mongodb used by omegaml storage

Storage configuration ¶

omegaml.defaults.OMEGA_STORE_BACKENDS = {'experiment.tracker': 'omegaml.backends.tracking.ExperimentBackend', 'ndarray.bin': 'omegaml.backends.npndarray.NumpyNDArrayBackend', 'pandas.csv': 'omegaml.backends.externaldata.PandasExternalData', 'pandas.rawdict': 'omegaml.backends.rawdict.PandasRawDictBackend', 'pipsrc.package': 'omegaml.backends.package.PythonPipSourcedPackageData', 'python.file': 'omegaml.backends.rawfiles.PythonRawFileBackend', 'python.package': 'omegaml.backends.package.PythonPackageData', 'sklearn.joblib': 'omegaml.backends.scikitlearn.ScikitLearnBackend', 'virtualobj.dill': 'omegaml.backends.virtualobj.VirtualObjectBackend'}¶

storage backends

Dictionary of pairs { 'kind': class }, where kind is the Metadata.kind of the stored object, and class is the python loadable name of the class that implements handling of this kind. The storage backends listed in this variable are automatically loaded.

omegaml.defaults.OMEGA_STORE_MIXINS = ['omegaml.mixins.store.ProjectedMixin', 'omegaml.mixins.store.LazyGetMixin', 'omegaml.mixins.store.virtualobj.VirtualObjectMixin', 'omegaml.mixins.store.package.PythonPackageMixin', 'omegaml.mixins.store.promotion.PromotionMixin', 'omegaml.mixins.mdf.iotools.IOToolsStoreMixin', 'omegaml.mixins.store.modelversion.ModelVersionMixin', 'omegaml.mixins.store.datarevision.DataRevisionMixin', 'omegaml.mixins.store.imexport.ObjectImportExportMixin', 'omegaml.mixins.store.extdmeta.SignatureMixin', 'omegaml.mixins.store.extdmeta.ScriptSignatureMixin', 'omegaml.mixins.store.extdmeta.ModelSignatureMixin', 'omegaml.mixins.store.requests.RequestCache', 'omegaml.mixins.store.passthrough.PassthroughMixin', 'omegaml.mixins.store.tracking.TrackableMetadataMixin', 'omegaml.mixins.store.tracking.UntrackableMetadataMixin']¶

storage mixins

List of storage mixin classes. The mixins listed here are automatically applied to each OmegaStore instance.

Celery Cluster configuration ¶

omegaml.defaults.OMEGA_BROKER = 'amqp://admin:foobar@localhost:5672//'¶: the celery broker name or URL

omegaml.defaults.OMEGA_RESULT_BACKEND = 'rpc://'¶: the celery backend name or URL

omegaml.defaults.OMEGA_CELERY_CONFIG = {'BROKER_CONNECTION_RETRY_ON_STARTUP': True, 'BROKER_HEARTBEAT': 0, 'BROKER_URL': 'amqp://admin:foobar@localhost:5672//', 'BROKER_USE_SSL': False, 'CELERYBEAT_SCHEDULE': {'execute_scripts': {'schedule': 60, 'task': 'omegaml.notebook.tasks.execute_scripts'}}, 'CELERYD_CONCURRENCY': 4, 'CELERY_ACCEPT_CONTENT': ['pickle', 'json'], 'CELERY_ALWAYS_EAGER': False, 'CELERY_DEFAULT_QUEUE': 'default', 'CELERY_RESULT_BACKEND': 'rpc://', 'CELERY_RESULT_EXPIRES': 3600, 'CELERY_RESULT_SERIALIZER': 'pickle', 'CELERY_TASK_RESULT_EXPIRES': 3600, 'CELERY_TASK_SERIALIZER': 'pickle'}¶

the celery configurations

This is used by omemgal to configure the celery application. Note that the configuration must be the same for both client and cluster worker.

Client-side configuration (constance)¶

These parameters are in the admin UI at http://localhost:5000/admin/constance/config:

BROKER_URL - this is the rabbitmq broker used by the Celery cluster. Set as ampq://public-omegaml-hostname:port/<vhost>/. Set vhost depending on your rabbitmq configuration. By default the vhost is an empty string
MONGO_HOST - set as public-mongodb-hostname:port
CELERY_ALWAYS_EAGER - if this True, all calls to the runtime are in fact executed locally on the calling machine. Note this also means that the REST API will not submit any tasks to the cluster.