Build an ML Model and update ML Catalog

Build an ML Model into the ML Catalog.

Category: Enrich Lakehouse Table | Tags: Enrichment

To use this activity within the API, use an ActivityCode of ML-BUILD-MODEL.

Example JSON

An example of what the Task Config would look like for a task using this activity. Some of these variables would be set at the group level to avoid duplication between tasks.

{
  "NotebookPath": "/Users/fred.nurks@example.com/MyRepo/My Notebook",
  "ModelSchemaName": "example_schema",
  "ModelName": "",
  "NotebookParameters": { "Param1": "Value1", "Param2": "Value2" }
}

Variable Reference

The following variables are supported:

• AdditionalNotebooks (Optional) - The path to other notebooks, Python files etc., referenced by the main notebook.

• DatabricksClusterId (Optional) - The Databricks Cluster to use for this task.

• ExtractControlVariableName (Optional) - For incremental loads only, the name to assign the Extract Control variable in State Config for the ExtractControl value derived from the Extract Control Query above.

• ExtractControlVariableSeedValue (Optional) - The initial value to set for the Extract Control variable in State Config - this will have no impact beyond the original seeding of the Extract Control variable in State Config.

• IsFederated (Optional) - Makes task available to other Insight Factories within this organisation.

• Links (Optional) - NULL

• MaximumNumberOfAttemptsAllowed (Optional) - The total number of times the running of this Task can be attempted.

• MinutesToWaitBeforeNextAttempt (Optional) - If a Task run fails, the number of minutes to wait before re-attempting the Task.

• ModelName (Required) - Name of the ML Model.

• ModelSchemaName (Required) - The Schema the ML Model resides in.

• NotebookParameters (Optional) - Parameters for use in the Databricks Notebook. This is JSON format e.g. { "Param1": "Value1", "Param2": "Value2" }.

• NotebookPath (Required) - The relative path to the Databricks Notebook.