Skip to main content

Create/Update a Training Feature Log

Run Enrichment Notebook that will create/update a Lakehouse Table.

Category: Machine Learning | Tags: ML

How it works

Add records to the Training Feature Log table '<<DeltaSchemaName>>.feature_log'

To use this activity within the API, use an ActivityCode of ML-FEATURE-LOG.

Example JSON​

An example of what the Task Config would look like for a task using this activity. Some of these variables would be set at the group level to avoid duplication between tasks.

{
  "ModelName": "",
  "FeatureEngineeringSchemaName": "example_schema",
  "FeatureEngineeringTableName": "my_table",
  "FeatureEngineeringIdColumnName": "",
  "DeltaSchemaName": "example_schema"
}

Variable Reference​

The following variables are supported:

  • DatabricksClusterId (Optional) - The Databricks Cluster to use for this task.

  • DeltaSchemaName (Required) - The name of the Schema this transformation lives in.

  • ExtractControlVariableName (Optional) - For incremental loads only, the name to assign the Extract Control variable in State Config for the ExtractControl value derived from the Extract Control Query above.

  • ExtractControlVariableSeedValue (Optional) - The initial value to set for the Extract Control variable in State Config - this will have no impact beyond the original seeding of the Extract Control variable in State Config.

  • FeatureEngineeringIdColumnName (Required) - The identifier column in the Feature Engineering table.

  • FeatureEngineeringSchemaName (Required) - The schema name of the Feature Engineering table this log is derived from.

  • FeatureEngineeringTableName (Required) - The table name of the Feature Engineering table this log is derived from.

  • IsFederated (Optional) - Makes task available to other Insight Factories within this organisation.

  • Links (Optional) - NULL

  • MaximumNumberOfAttemptsAllowed (Optional) - The total number of times the running of this Task can be attempted.

  • MinutesToWaitBeforeNextAttempt (Optional) - If a Task run fails, the number of minutes to wait before re-attempting the Task.

  • ModelName (Required) - Name of the ML Model.

  • SkipCreateVolumeAndSchema (Optional) - If a Schema and/or Volume has already been created, you can opt to skip this check - it will lead to better performance.