Skip to main content

Run Databricks Notebook

Run Notebook.

To use this activity within the API, use an ActivityCode of RUN-NOTEBOOK.

Example JSON​

An example of what the Task Config would look like for a task using this activity. Some of these variables would be set at the group level to avoid duplication between tasks.

{
    "DatabricksNotebookPath": "/Repos/Published/Notebooks/Climate/My Climate Notebook",
    "ParametersToPassToNotebook": {
        "MinRainfall": "5",
        "MaxTemperature": "38"
    }|||NEW

Variable Reference​

The following variables are supported:

  • NotebookPath - (Required) The relative path to the Databricks Notebook.

  • NotebookParameters - (Optional) Parameters for use in the Databricks Notebook. This is JSON format e.g. { "Param1": "Value1", "Param2": "Value2" }.

  • AdditionalNotebooks - (Optional) The Path to other Notebooks referenced by the main Notebook.

  • DeltaSchemaName - (Optional) The name of the Schema this transformation lives in.

  • DeltaTableName - (Optional) The name of the Lakehouse Table representing this transformation.

  • DatabricksClusterId - (Optional) The Id of the Databricks Cluster to use to run the Notebook.

  • ExtractControlVariableName - (Optional) For incremental loads only, the name to assign the Extract Control variable in State Config for the ExtractControl value derived from the Extract Control Query above.

  • ExtractControlVariableSeedValue - (Optional) The initial value to set for the Extract Control variable in State Config - this will have no impact beyond the original seeding of the Extract Control variable in State Config.

  • MaximumNumberOfAttemptsAllowed - (Optional) The total number of times the running of this Task can be attempted.

  • MinutesToWaitBeforeNextAttempt - (Optional) If a Task run fails, the number of minutes to wait before re-attempting the Task.

  • IsFederated - (Optional) Makes task available to other Insight Factories within this organisation.

  • Links - (Optional) NULL