Skip to main content

Export Lakehouse Notebook Results to Azure SQL DB

Export data from Lakehouse to Azure SQL DB.

To use this activity within the API, use an ActivityCode of DELTA-AZSQL.

Example JSON​

An example of what the Task Config would look like for a task using this activity. Some of these variables would be set at the group level to avoid duplication between tasks.

NULL

Variable Reference​

The following variables are supported:

  • SourceConnection - (Required) Source connection to use.

  • NotebookPath - (Required) The relative path to the Databricks Notebook.

  • NotebookParameters - (Optional) Parameters for use in the Databricks Notebook. This is JSON format e.g. { "Param1": "Value1", "Param2": "Value2" }.

  • TargetConnection - (Required) Target connection to use.

  • TargetSchemaName - (Required) Name of the schema within target Azure SQL Database (e.g. dbo).

  • TargetTableName - (Required) Name of the table within target Azure SQL Database (e.g. Climate).

  • AutoCreateTable - (Optional) Should the target table be created automatically?

  • SaveExportToHistory - (Optional) Save the exported data to history in the Lakehouse?

  • DatabricksClusterId - (Optional) The Id of the Databricks Cluster to use to run the Notebook or Query.

  • PreCopyScript - (Optional) Script to execute on target database prior to Copy.

  • WriteBehaviour - (Optional) The method of writing data to the target table.

  • UpsertKeyColumns - (Optional) A comma-separated list of column names comprising the unique key of the target table.

  • ExtractControlQuery - (Optional) For incremental loads only, a SQL query to get a 'high-water' mark for extract control. For instance, this could be the maximum value of an modified_date or an identity column. NOTE: The column returned must be aliased as ExtractControl e.g. select max(modified_date) as ExtractControl from some_table.

  • ExtractControlVariableName - (Optional) For incremental loads only, the name to assign the Extract Control variable in State Config for the ExtractControl value derived from the Extract Control Query above.

  • ExtractControlVariableSeedValue - (Optional) The initial value to set for the Extract Control variable in State Config - this will have no impact beyond the original seeding of the Extract Control variable in State Config.

  • DIUsToUseForCopyActivity - (Optional) Specifies the powerfulness of the copy executor. Value can be between 2 and 256. When left at default, the Data Factory dynamically applies the optimal DIU setting based on the source-sink pair and data pattern.

  • MaximumNumberOfAttemptsAllowed - (Optional) The total number of times the running of this Task can be attempted.

  • MinutesToWaitBeforeNextAttempt - (Optional) If a Task run fails, the number of minutes to wait before re-attempting the Task.

  • IsFederated - (Optional) Makes task available to other Insight Factories within this organisation.

  • Links - (Optional) NULL