Skip to main content

Export Lakehouse Query Results to Azure SQL DB

Execute a Query against the Lakehouse and export the results to Azure SQL DB.

To use this activity within the API, use an ActivityCode of DELTA-QUERY-AZSQL.

Example JSON​

An example of what the Task Config would look like for a task using this activity. Some of these variables would be set at the group level to avoid duplication between tasks.

NULL

Variable Reference​

The following variables are supported:

  • SourceConnection - (Required) Source connection to use.

  • DeltaQuery - (Optional) Query to run against the Lakehouse.

  • TargetConnection - (Required) Target connection to use.

  • TargetSchemaName - (Required) Name of the schema within target Azure SQL Database (e.g. dbo).

  • TargetTableName - (Required) Name of the table within target Azure SQL Database (e.g. Climate).

  • AutoCreateTable - (Optional) Should the target table be created automatically?

  • SaveExportToHistory - (Optional) Save the exported data to history in the Lakehouse?

  • DatabricksClusterId - (Optional) The Id of the Databricks Cluster to use to run the Notebook or Query.

  • PreCopyScript - (Optional) Script to execute on target database prior to Copy.

  • WriteBehaviour - (Optional) The method of writing data to the target table.

  • UpsertKeyColumns - (Optional) A comma-separated list of column names comprising the unique key of the target table.

  • ExtractControlQuery - (Optional) For incremental loads only, a SQL query to get a 'high-water' mark for extract control. For instance, this could be the maximum value of an modified_date or an identity column. NOTE: The column returned must be aliased as ExtractControl e.g. select max(modified_date) as ExtractControl from some_table.

  • ExtractControlVariableName - (Optional) For incremental loads only, the name to assign the Extract Control variable in State Config for the ExtractControl value derived from the Extract Control Query above.

  • ExtractControlVariableSeedValue - (Optional) The initial value to set for the Extract Control variable in State Config - this will have no impact beyond the original seeding of the Extract Control variable in State Config.

  • DIUsToUseForCopyActivity - (Optional) Specifies the powerfulness of the copy executor. Value can be between 2 and 256. When left at default, the Data Factory dynamically applies the optimal DIU setting based on the source-sink pair and data pattern.

  • MaximumNumberOfAttemptsAllowed - (Optional) The total number of times the running of this Task can be attempted.

  • MinutesToWaitBeforeNextAttempt - (Optional) If a Task run fails, the number of minutes to wait before re-attempting the Task.

  • IsFederated - (Optional) Makes task available to other Insight Factories within this organisation.

  • Links - (Optional) NULL