Skip to main content

Ingest SAP CDC (Incremental Load Only) as Parquet

Copy data from SAP CDC (Incremental Load only) to parquet format in Azure Data Lake Storage Gen2.

To use this activity within the API, use an ActivityCode of SAP-CDC-INCREMENTAL-ADLS.

Example JSON​

An example of what the Task Config would look like for a task using this activity. Some of these variables would be set at the group level to avoid duplication between tasks.

NULL

Variable Reference​

The following variables are supported:

  • SourceConnection - (Required) Source connection to use.

  • OdpContext - (Required) The context of the ODP data extraction.

  • OdpName - (Required) The name of the data source object to extract.

  • SapCheckpointKey - (Required) Checkpoint Key used for this activity's CDC workflow. WARNING - this must be unique for each dataset.

  • KeyColumns - (Required) Comma-separated string of the key columns of the dataset to extract.

  • TargetConnection - (Optional) Target connection to use.

  • DataLakeSystemFolder - (Required) Name of the folder in the Data Lake that acts as the parent folder for all datasets belonging to this System.

  • DataLakeDatasetFolder - (Required) Name of the folder in the Data Lake that the dataset will be stored under. Used with 'Data Lake System Folder' to form the fully qualified path to the dataset within the data Container in the Data Lake.

  • ElevateToDelta - (Optional) Ingest directly to Lakehouse Table

  • DeltaSchemaName - (Optional) The name of the Schema this transformation lives in. Required if Copy to Lakehouse Table is enabled.

  • DeltaTableName - (Optional) The name of the Lakehouse Table representing this transformation. Required if Copy to Lakehouse Table is enabled.

  • SizeOfSparkCompute - (Optional) The size of compute used in the Spark cluster.

  • IntegrationRuntime - (Required) Integration Runtime to use for Copy Activity.

  • MaximumNumberOfAttemptsAllowed - (Optional) The total number of times the running of this Task can be attempted.

  • MinutesToWaitBeforeNextAttempt - (Optional) If a Task run fails, the number of minutes to wait before re-attempting the Task.

  • RetainHistory - (Optional) Should the raw files be saved to the History Container to preserve them?

  • IsFederated - (Optional) Makes task available to other Insight Factories within this organisation.

  • Links - (Optional) NULL