Create a Task

Tasks are individual units of work. Each task uses an activity to define what it does—like ingesting data from a database or running a Databricks notebook.

Tasks can run in sequence or parallel based on their dependencies. Each task logs execution time, status, and errors.

Creating Tasks

Tasks can be created from the Tasks page or from within a Production Line:

1. Open your Production Line's Graph view
2. Click the + button in the side menu
3. Enter a Name
4. Select an Activity that matches what you want to do
5. Click on Save to end your editing session and save the task
6. Otherwise, click on Expand Task if you want to continue editing
7. Configure the task properties
8. Click Save

Bulk Creation

Need to create many similar tasks? Use the API with Python or PowerShell to automate task creation for scenarios like extracting all tables from a database.

Task Configuration

Each task has configuration properties determined by its activity. These can be set at:

• Task level — Specific to this task
• Task Group level — Shared across related tasks (shown with a 'G' icon)
• Activity level — Shared across all tasks belonging to an activity (shown with an 'A' icon)

See the Activity Reference for required properties per activity.

Cloning Tasks

Right-click a task and select Clone to create a duplicate. This is useful when you need to create similar tasks with slight variations—saving time and effort.

Task Dependencies

Tasks can depend on other tasks, controlling execution order. Dependencies define when a downstream task runs based on the result of upstream tasks.

Task flows display left to right in the graph view.

Setting Dependencies

Hover over a task node in the graph to reveal three colored handles on the right side:

• Green — Downstream task runs on success
• Red — Downstream task runs on failure
• Blue — Downstream task runs on completion (regardless of outcome)

Drag from the appropriate handle to create the dependency. You can also configure dependencies in the task Properties tab in the Workflow section.

See Workflows for details on dependency conditions and AND/OR logic.

Cross-production line dependencies

You can set dependencies on tasks from other production lines. Display these by selecting Show External in the graph toolbar.

What's Next?

• Workflows — Control when tasks run with dependency conditions
• Ingesting Data from a Database — Common ingestion pattern
• Ingesting Data from Files — CSV, Excel, JSON files
• Enriching Data with Databricks — Transform your data