Skip to main content

v6.0 Release Notes

Here's what's new in version 6.0.

New Features​

Blueprints​

Blueprints are a feature within Insight Factory that can rapidly speed up the creation of tasks and production lines. There are two flavours of blueprints: one for bulk-creating tasks based on a CSV or JSON file, and another for creating a production line based on a pre-existing workflow.

Pulse Health and Optimiser are General Availability​

These features will help you monitor the daily operations of your Insight Factory and fine-tune its performance month over month.

Markdown Help Text​

We have added Markdown to Help and Description metadata, enhancing the viewing experience.

Deployments Refresh​

The deployment workflow and user experience have been refreshed based on user feedback and are significantly improved. Highlights:

Progress Bar​

A new header visually shows which stage a deployment is at in the context of the whole process.

Task Metadata Deployment is Now Optional​

Users can now deploy notebooks without deploying the metadata for a task.

Additional Information in Metadata​

We now display much more information about what will be deployed.

Improved Layout​

The layout of the Deployments page has been improved to support more screen resolutions.

Settings​

A new Settings area is available for administrators to update factory-wide settings.

Several configuration entries better suited to Factory Settings have been moved to the Settings screens.

Unicode Support​

Prior to version 6.0, users were unable to import data from a data source with a Unicode name or call a notebook with a Unicode name. Version 6.0 brings Unicode support to Insight Factory, including deployments.

Emojis work too 🚀

All Codes Are Now Uppercase​

To avoid issues with case sensitivity, all codes have been converted to uppercase. The UI already enforced uppercase, but users of the API could create non-uppercase codes. The API has been updated to enforce uppercase.

Removal of Generic Data to Delta Activity​

The generic Data to Delta Activity has been replaced by these four activities specific to the type of file they are processing. This allows for a tailored set of options for each type and declutters the user interface significantly.

  • Convert Parquet File to Delta Table
  • Convert JSON File to Delta Table
  • Convert Delimited File to Delta Table
  • Convert XML File to Delta Table

In-app Help​

We are working to progressively introduce context-aware help directly into the development experience.

New Orchestrator​

Our Orchestrator has been vastly improved and updated to .NET 8. This allows for more efficient operation and improved error handling.

Data Lake Storage Optimisation​

Improvements in how we capture history, manage storage tiers, and control the input folders in the Lakehouse will lead to faster run times and cheaper storage costs.

New Activities​

Machine Learning Activities​

  • Build ML Model
  • Run Inference on ML Model
  • Create/Update Feature Store Feature Table
  • Update ML Feature Log

Ingest​

  • Convert XML File to Delta Table

Decommissioned Features​

Ability to Move Nodes on Production Line Graph​

While useful to some, this feature caused more problems than it solved. We are looking to replace this capability with the grouping of tasks into expanding and collapsing boxes within a Production Line.

'Create Downstream as Delta' via Production Line Graph​

As Data to Delta has been replaced by any one of four other more specific activities, this capability needs a rethink. We hope that Blueprints can be extended into this space in a future release.