Flows

Flows are configurations for delivering data to customers.

Listing Flows

Flows can be listed by clicking the "Flows" menu item on the project sidebar.

You can filter and sort your Flows.

Creating a data flow

You can create a data flow by clicking the plus icon in the toolbar of the data flow list page.

When settings up a data flow you set:

  • a name

  • a cron expression for when to start the collection (in UTC)

  • a number of hours within which to aim to collect the data

  • a number of hours within which to aim to push the data

  • a number of hours after which to consider the delviery closed

  • a definition of what the data flow consists of

  • a SLA definition:

    • % of the data to be collected in window for each source

    • % of the sources available at start to be pushed in window (that have a push configuration - sources in development or on hold not counted)

At the moment, a data flow can consist of a single collection.

It is on the roadmap to bring chaining into workbench as a particular type of data flow.

We create a delivery object in the app automatically when the data flow starts. This tracks the completion and quality of the delivery overall.

Currently the schedules need to be set within app.import.io. It is planned for the starting of data collection to be done within workbench based upon this configuration.

When snapshots are either started or imported, we automatically link the snapshot to an active delivery if there is one applicable.

If there are flows set up on a project, we will NOT automatically push data unless it is part of a flow.

When the data flow should be closed, we move the delivery to a CLOSED state, and any pending data collection is cancelled.

You can mark a Data Flow as inactive to remove it from action.

Deleting Flows

Flows cannot currently be deleted.