Snapshot Retry Options

In the event that a snapshot import fails, the JQ tranform for custom files changes or fails, or the runtime configuration of a snapshot changes, there could be a need to retry certain stages of a snapshot’s lifecycle.

The four options provided to handle these scenarios are:

  • Regenerate Custom Assets

  • Re-import Snapshot

  • Re-run Snapshot

  • Re-extract Snapshot

The above options can be found by clicking the ellipsis on the top-right corner of a snapshot’s home page. These options are only available in certain circumstances, relying on a snapshot’s status and other factors. More information about each option can be found below.

Regenerate Custom Assets

Generate Assets is the final stage of the import pipeline for a snapshot. If the snapshot’s collection has any linked destinations with a custom file type selected (custom text or custom parquet), they will be created during this stage.

In the event that the JQ transform associated with a snapshot fails for a custom file, the snapshot will fail to import (ie. status will be FAILED) with a relevant error message. If a snapshot fails because of the "Generate Assets" stage, this stage can be retried with the "Regenerate Custom Assets" option. This will only retry the "Generate Assets" stage. If the regeneration is successful, a snapshot can continue to QA to be pushed.

If a snapshot fails because of a JQ transform error, the JQ transform should be revised before trying to regenerate assets.

Re-import

In the event that a snapshot fails to import prior to the "Generate Assets" stage, a snapshot can be re-imported. A re-import can be used if there are schema incompatibilities between the extractor’s schema and DOC’s schema. Additionally, a re-import can be tried if the import fails for any other reason i(ie. a system failure or timeout).

Re-importing a snapshot will put the original snapshot into a status of SUPERCEDED and a new snapshot will be created using the crawl run and a new import.

Re-run

If an entirely new crawl run is necessary without a revision to an extractor’s runtime configuration, an option is available to re-run. The original snapshot will be moved to a status of SUPERCEDED and a new snapshot will be created, along with a new crawl run.

Re-extract

If an extractor needs to be updated before being re-imported, a snapshot can be re-extracted after the extractor is updated. The original snapshot will be SUPERCEDED and a new snapshot will be created to capture the re-extraction. The "Re-extract" stage will be added to the new snapshot’s import pipeline.

If a snapshot is SUPERCEDED, its new snapshot will have a link to the original visible on a snapshot’s home page.