Skip to main content

Using Webhooks

What are Webhooks?

Webhooks are a means for an application to communicate with web applications in real-time. Webhooks work by sending a POST request to a provided URL at a predetermined point in time. This request?s body typically contains some information of interest.

Within, Webhooks connect your extractors to external applications via a POST request made whenever a crawl run succeeds. This POST request?s body contains information such as the extractor ID, total URL count, and links to the extractor?s outputs, including JSON, CSV, and Excel files.

You should use Webhooks when you have an external application that consumes them. Webhooks can be utilized for efficient retrieval and storage, fully eliminating the need to poll for results.

How do I use Webhooks?

Step 1: Add a Webhook. A Webhook is composed of a URL and optional headers. These will later be used to send a POST request whose body contains your extractor?s output data.

Set up a webhook under the extractor?s settings.

Step 2: Run your extractor. From now on, whenever your extractor completes a crawl run successfully, it will POST to the URL provided.

View your sample output object in the extractor settings.