Skip to main content

Crawlrun information



Crawlrun information


Path Parameters

    crawlrunId uuidrequired

Query Parameters

    _apikey stringrequired

    You can find your API key in the dashboard under User Settings


Crawlrun information retrieved successfully


    guid uuid

    Unique identifier for the crawl run.



    [internal] The metadata of the object

    timestamp int64required

    The timestamp of the last change to the object

    creationTimestamp int64

    The timestamp of the object's creation

    patchTimestamp int64

    The timestamp of the last patch to the object

    objectGuid uuid

    The identifier of the object

    lastEditorGuid uuid

    The identifier of the last editor of the object

    ownerGuid uuid

    The identifier of the owner of the object

    creatorGuid uuid

    The identifier of the creator of the object

    orgGuid uuid

    The identifier of the organization the object belongs to

    deleted boolean

    The flag indicating if the object is deleted

    runtimeConfigId uuid

    [internal] Identifier for the runtime configuration used during the crawl run.

    extractorId uuid

    Identifier of the extractor that was executed.

    policyId uuid

    [internal] Policy used by this run. If null, the default policy is used.

    previousRunId uuid

    Chained crawl run that this run is based on. If null, this is the first run.

    startedAt date-time

    Timestamp marking the start of the crawl run, in milliseconds since Unix epoch.

    stoppedAt date-time

    Timestamp marking the end of the crawl run, in milliseconds since Unix epoch.

    inProgress int32

    Total number of urls in progress.

    requeued int32

    Total number of urls requeued.

    totalUrlCount int32

    Total number of urls to process.

    successUrlCount int32

    Number of successful urls processed process so far.

    failedUrlCount int32

    Number of failed urls processed so far.

    rowCount int32

    Number of rows returned so far.

    screenCaptureCount int32

    Number of screen captures taken.

    htmlExtractionCount int32

    Number of html extractions extracted.

    deniedByRobotsCount int32deprecated

    Number of urls denied by robots.txt.

    redactedRowCount int32deprecated

    Number of rows with redacted PII.

    queryCount int32

    Number of queries we used to process this run.

    proxyUsage int64

    Amount of data in bytes transferred via premium proxy during this run.

    state string


    State of this run, eg. STARTED, FINISHED, FAILED.

    urlListId uuid

    [internal] Id of Extractor urlList attachment when CrawlRun created

    inputsId uuid

    [internal] Id of Extractor inputs attachment when CrawlRun created

    triggerEvent string

    Possible values: [SCHEDULED, ADHOC]

    [internal] Event that triggered this run

    errorType string

    An enumerated error type for a failed run.

    errorMessage string

    Error message for a failed run.

    json uuid

    [internal] Attachment containing json of the rows.

    csv uuiddeprecated

    [internal] Attachment containing csv of the rows.

    xlsx uuiddeprecated

    [internal] Attachment containing xlsx of the rows.

    log uuiddeprecated

    [internal] Attachment containing log of the url results processed.

    diffId uuid

    [internal] Attachment containing new-line delimited json detailing the a diff between this run and the previous.

    sample uuiddeprecated

    [internal] Attachment containing json array of sample of rows for previewing the data.

    files uuid

    [internal] Attachment containing a zip file that contains assets downloaded as part of the crawl.



    A summary of the downloaded assets for this crawl.

    totalFiles int64

    The number of files downloaded

    totalSizeBytes int64

    The total size of the files downloaded in bytes

    crawlRunInputsId uuid

    [internal] An attachment for the line separated JSON of inputs for this crawl run. Utilized in the API case, where inputs change run to run.

    crawlRunUrlListId uuid

    [internal] An attachment for line separated list of urls for this crawl run. Utilized in the API case, where url lists change run to run.

    crawlRunStats uuid

    [internal] An attachment for quality metrics for the crawl run.



    Webhooks to call when run completes, overrides any extractor settings.

  • Array [

  • url stringrequired

    The url of the webhook



    The headers to use for webhook notifications

    property name* object

    The headers to use for webhook notifications

    payload string

    The pre-configured payload to send with each notification

  • ]

  • jsonFile string


    csvFile string


    xlsxFile string


    logFile string


    sampleFile string


    archive string


    inputUri string


    notificationUri string