Enable Requester Pays (HTML)

Overview

Customers wanting to access HTML files after a crawl run’s output has been pushed can enable a setting for each DOC Project that provides direct access to HTML files in S3 using a “Requester Pays” header. In order to opt into this functionality, relevant projects will need to update their settings and ensure their AWS account is set up to have access to the appropriate S3 bucket.

Project Settings (API)

For now, UI support does not exist for creating or updating settings for a project. Instead, settings will need to be created via the DOC API for each project requiring this setting.

API users will need:

cURL

curl -XPOST -H 'Authorization: UserToken {your DOC User Token}' -H "Content-type: application/json" -d '{ "data": { "requesterPaysHtml": true } }' 'https://workbench.import.io/api/orgs/{orgSlug}/projects/{projectId}/settings'

API Client

Method Endpoint Body

POST

https://workbench.import.io/api/orgs/{orgSlug}/projects/{projectId}/settings

{ "data": { "requesterPaysHtml": true } }

Accessing HTML Files via Requester Pays

Must provide x-amz-request-payer header. More information can be found here.