This actor checks results from Apify's scrapers or any other actor that stores its result to a dataset, and sends a notification if there are errors. It's designed to run from webhook.
You can set up webhook for Actor or Task with URL:
https://api.apify.com/v2/acts/drobnikj~check-crawler-results/runs?token=APIFY_API_TOKEN
Then you need to set up payload template for webhook data like:
{
"actId": {{resource.actId}},
"runId": {{resource.id}},
"options": {
"notifyTo": "jakub.drobnik@apify.com",
"minOutputtedPages": 10
}
}
You can call it from other Actor, for example:
await Apify.call('drobnikj/check-crawler-results', {
actId: 's7Jj8ik07gfV',
runId: 'sd86hGfHk0Uh6gF',
options: {
minOutputtedPages: 1000,
}
});
For a specific crawler set the following parameters:
https://api.apify.com/v2/acts/drobnikj~check-crawler-results/runs?token=APIFY_API_TOKEN
You can set up fields from options to finish webhook data.
- String
- Act ID you want to check
- String
- Run ID of actor you want to check
- String
- Dataset ID
- Object
- Number
- Number of results that actor checks
- Default is 100000
- Number
- Indicates minimum outputted items to checks.
- Object
- If jsonSchema is set actor check all sample results against schema.
- Boolean
- If compareWithPreviousExecution is set to
true
actor compare results with a previous execution. - If
tag
for execution is set compare actor result from previous results with the same tag. - It works only for the legacy crawler.
- String
- Mail where actor send notification if found error
- Object
- If actor found errors runs this actor.
- Example:
{
"id": "apify/send-mail",
"input": {
"to": "jakub.drobnik@apify.com",
"subject": "test on success",
"text": "No errors in crawler Amazon"
}
}
NOTE: If you didn't set input
, it set from input of main actor and errors output.
- Object
- If didn't find any errors runs this actor.
- Same format as
runActOnSuccess
All found errors will be in the errors
field.
{
"errors": [
"Run is not in SUCCEEDED status, act status: ABORTED",
"Crawler returns only 0 outputted pages and minumum is 100"
],
"executionAttrs": []
}