DataDog/datadog-lambda-js

Lambda reports error "failed to flush metrics" with no more detail.

rolfwessels opened this issue · 1 comments

Expected Behavior

I have a lamda function that makes and async http call. I expect the http response data to be sent to data dog as metric value.

const https = require("https");
const { datadog, sendDistributionMetric } = require("datadog-lambda-js");

const domain = "dev.domain";
const url = `https://${domain}/getMetric/`;

function send(desc, value) {
  console.log(`${desc}: ${value}`);
  sendDistributionMetric(desc, value);
}

const prefix = `lambda.${domain.replace(".", "_")}`;

function execHandler(event) {
  const promise = new Promise(function (resolve, reject) {
    https
      .get(url, (res) => {
        var body = "";
        res.on("data", function (chunk) {
          body += chunk;
        });
        res.on("end", function () {
          const result = JSON.parse(body);

          send(`${prefix}.status_code`, parseInt(res.statusCode));
          send(`${prefix}.value`, parseInt(result.val));
          resolve(res.statusCode);
        });
      })
      .on("error", (e) => {
        send(`${prefix}.request_failed`, 1);
        reject(Error(e));
      });
  });
  return promise;
}

module.exports.execHandler = datadog(execHandler);

exports.handler = datadog(execHandler);

Actual Behavior

Lambda runs but ends with failed to flush

Steps to Reproduce the Problem

  1. Add script above to lamda
  2. Set lambda layer to "arn:aws:lambda:eu-central-1:464622532012:layer:Datadog-Node12-x:43"
  3. Run

Specifications

  • Datadog Lambda Layer version: 43
  • Node version: Node12

Stacktrace

2021-01-25T12:34:47.273Z	044bb758-93eb-4721-8016-44462c29ec07	INFO	[dd.trace_id=702867017315363974 dd.span_id=7792049542437625856] lambda.dev_domain.status_code: 200
2021-01-25T12:34:47.273Z	044bb758-93eb-4721-8016-44462c29ec07	INFO	[dd.trace_id=702867017315363974 dd.span_id=7792049542437625856] lambda.dev_domain.value: 47
2021-01-25T12:34:47.656Z	044bb758-93eb-4721-8016-44462c29ec07	ERROR	[dd.trace_id=702867017315363974 dd.span_id=7792049542437625856] {"innerError":{},"status":"error","message":"datadog:failed to flush metrics"}

I re-did my lambda from scratch following the instructions here.

https://docs.datadoghq.com/serverless/installation/nodejs?tab=custom
Pay attention to these

  • Set your image’s CMD value to node_modules/datadog-lambda-js/dist/handler.handler. You can either set this directly in your Dockerfile or override the value using AWS.
  • Set the environment variable DD_LAMBDA_HANDLER to your original handler, for example, myfunc.handler.

I think the handler was not setup correctly and that is why it was failing. This is no longer failing but I still get no logs into data-dog. Back to the drawing board.