AWS SDK v3 traces display as http.request

[automartin5000]

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

1. Send request to AWS Lambda API that calls downstream AWS service
2. Trace doesn't show AWS service name

Specifications

• Datadog Lambda Layer version: 6.87.0
• Node version: 18

Stacktrace

N/A

Other Notes

Not sure if this is related, but our serverless Node 18 -> Node 18 APIs aren't being traced correctly

[astuyve]

Hi @automartin5000 - thanks for reaching out.

I'd be really interested to see your handler function and configuration, as I attempted to replicate this with sns and sqs and am able to see the libraries traced entirely:

I think it may be best if you open a support ticket so you can share more information (function code, config, etc) securely?

Thanks!

[automartin5000]

I'm fine opening a support ticket, but I'm also fine to anonymize any info and post it here. Seems like it could help others. What do you think would work best?

[automartin5000]

Also, not sure if this is relevant, but the 2 Lambda Functions in question are running NestJS with serverless-express

[automartin5000]

And I just looked, our handler is pretty stock, so I'm happy to post it here:

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import helmet from 'helmet';
import { INestApplication, ValidationPipe } from '@nestjs/common';
import serverlessExpress from '@vendia/serverless-express';
import { Callback, Context, Handler } from 'aws-lambda';
import { WinstonModule } from 'nest-winston';
import { getWinstonLoggerInstance } from './winston-logging';

async function serverlessBootstrap(): Promise<Handler> {
  const app = await NestFactory.create(AppModule, {
    logger: WinstonModule.createLogger(getWinstonLoggerInstance()),
  });
  app.use(helmet());
  configureValidationPipe(app);
  await app.init();

  const expressApp = app.getHttpAdapter().getInstance();
  return serverlessExpress({ app: expressApp });
}

let server: Handler;

export const lambdaHandler: Handler = async (
  event: any,
  context: Context,
  callback: Callback,
) => {
  server = server ?? (await serverlessBootstrap());
  return server(event, context, callback);
};

export const configureValidationPipe = (app: INestApplication) => {
  app.useGlobalPipes(
    new ValidationPipe({
      whitelist: true,
    }),
  );
};

[astuyve]

Thanks @automartin5000!

Can you share the code which interacts with patched libraries like aws-sdk?

[automartin5000]

Thanks @automartin5000!

Can you share the code which interacts with patched libraries like aws-sdk?

Sure! Dynamo example:

const marshallOptions = {
  removeUndefinedValues: true,
};

const dynamoOptions = getDynamoOptions(
  process.env.DYNAMO_ENDPOINT,
  process.env.AWS_DEFAULT_REGION,
);

const dynamoClient = new DynamoDBClient(dynamoOptions);
const dynamodb = DynamoDBDocument.from(dynamoClient, { marshallOptions });

const params = {
  TableName: tableName,
  Key: {
    hash: value,
  },
};
const response = await dynamodb.get(params);

[astuyve]

Ah okay - unless you are transpiling your import statements away into require statements, you'll need to utilize our custom loader in order for dd-trace to automatically trace modules.

This is still considered experimental in node, which is why we do not officially support this yet.
You may try setting NODE_OPTIONS='--loader dd-trace/loader-hook.mjs' as per this comment.

If this doesn't work, or you targeting an es version which uses require, please let me know and we can go from there.

Thanks!

[automartin5000]

I probably should have mentioned that we're using Webpack as well 😅 I'm guessing that has an impact based on what you're saying? Would you want to see that config?

[astuyve]

Ohhhhhh yes that does change things. Are you marking aws-sdk as external? Docs are here

[automartin5000]

Bummer, I thought that one was gonna work, but no dice. Same thing. Also, the Webpack config docs in your link don't mention aws-sdk, only in the Esbuild section. Maybe an opportunity to update the docs 😄 Here's our updated webpack config:

module.exports = (options, webpack) => {
    const lazyImports = [
      '@nestjs/microservices/microservices-module',
      '@nestjs/websockets/socket-module',
    ];
  
    return {
      ...options,
      externals: ["dd-trace", "datadog-lambda-js", "aws-sdk"],
        output: {
          ...options.output,
            libraryTarget: 'commonjs2',
          },
      plugins: [
        ...options.plugins,
        new webpack.IgnorePlugin({
          checkResource(resource) {
            if (lazyImports.includes(resource)) {
              try {
                require.resolve(resource);
              } catch (err) {
                return true;
              }
            }
            return false;
          },
        }),
      ],
    };
  };

[automartin5000]

@astuyve Solved with marking individual packages as external as describe here. Would note the additional noise in my follow up comment over there. I still am having service-to-service tracing issues. But I can close this one and open a new ticket for that. Thanks for the help!