netlify/netlify-plugin-gatsby

[Bug]: Function Body "Unsupported content-type"

vrish88 opened this issue · 2 comments

vrish88 commented

Summary

I'd like to use Netlify functions as a proxy to an ElasticSearch deployment. ElasticSeach uses the application/x-ndjson content type for its multi-query endpoint. I'm getting an error when my frontend makes a request to the function:

Error parsing body Error: Unsupported content-type: application/x-ndjson

Behavior wise, it appears that the request comes into the function with no body set, and so when it proxies it forward it sends no body to ElasticSearch.

I've enabled Custom Body Parsing through gatsby's internal mechanism and so the function works as expected when running the app with gatsby develop.

Note:

  • I also had to install co-body into my project's dev dependencies in order to get past this error: 
    ◈ Rewrote URL to /.netlify/functions/__api
Request from ::ffff:127.0.0.1: POST /.netlify/functions/__api
{"level":"error","message":"End - Error:"}
{"errorMessage":"Cannot find module 'co-body'\nRequire stack:\n- /Users/mlavrisha/workspace/gatsby-netlify-function-unknown-content-type/.netlify/functions-internal/__api/gatsbyFunction.js\n- /Users/mlavrisha/workspace/gatsby-netlify-function-unknown-content-type/.netlify/functions-internal/__api/__api.js

Steps to reproduce

  1. Build the app usingnpx netlify build
  2. Run the app using npx netlify dev
  3. Run the following curl command 
    curl -XPOST -H "Content-Type: application/x-ndjson" localhost:8888/api/body -d "yes"

Actual Behavior

The curl command returned: But my body is telling me undefined

Expected Behavior

The curl command should return: But my body is telling me yes

A link to a reproduction repository

https://github.com/vrish88/gatsby-netlify-function-unknown-content-type

Plugin version

3.2.4

More information about your build

  • I am building using the CLI
  • I am building using file-based configuration (netlify.toml)

What OS are you using?

OS X

Your netlify.toml file

`netlify.toml` 
# Paste content of your `netlify.toml` file here

Configuration

`gatsby-config.js` and options 
# Paste content of your `gatsby-config.js` file, and/or command line options here. Check there is no private info in there.

Environment

Environment 
  System:
    OS: macOS 12.3.1
    CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
    Memory: 153.19 MB / 32.00 GB
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 16.13.2 - /usr/local/opt/node@16/bin/node
    Yarn: 1.22.17 - /usr/local/bin/yarn
    npm: 8.1.2 - /usr/local/opt/node@16/bin/npm
  Browsers:
    Chrome: 103.0.5060.53
    Firefox: 101.0.1
    Safari: 15.4

Gatsby info

gatsby info 
  System:
    OS: macOS 12.3.1
    CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 16.13.2 - /usr/local/opt/node@16/bin/node
    Yarn: 1.22.17 - /usr/local/bin/yarn
    npm: 8.1.2 - /usr/local/opt/node@16/bin/npm
  Browsers:
    Chrome: 103.0.5060.53
    Firefox: 101.0.1
    Safari: 15.4
  npmPackages:
    gatsby: ^4.16.0 => 4.16.0
    gatsby-plugin-google-gtag: ^4.14.0 => 4.16.0
    gatsby-plugin-netlify: ^5.0.0 => 5.0.0
    gatsby-plugin-react-helmet: ^5.14.0 => 5.16.0
    gatsby-plugin-remove-fingerprints: 0.0.2 => 0.0.2
    gatsby-plugin-sass: ^5.14.0 => 5.16.0
    gatsby-plugin-umami: ^0.1.3 => 0.1.3
    gatsby-remark-autolink-headers: ^5.14.0 => 5.16.0
    gatsby-source-filesystem: ^4.14.0 => 4.16.0
    gatsby-transformer-remark: ^5.14.0 => 5.16.0

Your _redirects file

`_redirects` 
# @netlify/plugin-gatsby redirects start
/api/* /.netlify/functions/__api 200
# @netlify/plugin-gatsby redirects end

Builds logs (or link to your logs)

Build logs 
────────────────────────────────────────────────────────────────
  Netlify Build
────────────────────────────────────────────────────────────────

❯ Version
  @netlify/build 27.3.2

❯ Flags
  dry: false
  offline: false

❯ Current directory
workspace/digital-library

❯ Config file
workspace/digital-library/netlify.toml

❯ Context
  production

❯ Loading plugins
   - @netlify/plugin-gatsby@3.2.4 from Netlify app

────────────────────────────────────────────────────────────────
  1. @netlify/plugin-gatsby (onPreBuild event)
────────────────────────────────────────────────────────────────

Found a Gatsby cache. We’re about to go FAST. ⚡️

(@netlify/plugin-gatsby onPreBuild completed in 8.8s)

────────────────────────────────────────────────────────────────
  2. Build command from Netlify app
────────────────────────────────────────────────────────────────

> gatsby build

success compile gatsby files - 0.479s
success load gatsby config - 0.069s
success load plugins - 1.052s
success onPreInit - 0.003s
success delete worker cache from previous builds - 0.003s
success initialize cache - 0.031s
success copy gatsby files - 0.206s
success Compiling Gatsby Functions - 0.962s
success onPreBootstrap - 0.981s
success createSchemaCustomization - 0.009s
success Checking for changed pages - 0.004s
success source and transform nodes - 7.848s
info Writing GraphQL type definitions to workspace/digital-library/.cache/schema.gql
success building schema - 1.452s
success createPages - 2.598s
success createPagesStatefully - 0.059s
info Total nodes: 15552, SitePage nodes: 6036 (use --verbose for breakdown)
success Checking for changed pages - 0.032s
success Cleaning up stale page-data - 0.187s
success onPreExtractQueries - 0.002s
success extract queries from components - 0.290s
success write out redirect data - 0.004s
success onPostBootstrap - 0.001s
info bootstrap finished - 20.615s
success write out requires - 0.041s
success Building production JavaScript and CSS bundles - 5.605s
⠙ Building Rendering Engines
<w> [webpack.cache.PackFileCacheStrategy] Serializing big strings (3370kiB) impacts deserialization performance (consider using Buffer instead and decode when needed)
<w> [webpack.cache.PackFileCacheStrategy] Serializing big strings (3370kiB) impacts deserialization performance (consider using Buffer instead and decode when needed)
<w> [webpack.cache.PackFileCacheStrategy] Serializing big strings (3370kiB) impacts deserialization performanc
success Building Rendering Engines - 4.113s
success Building HTML renderer - 2.541s
success Execute page configs - 0.033s
success Validating Rendering Engines - 1.701s
success Caching Webpack compilations - 0.001s
success run queries in workers - 1.307s - 67/67 51.26/s
success Merge worker state - 0.003s
success Rewriting compilation hashes - 0.002s
success Building static HTML for pages - 7.978s - 6036/6036 756.58/s
warn [gatsby-plugin-netlify] Your site has 6036 pages, which means that the generated headers file could
become very large. Consider disabling "mergeCachingHeaders" in your plugin config
info [gatsby-plugin-netlify] Creating SSR/DSG redirects...
info [gatsby-plugin-netlify] Created 0 SSR/DSG redirects...
success onPostBuild - 0.017s
info Done building in 45.339532414 sec


Pages

┌ src/templates/work.js
├ src/pages/404.js
│ ├   /404/
│ └   /404.html
├ src/pages/about.js
│ └   /about/
├ src/pages/index.js
│ └   /
├ src/pages/search.js
│ └   /search/
└ src/api/search/[...].js
  └ λ /api/search/[...]

  ╭────────────────────────────────────────────────────────────────╮
  │                                                                │
  │   (SSG) Generated at build time                                │
  │ D (DSG) Deferred static generation - page generated at runtime │
  │ ∞ (SSR) Server-side renders at runtime (uses getServerData)    │
  │ λ (Function) Gatsby function                                   │
  │                                                                │
  ╰────────────────────────────────────────────────────────────────╯


(build.command completed in 1m 8.2s)

────────────────────────────────────────────────────────────────
  3. @netlify/plugin-gatsby (onBuild event)
────────────────────────────────────────────────────────────────

Enabling Gatsby API support

(@netlify/plugin-gatsby onBuild completed in 445ms)

────────────────────────────────────────────────────────────────
  4. Functions bundling
────────────────────────────────────────────────────────────────

Packaging Functions from .netlify/functions-internal directory:
 - __api/__api.js


(Functions bundling completed in 1.2s)

────────────────────────────────────────────────────────────────
  5. @netlify/plugin-gatsby (onPostBuild event)
────────────────────────────────────────────────────────────────

Enabling Gatsby API support

(@netlify/plugin-gatsby onPostBuild completed in 14.1s)

────────────────────────────────────────────────────────────────
  6. @netlify/plugin-gatsby (onSuccess event)
────────────────────────────────────────────────────────────────


(@netlify/plugin-gatsby onSuccess completed in 6ms)

────────────────────────────────────────────────────────────────
  Summary
────────────────────────────────────────────────────────────────

❯ @netlify/plugin-gatsby: Essential Gatsby Build Plugin ran successfully
  Stored the Gatsby cache to speed up future builds. 🔥

────────────────────────────────────────────────────────────────
  Netlify Build Complete
────────────────────────────────────────────────────────────────

(Netlify Build completed in 1m 34.5s)

Function logs

Function logs 
Executed function "/api/search/[...]" in 14ms
Request from ::ffff:127.0.0.1: POST /.netlify/functions/__api
Error parsing body Error: Unsupported content-type: application/x-ndjson
    at module.exports (workspace/digital-library/node_modules/co-body/lib/any.js:48:15)
    at gatsbyFunction (workspace/digital-library/.netlify/functions-internal/__api/gatsbyFunction.js:31:52)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  status: 415
} Readable {
  _readableState: ReadableState {
    objectMode: false,
    highWaterMark: 16384,
    buffer: BufferList { head: [Object], tail: [Object], length: 1 },
    length: 249,
    pipes: [],
    flowing: null,
    ended: true,
    endEmitted: false,
    reading: false,
    constructed: true,
    sync: true,
    needReadable: false,
    emittedReadable: true,
    readableListening: false,
    resumeScheduled: false,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: true,
    destroyed: false,
    errored: null,
    closed: false,
    closeEmitted: false,
    defaultEncoding: 'utf8',
    awaitDrainWriters: null,
    multiAwaitDrain: false,
    readingMore: true,
    dataEmitted: false,
    decoder: null,
    encoding: null,
    [Symbol(kPaused)]: null
  },
  _events: [Object: null prototype] {},
  _eventsCount: 0,
  _maxListeners: undefined,
  setTimeout: [Function: setTimeout],
  _read: [Function: _read],
  _destroy: [Function: _destroy],
  _addHeaderLines: [Function: _addHeaderLines],
  _addHeaderLine: [Function: _addHeaderLine],
  _dump: [Function: _dump],
  url: '/api/search/searchPage/_msearch',
  originalUrl: '/api/search/searchPage/_msearch',
  query: {},
  multiValueQuery: {},
  method: 'POST',
  rawHeaders: [
    'x-forwarded-for',
    '::ffff:127.0.0.1',
    'accept-language',
    'en-US,en;q=0.9',
    'accept-encoding',
    'gzip',
    'referer',
    'http://localhost:8888/search',
    'sec-fetch-dest',
    'empty',
    'sec-fetch-mode',
    'cors',
    'sec-fetch-site',
    'same-origin',
    'origin',
    'http://localhost:8888',
    'sec-ch-ua-platform',
    '"macOS"',
    'accept',
    'application/json',
    'content-type',
    'application/x-ndjson',
    'user-agent',
    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
    'sec-ch-ua-mobile',
    '?0',
    'sec-ch-ua',
    '".Not/A)Brand";v="99", "Google Chrome";v="103", "Chromium";v="103"',
    'content-length',
    '249',
    'proxy-connection',
    'keep-alive',
    'host',
    'localhost:8888',
    'connection',
    'close',
    'client-ip',
    '127.0.0.1'
  ],
  headers: {
    'x-forwarded-for': '::ffff:127.0.0.1',
    cookie: '__cypress.initial=true',
    'accept-language': 'en-US,en;q=0.9',
    'accept-encoding': 'gzip',
    referer: 'http://localhost:8888/search',
    'sec-fetch-dest': 'empty',
    'sec-fetch-mode': 'cors',
    'sec-fetch-site': 'same-origin',
    origin: 'http://localhost:8888',
    'sec-ch-ua-platform': '"macOS"',
    accept: 'application/json',
    'content-type': 'application/x-ndjson',
    'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
    'sec-ch-ua-mobile': '?0',
    'sec-ch-ua': '".Not/A)Brand";v="99", "Google Chrome";v="103", "Chromium";v="103"',
    'content-length': '249',
    'proxy-connection': 'keep-alive',
    host: 'localhost:8888',
    connection: 'close',
    'client-ip': '127.0.0.1'
  },
  netlifyFunctionParams: {
    event: {
      path: '/api/search/searchPage/_msearch',
      httpMethod: 'POST',
      queryStringParameters: {},
      multiValueQueryStringParameters: {},
      headers: [Object],
      multiValueHeaders: [Object],
      body: '{"preference":"SearchResult"}\n' +
        '{"query":{"match_all":{}},"highlight":{"pre_tags":["<mark>"],"post_tags":["</mark>"],"type":"unified","fragment_size":0,"fields":{"content":{},"title":{}}},"size":10,"_source":{"includes":["*"],"excludes":[]},"from":0}\n',
      isBase64Encoded: false,
      rawUrl: 'http://localhost:8888/api/search/searchPage/_msearch',
      rawQuery: ''
    },
    context: {
      done: [Function: bound ],
      fail: [Function: bound ],
      succeed: [Function: bound ],
      getRemainingTimeInMillis: [Function: bound ],
      callbackWaitsForEmptyEventLoop: false,
      functionName: 'handler',
      functionVersion: '1.0',
      invokedFunctionArn: 'arn:aws:lambda:us-east-1:764032913920:function:handler:1.0',
      memoryLimitInMB: '125',
      awsRequestId: '5370bf08-adad-7d22-4520-c1146a7703fa',
      logGroupName: 'Group name',
      logStreamName: 'Stream name',
      identity: null,
      clientContext: {},
      _stopped: false
    }
  },
  getHeader: [Function (anonymous)],
  getHeaders: [Function (anonymous)],
  cookies: { '__cypress.initial': 'true' },
  [Symbol(kCapture)]: false
}
Running search/[...]
Executed function "/api/search/[...]" in 15ms
vrish88 commented

FWIW it appears that this error comes from this section of code.

sarahetter commented

Closing as stale, please re-open if relevant