fluent/fluent-plugin-sql

Outputs SQL with RETURNING

dobesv opened this issue · 2 comments

For some reason the database is complaining that the sql output is adding RETURNING "id" to the end of the INSERT statements, e.g.

2020-08-04T17:45:32.406110387Z  2020-08-04 17:45:32.406 UTC [44840] STATEMENT:  INSERT INTO "events" ("timestamp","assignmentId","event","formativeId","category","label","name","sectionId","type","urlPattern","route","userId","studentId","value","selectedTab","questionNumber","viewPortHeight","viewPortWidth","pilotUser","partnerUser","premiumUser","student","teacher","utm_campaign","utm_content","utm_medium","utm_source","browserVersion","operatingSystemVersion","browser","operatingSystem","googleClientId","anonymousId","experimentId","method") VALUES ('2020-08-04 17:45:09',NULL,...snip...) RETURNING "id"
2020-08-04T17:45:32.408085777Z  2020-08-04 17:45:32.407 UTC [44840] ERROR:  column "id" does not exist at character 786

There's no id mentioned in the mapping:

<match **>
    @id yugabyte
    @type sql
    @log_level info

    adapter postgresql
    host "#{ENV['YBHOST']}"
    port "#{ENV['YBPORT']}"
    username "#{ENV['YBUSER']}"
    password "#{ENV['YBPASSWORD']}"
    database "#{ENV['YBDATABASE']}"

    <inject>
      time_key timestamp
      time_type string
      time_format %FT%T%.%N:z
      utc true
    </inject>

    <table>
      table events
      column_mapping 'timestamp,assignmentId,event,formativeId,category,label,name,sectionId,type,urlPattern,route,userId,studentId,value,selectedTab,questionNumber,viewPortHeight,viewPortWidth,pilotUser,partnerUser,premiumUser,student,teacher,utm_campaign,utm_content,utm_medium,utm_source,browserVersion,operatingSystemVersion,browser,operatingSystem,googleClientId,anonymousId,experimentId,method'
    </table>
  </match>

Does anyone know why it is doing this and what might prevent it? Maybe I need to generate an id field?

I added a database trigger to auto-fill the id column as a workaround for this. If this is expected behavior it probably should be documented.

Hi @dobesv , did you not get into any issue with fluentd after installing the plugin? I have td-agent running and the service fails to start after I installed the plugin.