mallocator/Elasticsearch-Exporter

v2 export 'Caught exception in Main process: Error: ENOENT'

lighta opened this issue · 2 comments

lighta commented

So quick setup (here)
OS = Clean ubuntu/trusty64 (vagrant)
install =
git clone https://github.com/mallocator/Elasticsearch-Exporter.git
cd Elasticsearch-Exporter/
git checkout v2.0.0
npm install colors
npm install through
npm install async
npm install util
(I can provide a vagrant file if you interested for test)

1 checking we have some data =
root@imgramDashboard:/opt/asci/Elasticsearch-Exporter# curl 'localhost:9200/_cat/indices?v'
health status index pri rep docs.count docs.deleted store.size pri.store.size
yellow open crashs 5 1 64286 0 5.9mb 5.9mb
yellow open .kibana 1 1 1 0 2.5kb 2.5kb
yellow open pruns 5 1 96921 0 20.5mb 20.5mb
yellow open events 5 1 827 1485 656.1kb 656.1kb
yellow open tests 5 1 3621057 617 476.3mb 476.3mb

Now the tests case =

  1. root@imgramDashboard:/opt/asci/Elasticsearch-Exporter# nodejs exporter.js -t file -sh localhost -si crashs -st crash -tf crashs.dump
    Elasticsearch Exporter - Version 2.0.0
    Caught exception in Main process: Error: ENOENT, open 'crashs.dump.meta'
    Error: ENOENT, open 'crashs.dump.meta'
    Exit code 2: Uncaught Exception
    Fetched Entries: 0 documents
    Processed Entries: 0 documents
    Source DB Size: 0 documents
  2. root@imgramDashboard:/opt/asci/Elasticsearch-Exporter# touch crashs.dump.meta
  3. root@imgramDashboard:/opt/asci/Elasticsearch-Exporter# nodejs exporter.js -t file -sh localhost -si crashs -st crash -tf crashs.dump
    Elasticsearch Exporter - Version 2.0.0
    Caught exception in Main process: SyntaxError: Unexpected end of input
    SyntaxError: Unexpected end of input
    at Object.parse (native)
    at /opt/asci/Elasticsearch-Exporter/drivers/file.driver.js:72:21
    at fs.js:268:14
    at Object.oncomplete (fs.js:107:15)
    Exit code 2: Uncaught Exception
    Fetched Entries: 0 documents
    Processed Entries: 0 documents
    Source DB Size: 0 documents
  4. root@imgramDashboard:/opt/asci/Elasticsearch-Exporter# cp /vagrant/save/events.dump.meta $BASE_PATH/repos/dumps/
  5. root@imgramDashboard:/opt/asci/Elasticsearch-Exporter# nodejs exporter.js -t file -sh localhost -si crashs -st crash -tf $BASE_PATH/repos/dumps/events.dump
    Elasticsearch Exporter - Version 2.0.0
    The driver reported an error: { [Error: socket hang up] code: 'ECONNRESET', sslError: undefined }
    Exit code 4: driver is passing on an error
    Fetched Entries: 0 documents
    Processed Entries: 0 documents
    Source DB Size: 0 documents

Desc/Analysys=

  1. The default command to export data, we want to dump 'crashs/crash' into crash.dump
    => Error he seem to want / not create the meta file (mapping)
  2. Test to fix, create an empty meta file for him.
    => Error doesn't detect the meta is invalid (not a clear message)
  3. Load a valid meta file. (actually created from v1)
    => Error want ssl by default..

So quite simple, the export option doesn't produce the .meta file anymore but seem to rely on it. To continue. So ye you could said we don't produce the meta if you didn't ask for it anymore but then it shouldn't prevent the data dump.

mallocator commented

The file driver so far is only a copy of the old driver and hasn't been adapted to the new project layout. But there will still be two files in the end, even though the data file will not be backwards compatible.

mallocator commented

The file driver is now implemented, but it now works completely differently so that there's actually a directory structure with multiple files each for a type in an index.