VIDA-NYU/domain_discovery_tool

No such file or directory for server/elastic/config.json

aecio opened this issue · 5 comments

aecio commented

DDT fails to initialize due to IOError.

Environment variable DD_API_HOME is already set:

$ echo $DD_API_HOME
~/workspace/domain_discovery_API

but error still occurs:

$~/workspace/domain_discovery_tool_react$ source activate ddt

(ddt) $~/workspace/domain_discovery_tool_react$ ./bin/ddt-dev 
DDT_HOME  : ~/workspace/domain_discovery_tool_react
NLTK_DATA : ~/workspace/domain_discovery_tool_react/nltk_data
ACHE_HOME : ~/.anaconda2/envs/ddt/bin/../lib/ache/
PYTHONPATH: ~/workspace/domain_discovery_API/../:~/workspace/domain_discovery_API:~/workspace/domain_discovery_tool_react:~/workspace/domain_discovery_tool_react/server:
ELASTICSEARCH_SERVER  http://localhost:9200/
ELASTICSEARCH_USER  
Traceback (most recent call last):
  File "~/workspace/domain_discovery_tool_react/server/server.py", line 53, in <module>
    server = DDTServer()
  File "~/workspace/domain_discovery_tool_react/server/server.py", line 33, in __init__
    self._ddtModel = DomainModel()
  File "~/workspace/domain_discovery_API/models/domain_discovery_model.py", line 73, in __init__
    create_config_index()
  File "~/workspace/domain_discovery_tool_react/server/elastic/create_index.py", line 58, in create_config_index
IOError: [Errno 2] No such file or directory: '~/workspace/domain_discovery_tool_react/server/elastic/config.json'
yamsgithub commented

Can you try giving the whole path and not ~/workspace.

aecio commented

I actually gave it the whole path, I just replaced my home directory by ~/ before posting on github.

yamsgithub commented

Ah...looks like you still have the DD API folders in your domain_discovery_tool_react. Do you still have folders models, elastic etc in your DDT server folder?

aecio commented

I had them:

$ ls -lh server/
total 44K
-rw-rw-r-- 1 aeciosantos aeciosantos  297 Feb 22 16:07 config.conf
-rw-rw-r-- 1 aeciosantos aeciosantos  230 Feb 22 15:51 config.conf-in
-rw-rw-r-- 1 aeciosantos aeciosantos  11K Feb 22 16:09 crawler_model_adapter.pyc
drwxrwxr-x 2 aeciosantos aeciosantos 4.0K Apr 13 16:18 elastic
-rw-rw-r-- 1 aeciosantos aeciosantos    0 Feb 22 15:51 __init__.py
drwxrwxr-x 2 aeciosantos aeciosantos 4.0K Apr 13 13:23 models
drwxrwxr-x 2 aeciosantos aeciosantos 4.0K Apr 13 13:23 online_classifier
drwxrwxr-x 2 aeciosantos aeciosantos 4.0K Apr 13 13:23 ranking
drwxrwxr-x 3 aeciosantos aeciosantos 4.0K Apr 13 13:23 seeds_generator
-rw-rw-r-- 1 aeciosantos aeciosantos 2.5K Apr 13 13:23 server.py

But they had only the binaries, not source code:

$ ls server/models/
total 64K
-rw-rw-r-- 1 aeciosantos aeciosantos 57K Feb 22 16:09 crawlermodel.pyc
-rw-rw-r-- 1 aeciosantos aeciosantos 168 Feb 22 16:09 __init__.pyc

I removed everything as in a new clone of the master branch, and ran make again and it apparently worked.

But I got this "index_not_found_exception" error below using ES 1.5. What ES version are you using?

[13/Apr/2017:17:17:30] HTTP 
Request Headers:
  Content-Length: 9
  REFERER: http://0.0.0.0:8084/
  HOST: 0.0.0.0:8084
  ORIGIN: http://0.0.0.0:8084
  CONNECTION: keep-alive
  Remote-Addr: 127.0.0.1
  ACCEPT: */*
  USER-AGENT: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36
  X-REQUESTED-WITH: XMLHttpRequest
  ACCEPT-LANGUAGE: en-US,en;q=0.8,pt-BR;q=0.6,pt;q=0.4
  Content-Type: application/x-www-form-urlencoded; charset=UTF-8
  ACCEPT-ENCODING: gzip, deflate
[13/Apr/2017:17:17:30] HTTP 
Traceback (most recent call last):
  File "/home/aeciosantos/.anaconda2/envs/ddt/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 670, in respond
    response.body = self.handler()
  File "/home/aeciosantos/.anaconda2/envs/ddt/lib/python2.7/site-packages/cherrypy/lib/encoding.py", line 220, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/home/aeciosantos/.anaconda2/envs/ddt/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 60, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/home/aeciosantos/workspace/domain_discovery_API/server.py", line 63, in getAvailableDomains
    res = self._model.getAvailableDomains()
  File "/home/aeciosantos/workspace/domain_discovery_API/models/domain_discovery_model.py", line 106, in getAvailableDomains
    self._domains = get_available_domains(self._es)
  File "/home/aeciosantos/workspace/domain_discovery_API/elastic/get_config.py", line 16, in get_available_domains
    size=100
  File "/home/aeciosantos/.anaconda2/envs/ddt/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/home/aeciosantos/.anaconda2/envs/ddt/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 625, in search
    doc_type, '_search'), params=params, body=body)
  File "/home/aeciosantos/.anaconda2/envs/ddt/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/home/aeciosantos/.anaconda2/envs/ddt/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 128, in perform_request
    self._raise_error(response.status, raw_data)
  File "/home/aeciosantos/.anaconda2/envs/ddt/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 124, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
NotFoundError: TransportError(404, u'index_not_found_exception', u'no such index')
127.0.0.1 - - [13/Apr/2017:17:17:30] "POST /getAvailableDomains HTTP/1.1" 500 2927 "http://0.0.0.0:8084/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
yamsgithub commented