defog update - Out of range float values are not JSON compliant
TutajITeraz opened this issue · 4 comments
Hello,
When executing getting-started instruction:
https://docs.defog.ai/getting-started
on simple 2 tables.
defog_metadata.csv file looks fine
but an error occurred:
$ defog update defog_metadata.csv
Connection details found. Reading connection details from file...
Connection details read from /home/lukasz/.defog/connection.json.
Traceback (most recent call last):
File "/home/lukasz/.local/lib/python3.11/site-packages/requests/models.py", line 511, in prepare_body
body = complexjson.dumps(json, allow_nan=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/json/__init__.py", line 238, in dumps
**kw).encode(obj)
^^^^^^^^^^^
File "/usr/lib/python3.11/json/encoder.py", line 200, in encode
chunks = self.iterencode(o, _one_shot=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/json/encoder.py", line 258, in iterencode
return _iterencode(o, 0)
^^^^^^^^^^^^^^^^^
ValueError: Out of range float values are not JSON compliant
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/lukasz/.local/bin/defog", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/lukasz/.local/lib/python3.11/site-packages/defog/cli.py", line 45, in main
update()
File "/home/lukasz/.local/lib/python3.11/site-packages/defog/cli.py", line 289, in update
resp = df.update_db_schema_csv(filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lukasz/.local/lib/python3.11/site-packages/defog/__init__.py", line 1016, in update_db_schema_csv
r = requests.post(
^^^^^^^^^^^^^^
File "/home/lukasz/.local/lib/python3.11/site-packages/requests/api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lukasz/.local/lib/python3.11/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lukasz/.local/lib/python3.11/site-packages/requests/sessions.py", line 575, in request
prep = self.prepare_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lukasz/.local/lib/python3.11/site-packages/requests/sessions.py", line 486, in prepare_request
p.prepare(
File "/home/lukasz/.local/lib/python3.11/site-packages/requests/models.py", line 371, in prepare
self.prepare_body(data, files, json)
File "/home/lukasz/.local/lib/python3.11/site-packages/requests/models.py", line 513, in prepare_body
raise InvalidJSONError(ve, request=self)
requests.exceptions.InvalidJSONError: Out of range float values are not JSON compliant
Hi @TutajITeraz , could you share a minimal/sample CSV file to reproduce this error? My guess based on the limited info here is that it might be due to NaN's in your CSV. If so you can replace those values with some other value.
defog_metadata.csv
I have cut out some rows from this file, and still have the same an error.
If i delete the last row - then it works. So if you mean that NaN's are just empty values, then is it a problem that defog init did not generate them? Or is it a problem with my database? Or is this a normal procedure to fix the column manually?
Thanks for reporting, this is due to empty values for column_description in the last row. We have fixed our library to handle empty values in PR #19
$ defog update defog_metadata.csv
Connection details found. Reading connection details from file...
Connection details read from /Users/jp/.defog/connection.json.
Your schema has been updated. You're ready to start querying!
You can pull the repository, navigate to the root folder and install the latest version with pip install -e .
Closing as issue is fixed