neo4j-contrib/neo4j_doc_manager

Unicode support in python 2.7 - UnicodeEncodeError during insert

Opened this issue · 0 comments

2015-10-08 15:10:24,110 [CRITICAL] mongo_connector.oplog_manager:543 - Exception during collection dump
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/mongo_connector/oplog_manager.py", line 495, in do_dump
    upsert_all(dm)
  File "/Library/Python/2.7/site-packages/mongo_connector/oplog_manager.py", line 479, in upsert_all
    dm.bulk_upsert(docs_to_dump(namespace), mapped_ns, long_ts)
  File "/Users/lyonwj/neotechnology/devtmp/neo4j_doc_manager/mongo_connector/doc_managers/neo4j_doc_manager.py", line 78, in bulk_upsert
    builder = NodesAndRelationshipsBuilder(doc, doc_type, doc_id)
  File "/Users/lyonwj/neotechnology/devtmp/neo4j_doc_manager/mongo_connector/doc_managers/nodes_and_relationships_builder.py", line 17, in __init__
    self.build_nodes_query(doc_type, doc, doc_id)
  File "/Users/lyonwj/neotechnology/devtmp/neo4j_doc_manager/mongo_connector/doc_managers/nodes_and_relationships_builder.py", line 30, in build_nodes_query
    self.build_nodes_query(key, document[key], id)
  File "/Users/lyonwj/neotechnology/devtmp/neo4j_doc_manager/mongo_connector/doc_managers/nodes_and_relationships_builder.py", line 29, in build_nodes_query
    self.build_relationships_query(doc_type, key, id, id)
  File "/Users/lyonwj/neotechnology/devtmp/neo4j_doc_manager/mongo_connector/doc_managers/nodes_and_relationships_builder.py", line 82, in build_relationships_query
    statement = "MATCH (a:`{main_type}`), (b:`{node_type}`) WHERE a._id={{doc_id}} AND b._id ={{explicit_id}} CREATE (a)-[r:`{relationship_type}`]->(b)".format(main_type=main_type, node_type=node_type, relationship_type=relationship_type)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xeb' in position 16: ordinal not in range(128)

Unicode handling is built-in in python 3.4 so this error only occurs with Python 2.7.

To reproduce, using Python 2.7 with mongo connector running use mongoimport to import this collection dump